You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Quanlong Huang (Code Review)" <ge...@cloudera.org> on 2022/11/03 03:30:34 UTC

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Quanlong Huang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19195


Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................

IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

FileSystemUtil has static code to get configuration from
BackendConfig.INSTANCE which could be null in some FE tests. In commit
c1610a163 of IMPALA-11469, we fixed the issue by modifying the failed FE
tests to extend FrontendTestBase which can make sure
BackendConfig.INSTANCE is initialized. However, AcidUtilsTest and
TestCaseLoaderTest also have the issue. But they are missed since the
issue depends on the test order. If a test that inits
BackendConfig.INSTANCE runs first, the following tests won't suffer this
issue.

To avoid new tests hitting this issue, this patch inits
BackendConfig.INSTANCE in need in the static code of FileSystemUtil.
Also adds a warning mentioning this should only happen in tests.

Tests:
 - Run FE tests one by one so each test won't depend on any previous
   env. Only found AcidUtilsTest and TestCaseLoaderTest have the issue.
   Verified this patch fixes the issue in these two tests.

Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
---
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
1 file changed, 6 insertions(+), 0 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/19195/1
-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Joe McDonnell (Code Review)" <ge...@cloudera.org>.
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 22:55:12 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java:

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@870
PS3, Line 870:       BackendConfig.create(new TBackendGflags());
Is there a property set only in tests we could assert on to ensure this is true?



-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 16:17:50 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 1:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG@11
PS1, Line 11: c1610a163
Does this new patch make c1610a163 redundant or do we still need the changes in it?


http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG@20
PS1, Line 20: in need
Maybe 'lazily' would be more understandable?



-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Thu, 03 Nov 2022 13:27:54 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Michael Smith (Code Review)" <ge...@cloudera.org>.
Michael Smith has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3: Code-Review+1

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java:

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@870
PS3, Line 870:       BackendConfig.create(new TBackendGflags());
> Good point! I think RuntimeEnv.INSTANCE.isTestEnv() is the most relevant on
Works for me.



-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 22:35:09 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11784/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 02:08:16 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11785/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 02:09:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Daniel Becker (Code Review)" <ge...@cloudera.org>.
Daniel Becker has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3: Code-Review+1

Thanks


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 13:06:32 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................

IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

FileSystemUtil has static code to get configuration from
BackendConfig.INSTANCE which could be null in some FE tests. In commit
c1610a163 of IMPALA-11469, we fixed the issue by modifying the failed FE
tests to extend FrontendTestBase which can make sure
BackendConfig.INSTANCE is initialized. However, AcidUtilsTest and
TestCaseLoaderTest also have the issue. But they are missed since the
issue depends on the test order. If a test that inits
BackendConfig.INSTANCE runs first, the following tests won't suffer this
issue.

To avoid new tests hitting this issue again, this patch inits
BackendConfig.INSTANCE lazily in the static code of FileSystemUtil.
Also adds a warning mentioning this should only happen in tests. Note
that in impalad and catalogd, BackendConfig.INSTANCE is initialized in
constructors of JniFrontend and JniCatalog.

The changes of c1610a163 is redundant after this patch so they are
reverted.

Tests:
 - Run FE tests one by one so each test won't depend on any previous
   env. Only found AcidUtilsTest and TestCaseLoaderTest have the issue.
   Verified this patch fixes the issue in these two tests.

Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Reviewed-on: http://gerrit.cloudera.org:8080/19195
Reviewed-by: Daniel Becker <da...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
Reviewed-by: Michael Smith <mi...@cloudera.com>
Reviewed-by: Joe McDonnell <jo...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java
3 files changed, 8 insertions(+), 4 deletions(-)

Approvals:
  Daniel Becker: Looks good to me, but someone else must approve
  Impala Public Jenkins: Verified
  Michael Smith: Looks good to me, but someone else must approve
  Joe McDonnell: Looks good to me, approved

-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 4
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <jo...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
File fe/src/main/java/org/apache/impala/common/FileSystemUtil.java:

http://gerrit.cloudera.org:8080/#/c/19195/3/fe/src/main/java/org/apache/impala/common/FileSystemUtil.java@870
PS3, Line 870:       BackendConfig.create(new TBackendGflags());
> Is there a property set only in tests we could assert on to ensure this is 
Good point! I think RuntimeEnv.INSTANCE.isTestEnv() is the most relevant one. But it's still not set in all tests (tracked in IMPALA-9780). What about adding the assertion in IMPALA-9780?



-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 22:31:20 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/11777/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 1
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Comment-Date: Thu, 03 Nov 2022 03:50:23 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19195

to look at the new patch set (#2).

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................

IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

FileSystemUtil has static code to get configuration from
BackendConfig.INSTANCE which could be null in some FE tests. In commit
c1610a163 of IMPALA-11469, we fixed the issue by modifying the failed FE
tests to extend FrontendTestBase which can make sure
BackendConfig.INSTANCE is initialized. However, AcidUtilsTest and
TestCaseLoaderTest also have the issue. But they are missed since the
issue depends on the test order. If a test that inits
BackendConfig.INSTANCE runs first, the following tests won't suffer this
issue.

To avoid new tests hitting this issue again, this patch inits
BackendConfig.INSTANCE lazily in the static code of FileSystemUtil.
Also adds a warning mentioning this should only happen in tests. Note
that in impalad and catalogd, BackendConfig.INSTANCE is initialized in
constructors of JniFrontend and JniCatalog.

The changes of c1610a163 is redundant after this patch so they are
reverted.

Tests:
 - Run FE tests one by one so each test won't depend on any previous
   env. Only found AcidUtilsTest and TestCaseLoaderTest have the issue.
   Verified this patch fixes the issue in these two tests.

Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
---
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java
3 files changed, 17 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/19195/2
-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 2
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3: Verified+1


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 16:28:59 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8774/ DRY_RUN=true


-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 11:19:53 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Hello Daniel Becker, Michael Smith, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/19195

to look at the new patch set (#3).

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................

IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

FileSystemUtil has static code to get configuration from
BackendConfig.INSTANCE which could be null in some FE tests. In commit
c1610a163 of IMPALA-11469, we fixed the issue by modifying the failed FE
tests to extend FrontendTestBase which can make sure
BackendConfig.INSTANCE is initialized. However, AcidUtilsTest and
TestCaseLoaderTest also have the issue. But they are missed since the
issue depends on the test order. If a test that inits
BackendConfig.INSTANCE runs first, the following tests won't suffer this
issue.

To avoid new tests hitting this issue again, this patch inits
BackendConfig.INSTANCE lazily in the static code of FileSystemUtil.
Also adds a warning mentioning this should only happen in tests. Note
that in impalad and catalogd, BackendConfig.INSTANCE is initialized in
constructors of JniFrontend and JniCatalog.

The changes of c1610a163 is redundant after this patch so they are
reverted.

Tests:
 - Run FE tests one by one so each test won't depend on any previous
   env. Only found AcidUtilsTest and TestCaseLoaderTest have the issue.
   Verified this patch fixes the issue in these two tests.

Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
---
M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java
M fe/src/test/java/org/apache/impala/catalog/FileMetadataLoaderTest.java
M fe/src/test/java/org/apache/impala/common/FileSystemUtilTest.java
3 files changed, 8 insertions(+), 4 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/95/19195/3
-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/19195 )

Change subject: IMPALA-11699: Fix NPE in FE tests thrown from static code of FileSystemUtil
......................................................................


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG
Commit Message:

http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG@11
PS1, Line 11: c1610a163
> Does this new patch make c1610a163 redundant or do we still need the change
Yeah, we can revert c1610a163 in this patch BTW.


http://gerrit.cloudera.org:8080/#/c/19195/1//COMMIT_MSG@20
PS1, Line 20: lazily 
> Maybe 'lazily' would be more understandable?
Done



-- 
To view, visit http://gerrit.cloudera.org:8080/19195
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I5c056791406cd4535a7e43889dbb73d153b06f0a
Gerrit-Change-Number: 19195
Gerrit-PatchSet: 3
Gerrit-Owner: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Daniel Becker <da...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Michael Smith <mi...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Fri, 04 Nov 2022 01:52:35 +0000
Gerrit-HasComments: Yes