You are viewing a plain text version of this content. The canonical link for it is here.

Posted to reviews@impala.apache.org by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org> on 2020/09/04 23:58:17 UTC

[Impala-ASF-CR] WIP IMPALA-10140: Fix CatalogExeception for creating database with sync ddl as true

Wenzhe Zhou has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16421

Change subject: WIP IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true
......................................................................

WIP IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true

IMPALA-7961 handle the cases for "creating table if not exist" and
sync_ddl as true. In ENGESC-3589, customer report similar issue which
happened for "create database if database not exist" and sync_ddl as
true. To fix the issue, add the similar fixing as the fixing for
IMPALA-7961 to CatalogOpExecutor.createDatabase() function.

Testing:
- TODO manual tests
Since this is a racy bug, I could only reproduce it by forcing
frequent topic update log GCs along with a specific sequence of
actions. Couldn't reproduce it with the patch.
- Passed exhaustive test.

Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
1 file changed, 27 insertions(+), 3 deletions(-)

git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16421/1
--
To view, visit http://gerrit.cloudera.org:8080/16421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Gerrit-Change-Number: 16421
Gerrit-PatchSet: 1
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>

[Impala-ASF-CR] IMPALA-10140: Fix CatalogExeception for creating database with sync ddl as true

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.

Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16421 )

Change subject: IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true
......................................................................

IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true

IMPALA-7961 handle the cases for query "create table if not exists"
with sync_ddl as true. Customers reported similar issue which happened
for query "create database if not exists" with sync_ddl as true.
This patch adds the similar fixing as the fixing for IMPALA-7961 to
function CatalogOpExecutor.createDatabase() to fix the issue.

Testing:
 - Manual tests
   Since this is a racy bug, I could only reproduce it by forcing
   frequent topicUpdateLog GCs along with a specific sequence of
   actions, like: run some DDLs and REFRESHs to trigger a GC in
   topicUpdateLog, then run query "create database if not exists" with
   sync_ddl as true. Verified that the issue couldn't be reproduced
   after applying this patch.
 - Passed exhaustive test.

Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Reviewed-on: http://gerrit.cloudera.org:8080/16421
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
1 file changed, 27 insertions(+), 3 deletions(-)

Approvals:
  Impala Public Jenkins: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/16421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Gerrit-Change-Number: 16421
Gerrit-PatchSet: 5
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10140: Fix CatalogExeception for creating database with sync ddl as true

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.

Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16421 )

Change subject: IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/16421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Gerrit-Change-Number: 16421
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Wed, 09 Sep 2020 01:38:05 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-10140: Fix CatalogExeception for creating database with sync ddl as true

Posted by "Wenzhe Zhou (Code Review)" <ge...@cloudera.org>.

Wenzhe Zhou has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/16421 )

Change subject: IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true
......................................................................

IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true

IMPALA-7961 handle the cases for query "create table if not exists"
with sync_ddl as true. Customers reported similar issue which happened
for query "create database if not exists" with sync_ddl as true.
This patch adds the similar fixing as the fixing for IMPALA-7961 to
function CatalogOpExecutor.createDatabase() to fix the issue.

Testing:
 - Manual tests
   Since this is a racy bug, I could only reproduce it by forcing
   frequent topicUpdateLog GCs along with a specific sequence of
   actions, like: run some DDLs and REFRESHs to trigger a GC in
   topicUpdateLog, then run query "create database if not exists" with
   sync_ddl as true. Verified that the issue couldn't be reproduced
   after applying this patch.
 - Passed exhaustive test.

Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
---
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
1 file changed, 27 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/16421/3
-- 
To view, visit http://gerrit.cloudera.org:8080/16421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Gerrit-Change-Number: 16421
Gerrit-PatchSet: 3
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-10140: Fix CatalogExeception for creating database with sync ddl as true

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.

Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/16421 )

Change subject: IMPALA-10140: Fix CatalogExeception for creating database with sync_ddl as true
......................................................................


Patch Set 2:

(1 comment)

LGTM, just have a minor comment.

http://gerrit.cloudera.org:8080/#/c/16421/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
File fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java:

http://gerrit.cloudera.org:8080/#/c/16421/2/fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java@1298
PS2, Line 1298:         if (syncDdl) {
Could you move the "syncDdl" check before acquiring the db lock? If syncDdl=false, we don't need the added logic here.



-- 
To view, visit http://gerrit.cloudera.org:8080/16421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Id623118f8938f416414c45d93404fb70d036a9df
Gerrit-Change-Number: 16421
Gerrit-PatchSet: 2
Gerrit-Owner: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Comment-Date: Tue, 08 Sep 2020 07:29:05 +0000
Gerrit-HasComments: Yes