You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Zoltan Borok-Nagy (Code Review)" <ge...@cloudera.org> on 2020/12/14 14:24:48 UTC
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16873
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
For MAP types field id resolution indexes the top-level columns
via the current 'table_idx - 1'. In this case table_idx is either
SchemaPathConstants::MAP_KEY or SchemaPathConstants::MAP_VALUE which
are 0 and 1 respectively. Hence 'table_idx - 1' can be -1 which is not
a valid index for a vector, hence we get an ASAN error. Even if
'table_idx - 1' is zero we get a wrong field id.
Note that at this point in the schema resolution we have successfully
found a MAP type with a matching field id, therefore it is safe to
resolve the child via the value of 'table_idx' (which is the position
of the child, MAP_KEY or MAP_VALUE).
Testing:
* Built impala with ASAN (buildall.sh -notests -skiptests -asan), then
executed test_iceberg_query
Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
---
M be/src/exec/parquet/parquet-metadata-utils.cc
M tests/query_test/test_scanners.py
2 files changed, 3 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/73/16873/1
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "wangsheng (Code Review)" <ge...@cloudera.org>.
wangsheng has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 1: Code-Review+1
Thanks for fixed this bug, Zoltan!
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Dec 2020 02:35:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 2:
Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/6778/ DRY_RUN=false
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Dec 2020 11:58:58 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/7851/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Mon, 14 Dec 2020 14:47:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 2: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Dec 2020 11:58:57 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 2: Verified+1
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 2
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Dec 2020 17:31:42 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
For MAP types field id resolution indexes the top-level columns
via the current 'table_idx - 1'. In this case table_idx is either
SchemaPathConstants::MAP_KEY or SchemaPathConstants::MAP_VALUE which
are 0 and 1 respectively. Hence 'table_idx - 1' can be -1 which is not
a valid index for a vector, hence we get an ASAN error. Even if
'table_idx - 1' is zero we get a wrong field id.
Note that at this point in the schema resolution we have successfully
found a MAP type with a matching field id, therefore it is safe to
resolve the child via the value of 'table_idx' (which is the position
of the child, MAP_KEY or MAP_VALUE).
Testing:
* Built impala with ASAN (buildall.sh -notests -skiptests -asan), then
executed test_iceberg_query
Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Reviewed-on: http://gerrit.cloudera.org:8080/16873
Reviewed-by: Impala Public Jenkins <im...@cloudera.com>
Tested-by: Impala Public Jenkins <im...@cloudera.com>
---
M be/src/exec/parquet/parquet-metadata-utils.cc
M tests/query_test/test_scanners.py
2 files changed, 3 insertions(+), 13 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 3
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
[Impala-ASF-CR] IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
Posted by "Gabor Kaszab (Code Review)" <ge...@cloudera.org>.
Gabor Kaszab has posted comments on this change. ( http://gerrit.cloudera.org:8080/16873 )
Change subject: IMPALA-10393: Iceberg field id-based column resolution fails in ASAN builds
......................................................................
Patch Set 1: Code-Review+2
--
To view, visit http://gerrit.cloudera.org:8080/16873
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I41e8daaebe8a6024716e6c22f6ccd819f43508bd
Gerrit-Change-Number: 16873
Gerrit-PatchSet: 1
Gerrit-Owner: Zoltan Borok-Nagy <bo...@cloudera.com>
Gerrit-Reviewer: Gabor Kaszab <ga...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: wangsheng <sk...@163.com>
Gerrit-Comment-Date: Tue, 15 Dec 2020 11:37:56 +0000
Gerrit-HasComments: No