You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org> on 2022/04/18 18:39:28 UTC
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Fang-Yu Rao has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18423
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
This patch preserves the behavior of concat_ws() before IMPALA-8891 by
introducing a query option of 'CONCAT_WS_RETURN_NULL_IF_ANY_IS_NULL',
which is set to false by default.
Specifically, when 'CONCAT_WS_RETURN_NULL_IF_ANY_IS_NULL' is set to
true, concat_ws() will return NULL as long as any of the argument values
is NULL, which corresponds to the behavior before IMPALA-8891.
Testing:
- Added backend tests to verifiy the behavior before IMPALA-8891 could
be preserved once the query option above is set to true.
Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
---
M be/src/exprs/expr-test.cc
M be/src/exprs/string-functions-ir.cc
M be/src/service/query-options.cc
M be/src/service/query-options.h
M common/thrift/ImpalaService.thrift
M common/thrift/Query.thrift
6 files changed, 39 insertions(+), 3 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/23/18423/1
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18423 )
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18423/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18423/1//COMMIT_MSG@10
PS1, Line 10: query option of 'CONCAT_WS_RETURN_NULL_IF_ANY_IS_NULL'
> My understanding is that generally skipping nulls is the expected behavior
I tend to the current query option approach since it can help users migrate queries to the new behavior. So this won't be an upgrade blocker.
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Apr 2022 01:27:01 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18423 )
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
Patch Set 1:
Hi all, please let me know if you have any comment or suggestion. Thank you very much for the help!
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 Apr 2022 18:40:11 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18423 )
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
Patch Set 1:
Build Successful
https://jenkins.impala.io/job/gerrit-code-review-checks/10462/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Comment-Date: Mon, 18 Apr 2022 18:59:14 +0000
Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Posted by "Csaba Ringhofer (Code Review)" <ge...@cloudera.org>.
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/18423 )
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
Patch Set 1: Code-Review+1
(1 comment)
http://gerrit.cloudera.org:8080/#/c/18423/1//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/18423/1//COMMIT_MSG@10
PS1, Line 10: query option of 'CONCAT_WS_RETURN_NULL_IF_ANY_IS_NULL'
My understanding is that generally skipping nulls is the expected behavior instead of returning NULL - checked Hive, postgresql, MsSql, MySql, and all behave this way.
So I don't expect our "old behavior" to be common request by users. To make the change simpler, we could simply use a flag instead.
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Comment-Date: Tue, 19 Apr 2022 16:08:15 +0000
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11237: Preserve the behavior of concat ws() before IMPALA-8891
Posted by "Fang-Yu Rao (Code Review)" <ge...@cloudera.org>.
Fang-Yu Rao has posted comments on this change. ( http://gerrit.cloudera.org:8080/18423 )
Change subject: IMPALA-11237: Preserve the behavior of concat_ws() before IMPALA-8891
......................................................................
Patch Set 1:
Thanks for the comments Csaba and Quanlong!
I also prepared another patch at https://gerrit.cloudera.org/c/18440/ that uses the approach suggested by Csaba.
It's not that easy to determine which approach is better.
For a user that relies on Impala's old behavior, it may be easier for the user to start the Impala daemons with 'concat_ws_return_null_if_any_is_null' set to true. No change has to be made to the user's SQL scripts with respect to the behavior of CONCAT_WS().
However, it may be easier for such a user to start adapting to the new behavior of CONCAT_WS() in that no restart of Impala daemons is required.
Let me know if I missed something. I think I am fine with both approaches.
--
To view, visit http://gerrit.cloudera.org:8080/18423
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I6fe11971e0c56b868bdf8bf1b071d40d9837acda
Gerrit-Change-Number: 18423
Gerrit-PatchSet: 1
Gerrit-Owner: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Csaba Ringhofer <cs...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fa...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Vincent Tran <vt...@cloudera.com>
Gerrit-Comment-Date: Fri, 22 Apr 2022 04:15:13 +0000
Gerrit-HasComments: No