You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@impala.apache.org by "Aman Sinha (Code Review)" <ge...@cloudera.org> on 2022/04/16 22:32:38 UTC

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Aman Sinha has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18421


Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................

IMPALA-11247: Test script changes for materialized views

IMPALA-10723 added support for treating materialized views as tables.
In certain test configurations, the rebuild of the materialized views
(which is done via Hive) was not populating the data in the MV. In
this patch, I have changed the source tables of materialized views
to be full-acid instead of insert-only transactional tables. This
enables the tests to succeed. Insert-only source tables are also
meant to work for the MV rebuild but that is a Hive issue that will
be investigated separately.

Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
---
M testdata/datasets/functional/functional_schema_template.sql
1 file changed, 10 insertions(+), 6 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18421/1
-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 1
Gerrit-Owner: Aman Sinha <am...@cloudera.com>

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 4:

> Patch Set 3: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8051/

1 test failed in this because I had missed updating the AuthorizationStmtTest.java after removing the alltypes_transactional source table.  This is fixed in patchset 4.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 4
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 14:51:56 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18421

to look at the new patch set (#2).

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................

IMPALA-11247: Test script changes for materialized views

IMPALA-10723 added support for treating materialized views as tables.
In certain test configurations, the rebuild of the materialized views
(which is done via Hive) was not populating the data in the MV. In
this patch, I have changed the source tables of materialized views
to be full-acid instead of insert-only transactional tables. This
enables the tests to succeed. Insert-only source tables are also
meant to work for the MV rebuild but that is a Hive issue that will
be investigated separately.

Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
---
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
2 files changed, 8 insertions(+), 53 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18421/2
-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 2
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 5:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8052/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 5
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 14:54:17 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 5:

> Patch Set 5: Verified-1
> 
> Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8052/

1 test failure in query_test.test_acid.TestAcid.test_acid_compute_stats which is a flaky test previously reported in https://issues.apache.org/jira/browse/IMPALA-11160.

One small change still needs to be done: the compute-table-stats.sh still has alltypes_transactional and jointbl_transactional which were removed in the last patch set.  I will update the compute-table-stats.sh


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 5
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 21:09:49 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 1:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10457/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 1
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sat, 16 Apr 2022 22:53:01 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 4:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10459/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 4
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 15:09:48 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................

IMPALA-11247: Test script changes for materialized views

IMPALA-10723 added support for treating materialized views as tables.
In certain test configurations, the rebuild of the materialized views
(which is done via Hive) was not populating the data in the MV. In
this patch, I have changed the source tables of materialized views
to be full-acid instead of insert-only transactional tables. This
enables the tests to succeed. Insert-only source tables are also
meant to work for the MV rebuild but that is a Hive issue that will
be investigated separately.

Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Reviewed-on: http://gerrit.cloudera.org:8080/18421
Reviewed-by: Aman Sinha <am...@cloudera.com>
Tested-by: Aman Sinha <am...@cloudera.com>
---
M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java
M testdata/bin/compute-table-stats.sh
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
4 files changed, 12 insertions(+), 58 deletions(-)

Approvals:
  Aman Sinha: Looks good to me, approved; Verified

-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 8
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 7: Verified+1 Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 7
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 22:36:39 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 2:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10458/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 2
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 02:42:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18421

to look at the new patch set (#6).

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................

IMPALA-11247: Test script changes for materialized views

IMPALA-10723 added support for treating materialized views as tables.
In certain test configurations, the rebuild of the materialized views
(which is done via Hive) was not populating the data in the MV. In
this patch, I have changed the source tables of materialized views
to be full-acid instead of insert-only transactional tables. This
enables the tests to succeed. Insert-only source tables are also
meant to work for the MV rebuild but that is a Hive issue that will
be investigated separately.

Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
---
M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java
M testdata/bin/compute-table-stats.sh
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
4 files changed, 12 insertions(+), 58 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18421/6
-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 6
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 3: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8051/


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 3
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 11:27:10 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 3: Code-Review+2


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 3
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 04:23:41 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 3:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/8051/ DRY_RUN=false


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 3
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 07:03:36 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 2:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18421/1/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:

http://gerrit.cloudera.org:8080/#/c/18421/1/testdata/datasets/functional/functional_schema_template.sql@2562
PS1, Line 2562: INSERT OVERWRITE TABLE {db_name}{db_suffix}.{table_name} PARTITION (year, month)
> Does this work for parquet? Or should we remove these?
I see..I didn't know that the default mode for functional_orc_def.alltypes and functional_orc_def.jointbl is full-ACID.  It makes sense to use those. I have modified the MV definitions accordingly and removed the references. to alltypes_transactional and jointbl_transactional from both files.
Regarding Parquet, my original intent was to have source tables in both ORC and Parquet formats since I only needed insert-only transactional tables (no row level updates are being done to any of the source).  However, there is some problem doing an MV rebuild if the source table is insert-only especially when done through the test load script.  So,  for now let's restrict the test case to ORC full-ACID.



-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 2
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 02:35:46 +0000
Gerrit-HasComments: Yes

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 5: Verified-1

Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/8052/


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 5
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 19:16:00 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Hello Quanlong Huang, Wenzhe Zhou, Impala Public Jenkins, 

I'd like you to reexamine a change. Please visit

    http://gerrit.cloudera.org:8080/18421

to look at the new patch set (#4).

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................

IMPALA-11247: Test script changes for materialized views

IMPALA-10723 added support for treating materialized views as tables.
In certain test configurations, the rebuild of the materialized views
(which is done via Hive) was not populating the data in the MV. In
this patch, I have changed the source tables of materialized views
to be full-acid instead of insert-only transactional tables. This
enables the tests to succeed. Insert-only source tables are also
meant to work for the MV rebuild but that is a Hive issue that will
be investigated separately.

Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
---
M fe/src/test/java/org/apache/impala/authorization/AuthorizationStmtTest.java
M testdata/datasets/functional/functional_schema_template.sql
M testdata/datasets/functional/schema_constraints.csv
3 files changed, 11 insertions(+), 56 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/18421/4
-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 4
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Aman Sinha (Code Review)" <ge...@cloudera.org>.
Aman Sinha has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 6: Verified+1 Code-Review+2

Carry forward Quanlong's +2. 
Marking Verified since the one test failure is a known (flaky ?) issue IMPALA-11160.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 6
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 21:56:21 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Impala Public Jenkins (Code Review)" <ge...@cloudera.org>.
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 6:

Build Successful 

https://jenkins.impala.io/job/gerrit-code-review-checks/10460/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests.


-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 6
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 21:31:34 +0000
Gerrit-HasComments: No

[Impala-ASF-CR] IMPALA-11247: Test script changes for materialized views

Posted by "Quanlong Huang (Code Review)" <ge...@cloudera.org>.
Quanlong Huang has posted comments on this change. ( http://gerrit.cloudera.org:8080/18421 )

Change subject: IMPALA-11247: Test script changes for materialized views
......................................................................


Patch Set 1:

(1 comment)

http://gerrit.cloudera.org:8080/#/c/18421/1/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:

http://gerrit.cloudera.org:8080/#/c/18421/1/testdata/datasets/functional/functional_schema_template.sql@2562
PS1, Line 2562: TBLPROPERTIES ('transactional'='true');
Does this work for parquet? Or should we remove these?
https://github.com/apache/impala/blob/dfc2f175bdd10106e6984fc150ede9891a03eeb0/testdata/datasets/functional/schema_constraints.csv#L302-L303

If we only need ORC tables, the functional_orc_def.alltypes is already full-ACID. Can we use it directly?



-- 
To view, visit http://gerrit.cloudera.org:8080/18421
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I349faa0ad36ec8ca6f574f7f92d9a32fb7d0d344
Gerrit-Change-Number: 18421
Gerrit-PatchSet: 1
Gerrit-Owner: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <am...@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <im...@cloudera.com>
Gerrit-Reviewer: Quanlong Huang <hu...@gmail.com>
Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
Gerrit-Comment-Date: Sun, 17 Apr 2022 00:27:57 +0000
Gerrit-HasComments: Yes