You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2020/09/30 01:32:06 UTC
[GitHub] [phoenix] ChinmaySKulkarni opened a new pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
ChinmaySKulkarni opened a new pull request #903:
URL: https://github.com/apache/phoenix/pull/903
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-712339391
Thanks for the review @jpisaac , I will modify the test and commit after a successful preCommit
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-701106455
Please review @yanxinyi @jpisaac @gjacoby126 @kadirozde @twdsilva
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni merged pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni merged pull request #903:
URL: https://github.com/apache/phoenix/pull/903
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-708638034
Thanks for the review @twdsilva!
@kadirozde @gjacoby126 @jpisaac @yanxinyi as discussed offline, I have written a design doc outlining my approach: [PHOENIX-6142 Design Doc](https://docs.google.com/document/d/1ZWysagBG84yapT6bIyeKljC-2ydQSgtQ63jl7eiZnFs/edit#). Please take a look. It would be great to get your reviews as well. The push I did after Thomas's review does not contain any code changes. They are just extra comments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-709674292
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 30s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 26m 21s | 4.x passed |
| +1 :green_heart: | compile | 0m 54s | 4.x passed |
| +1 :green_heart: | checkstyle | 3m 2s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 43s | 4.x passed |
| +0 :ok: | spotbugs | 2m 50s | phoenix-core in 4.x has 956 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 22m 49s | the patch passed |
| +1 :green_heart: | compile | 0m 56s | the patch passed |
| +1 :green_heart: | javac | 0m 56s | the patch passed |
| -1 :x: | checkstyle | 3m 7s | phoenix-core: The patch generated 417 new + 5804 unchanged - 345 fixed = 6221 total (was 6149) |
| -1 :x: | whitespace | 0m 0s | The patch 1 line(s) with tabs. |
| -1 :x: | javadoc | 0m 43s | phoenix-core generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) |
| -1 :x: | spotbugs | 3m 6s | phoenix-core generated 1 new + 953 unchanged - 3 fixed = 954 total (was 956) |
||| _ Other Tests _ |
| -1 :x: | unit | 125m 23s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 38s | The patch does not generate ASF License warnings. |
| | | 193m 48s | |
| Reason | Tests |
|-------:|:------|
| FindBugs | module:phoenix-core |
| | Found reliance on default encoding in org.apache.phoenix.util.ViewUtil.findImmediateRelatedViews(Table, byte[], byte[], byte[], PTable$LinkType, long):in org.apache.phoenix.util.ViewUtil.findImmediateRelatedViews(Table, byte[], byte[], byte[], PTable$LinkType, long): String.getBytes() At ViewUtil.java:[line 287] |
| Failed junit tests | phoenix.end2end.PhoenixRuntimeIT |
| | phoenix.end2end.ToCharFunctionIT |
| | phoenix.end2end.DeleteIT |
| | phoenix.end2end.IndexToolForNonTxGlobalIndexIT |
| | phoenix.end2end.UpsertSelectIT |
| | phoenix.end2end.index.ImmutableIndexExtendedIT |
| | phoenix.end2end.DropSchemaIT |
| | phoenix.end2end.index.IndexMaintenanceIT |
| | phoenix.end2end.OrphanViewToolIT |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/903 |
| JIRA Issue | PHOENIX-6142 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux c35418f930ab 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 264310b |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/whitespace-tabs.txt |
| javadoc | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/diff-javadoc-javadoc-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/new-spotbugs-phoenix-core.html |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/testReport/ |
| Max. process+thread count | 6414 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/6/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-710732902
None of the tests that failed in the build above are failing locally.
Re-running the preCommit manually to get another run.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty removed a comment on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
stoty removed a comment on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-709493485
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 6s | 4.x passed |
| +1 :green_heart: | compile | 1m 3s | 4.x passed |
| +1 :green_heart: | checkstyle | 2m 30s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 51s | 4.x passed |
| +0 :ok: | spotbugs | 3m 23s | phoenix-core in 4.x has 956 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 1m 2s | root in the patch failed. |
| -1 :x: | compile | 0m 53s | phoenix-core in the patch failed. |
| -1 :x: | javac | 0m 53s | phoenix-core in the patch failed. |
| -1 :x: | checkstyle | 2m 37s | phoenix-core: The patch generated 459 new + 5762 unchanged - 387 fixed = 6221 total (was 6149) |
| -1 :x: | whitespace | 0m 0s | The patch 1 line(s) with tabs. |
| -1 :x: | javadoc | 0m 47s | phoenix-core generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) |
| -1 :x: | spotbugs | 0m 48s | phoenix-core in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 50s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 48m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/903 |
| JIRA Issue | PHOENIX-6142 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 724d19548194 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 264310b |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| javac | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/whitespace-tabs.txt |
| javadoc | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/diff-javadoc-javadoc-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-spotbugs-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/testReport/ |
| Max. process+thread count | 94 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-709493485
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 0m 32s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.x Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 6s | 4.x passed |
| +1 :green_heart: | compile | 1m 3s | 4.x passed |
| +1 :green_heart: | checkstyle | 2m 30s | 4.x passed |
| +1 :green_heart: | javadoc | 0m 51s | 4.x passed |
| +0 :ok: | spotbugs | 3m 23s | phoenix-core in 4.x has 956 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| -1 :x: | mvninstall | 1m 2s | root in the patch failed. |
| -1 :x: | compile | 0m 53s | phoenix-core in the patch failed. |
| -1 :x: | javac | 0m 53s | phoenix-core in the patch failed. |
| -1 :x: | checkstyle | 2m 37s | phoenix-core: The patch generated 459 new + 5762 unchanged - 387 fixed = 6221 total (was 6149) |
| -1 :x: | whitespace | 0m 0s | The patch 1 line(s) with tabs. |
| -1 :x: | javadoc | 0m 47s | phoenix-core generated 1 new + 99 unchanged - 1 fixed = 100 total (was 100) |
| -1 :x: | spotbugs | 0m 48s | phoenix-core in the patch failed. |
||| _ Other Tests _ |
| -1 :x: | unit | 0m 50s | phoenix-core in the patch failed. |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 48m 25s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.40 ServerAPI=1.40 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/903 |
| JIRA Issue | PHOENIX-6142 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux 724d19548194 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.x / 264310b |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| mvninstall | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-mvninstall-root.txt |
| compile | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| javac | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-compile-phoenix-core.txt |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| whitespace | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/whitespace-tabs.txt |
| javadoc | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/diff-javadoc-javadoc-phoenix-core.txt |
| spotbugs | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-spotbugs-phoenix-core.txt |
| unit | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/artifact/yetus-general-check/output/patch-unit-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/testReport/ |
| Max. process+thread count | 94 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-903/4/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] jpisaac commented on a change in pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
jpisaac commented on a change in pull request #903:
URL: https://github.com/apache/phoenix/pull/903#discussion_r507430114
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewMetadataIT.java
##########
@@ -294,7 +321,246 @@ public void testRecreateDroppedTableWithChildViews() throws Exception {
}
}
- private void runDropChildViewsTask() {
+ @Test
+ public void testAlterTableIsResilientToOrphanLinks() throws SQLException {
+ final String parent1TableName = generateUniqueName();
+ final String parent2TableName = generateUniqueName();
+ final String viewName = "V_" + generateUniqueName();
+ // Note that this column name is the same as the new column on the child view
+ final String alterTableDDL = "ALTER TABLE %s ADD NEW_COL1 VARCHAR";
+ createOrphanLink(SCHEMA1, parent1TableName, parent2TableName, SCHEMA2, viewName);
+
+ try (Connection conn = DriverManager.getConnection(getUrl());
+ Statement stmt = conn.createStatement()) {
+ // Should not fail since this table is unrelated to the view in spite of
+ // the orphan parent->child link
+ stmt.execute(String.format(alterTableDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent2TableName)));
+ try {
+ stmt.execute(String.format(alterTableDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent1TableName)));
+ fail("Adding column should be disallowed since there is a conflicting column type "
+ + "on the child view");
+ } catch (SQLException sqlEx) {
+ assertEquals(CANNOT_MUTATE_TABLE.getErrorCode(), sqlEx.getErrorCode());
+ }
+ }
+ }
+
+ @Test
+ public void testDropTableIsResilientToOrphanLinks() throws SQLException {
+ final String parent1TableName = generateUniqueName();
+ final String parent2TableName = generateUniqueName();
+ final String viewName = "V_" + generateUniqueName();
+ final String dropTableNoCascadeDDL = "DROP TABLE %s ";
+ createOrphanLink(SCHEMA1, parent1TableName, parent2TableName, SCHEMA2, viewName);
+
+ try (Connection conn = DriverManager.getConnection(getUrl());
+ Statement stmt = conn.createStatement()) {
+ // Should not fail since this table is unrelated to the view in spite of
+ // the orphan parent->child link
+ stmt.execute(String.format(dropTableNoCascadeDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent2TableName)));
+ try {
+ stmt.execute(String.format(dropTableNoCascadeDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent1TableName)));
+ fail("Drop table without cascade should fail since there is a child view");
+ } catch (SQLException sqlEx) {
+ assertEquals(CANNOT_MUTATE_TABLE.getErrorCode(), sqlEx.getErrorCode());
+ }
+ }
+ }
+
+ /**
+ * Create a view hierarchy:
+ * parent1 parent2
+ * | |
+ * level1view1 level1view2
+ * | |
+ * t001.level2view1 t001.level2view2
+ * |
+ * t001.level3view1
+ *
+ * We induce orphan links by recreating the same view names on top of different parents
+ */
+ @Test
+ public void testViewHierarchyWithOrphanLinks() throws Exception {
+ final List<TableInfo> expectedLegitChildViewsListForParent1 = new ArrayList<>();
+ final List<TableInfo> expectedLegitChildViewsListForParent2 = new ArrayList<>();
+ final String tenantId = "t001";
+ final String parent1TableName = "P1_" + generateUniqueName();
+ final String parent2TableName = "P2_" + generateUniqueName();
+ final String level1ViewName1 = "L1_V_1_" + generateUniqueName();
+ final String level1ViewName2 = "L1_V_2_" + generateUniqueName();
+ final String level2ViewName1 = "L2_V_1_" + generateUniqueName();
+ final String level2ViewName2 = "L2_V_2_" + generateUniqueName();
+ final String level3ViewName1 = "L3_V_1_" + generateUniqueName();
+ createOrphanLink(BASE_TABLE_SCHEMA, parent1TableName, parent2TableName,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1);
+ expectedLegitChildViewsListForParent1.add(new TableInfo(
+ null, CHILD_VIEW_LEVEL_1_SCHEMA.getBytes(), level1ViewName1.getBytes()));
+
+ try (Connection conn = DriverManager.getConnection(getUrl())) {
+ // Create a legit view on top of parent2
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_1_DDL,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName2,
+ BASE_TABLE_SCHEMA, parent2TableName));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(
+ null, CHILD_VIEW_LEVEL_1_SCHEMA.getBytes(), level1ViewName2.getBytes()));
+ }
+ Properties props = new Properties();
+ props.put(TENANT_ID_ATTRIB, tenantId);
+ try (Connection conn = DriverManager.getConnection(getUrl(), props)) {
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName1,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1));
+ expectedLegitChildViewsListForParent1.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_2_SCHEMA.getBytes(), level2ViewName1.getBytes()));
+
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName2));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_2_SCHEMA.getBytes(), level2ViewName2.getBytes()));
+
+ // Try to recreate the same view on a different global view to create an orphan link
+ try {
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1));
+ fail("Creating the same view again should have failed");
+ } catch (TableAlreadyExistsException ignored) {
+ // expected
+ }
+ // Create a third level view
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_3_DDL,
+ CHILD_VIEW_LEVEL_3_SCHEMA, level3ViewName1,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_3_SCHEMA.getBytes(), level3ViewName1.getBytes()));
+ }
+ /*
+ After this setup, SYSTEM.CHILD_LINK parent->child linking rows will look like this:
+ parent1->level1view1
+ parent2->level1view1 (orphan)
+ parent2->level1view2
+
+ level1view1->t001.level2view1
+ level1view1->t001.level2view2 (orphan)
+ level1view2->t001.level2view2
+
+ t001.level2view2->t001.level3view1
+ */
+
+ try (Connection conn = DriverManager.getConnection(getUrl())) {
+ ConnectionQueryServices cqs = conn.unwrap(PhoenixConnection.class).getQueryServices();
+ try (Table childLinkTable = cqs.getTable(SchemaUtil.getPhysicalName(
+ SYSTEM_LINK_HBASE_TABLE_NAME.toBytes(), cqs.getProps()).getName())) {
+ Pair<List<PTable>, List<TableInfo>> allDescendants =
+ ViewUtil.findAllDescendantViews(childLinkTable, cqs.getConfiguration(),
+ EMPTY_BYTE_ARRAY, BASE_TABLE_SCHEMA.getBytes(),
+ parent1TableName.getBytes(), HConstants.LATEST_TIMESTAMP, false);
+ List<PTable> legitChildViews = allDescendants.getFirst();
+ List<TableInfo> orphanViews = allDescendants.getSecond();
+ // All of the orphan links are legit views of the other parent so they don't count
+ // as orphan views for this parent
+ assertTrue(orphanViews.isEmpty());
+ assertLegitChildViews(expectedLegitChildViewsListForParent1, legitChildViews);
+
+ allDescendants = ViewUtil.findAllDescendantViews(childLinkTable,
+ cqs.getConfiguration(), EMPTY_BYTE_ARRAY, BASE_TABLE_SCHEMA.getBytes(),
+ parent2TableName.getBytes(), HConstants.LATEST_TIMESTAMP, false);
+ legitChildViews = allDescendants.getFirst();
+ orphanViews = allDescendants.getSecond();
+ // All of the orphan links are legit views of the other parent so they don't count
+ // as orphan views for this parent
+ assertTrue(orphanViews.isEmpty());
+ assertLegitChildViews(expectedLegitChildViewsListForParent2, legitChildViews);
+
+ // Drop the legitimate level 1 view that was on top of parent1
Review comment:
Wondering whether you want to add a test -
that validates that the order of dropping does not matter, in other words dropping level1view2 first.
Also, wondering whether adding multiple global views(level1 views for a parent) would make for a more robust test suite?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] twdsilva commented on pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
twdsilva commented on pull request #903:
URL: https://github.com/apache/phoenix/pull/903#issuecomment-703030957
+1, nice work thanks for fixing all these scenarios.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ChinmaySKulkarni commented on a change in pull request #903: PHOENIX-6142: Make DDL operations resilient to orphan parent->child linking rows in SYSTEM.CHILD_LINK
Posted by GitBox <gi...@apache.org>.
ChinmaySKulkarni commented on a change in pull request #903:
URL: https://github.com/apache/phoenix/pull/903#discussion_r507951694
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/end2end/ViewMetadataIT.java
##########
@@ -294,7 +321,246 @@ public void testRecreateDroppedTableWithChildViews() throws Exception {
}
}
- private void runDropChildViewsTask() {
+ @Test
+ public void testAlterTableIsResilientToOrphanLinks() throws SQLException {
+ final String parent1TableName = generateUniqueName();
+ final String parent2TableName = generateUniqueName();
+ final String viewName = "V_" + generateUniqueName();
+ // Note that this column name is the same as the new column on the child view
+ final String alterTableDDL = "ALTER TABLE %s ADD NEW_COL1 VARCHAR";
+ createOrphanLink(SCHEMA1, parent1TableName, parent2TableName, SCHEMA2, viewName);
+
+ try (Connection conn = DriverManager.getConnection(getUrl());
+ Statement stmt = conn.createStatement()) {
+ // Should not fail since this table is unrelated to the view in spite of
+ // the orphan parent->child link
+ stmt.execute(String.format(alterTableDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent2TableName)));
+ try {
+ stmt.execute(String.format(alterTableDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent1TableName)));
+ fail("Adding column should be disallowed since there is a conflicting column type "
+ + "on the child view");
+ } catch (SQLException sqlEx) {
+ assertEquals(CANNOT_MUTATE_TABLE.getErrorCode(), sqlEx.getErrorCode());
+ }
+ }
+ }
+
+ @Test
+ public void testDropTableIsResilientToOrphanLinks() throws SQLException {
+ final String parent1TableName = generateUniqueName();
+ final String parent2TableName = generateUniqueName();
+ final String viewName = "V_" + generateUniqueName();
+ final String dropTableNoCascadeDDL = "DROP TABLE %s ";
+ createOrphanLink(SCHEMA1, parent1TableName, parent2TableName, SCHEMA2, viewName);
+
+ try (Connection conn = DriverManager.getConnection(getUrl());
+ Statement stmt = conn.createStatement()) {
+ // Should not fail since this table is unrelated to the view in spite of
+ // the orphan parent->child link
+ stmt.execute(String.format(dropTableNoCascadeDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent2TableName)));
+ try {
+ stmt.execute(String.format(dropTableNoCascadeDDL,
+ SchemaUtil.getTableName(SCHEMA1, parent1TableName)));
+ fail("Drop table without cascade should fail since there is a child view");
+ } catch (SQLException sqlEx) {
+ assertEquals(CANNOT_MUTATE_TABLE.getErrorCode(), sqlEx.getErrorCode());
+ }
+ }
+ }
+
+ /**
+ * Create a view hierarchy:
+ * parent1 parent2
+ * | |
+ * level1view1 level1view2
+ * | |
+ * t001.level2view1 t001.level2view2
+ * |
+ * t001.level3view1
+ *
+ * We induce orphan links by recreating the same view names on top of different parents
+ */
+ @Test
+ public void testViewHierarchyWithOrphanLinks() throws Exception {
+ final List<TableInfo> expectedLegitChildViewsListForParent1 = new ArrayList<>();
+ final List<TableInfo> expectedLegitChildViewsListForParent2 = new ArrayList<>();
+ final String tenantId = "t001";
+ final String parent1TableName = "P1_" + generateUniqueName();
+ final String parent2TableName = "P2_" + generateUniqueName();
+ final String level1ViewName1 = "L1_V_1_" + generateUniqueName();
+ final String level1ViewName2 = "L1_V_2_" + generateUniqueName();
+ final String level2ViewName1 = "L2_V_1_" + generateUniqueName();
+ final String level2ViewName2 = "L2_V_2_" + generateUniqueName();
+ final String level3ViewName1 = "L3_V_1_" + generateUniqueName();
+ createOrphanLink(BASE_TABLE_SCHEMA, parent1TableName, parent2TableName,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1);
+ expectedLegitChildViewsListForParent1.add(new TableInfo(
+ null, CHILD_VIEW_LEVEL_1_SCHEMA.getBytes(), level1ViewName1.getBytes()));
+
+ try (Connection conn = DriverManager.getConnection(getUrl())) {
+ // Create a legit view on top of parent2
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_1_DDL,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName2,
+ BASE_TABLE_SCHEMA, parent2TableName));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(
+ null, CHILD_VIEW_LEVEL_1_SCHEMA.getBytes(), level1ViewName2.getBytes()));
+ }
+ Properties props = new Properties();
+ props.put(TENANT_ID_ATTRIB, tenantId);
+ try (Connection conn = DriverManager.getConnection(getUrl(), props)) {
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName1,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1));
+ expectedLegitChildViewsListForParent1.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_2_SCHEMA.getBytes(), level2ViewName1.getBytes()));
+
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName2));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_2_SCHEMA.getBytes(), level2ViewName2.getBytes()));
+
+ // Try to recreate the same view on a different global view to create an orphan link
+ try {
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_2_DDL,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2,
+ CHILD_VIEW_LEVEL_1_SCHEMA, level1ViewName1));
+ fail("Creating the same view again should have failed");
+ } catch (TableAlreadyExistsException ignored) {
+ // expected
+ }
+ // Create a third level view
+ conn.createStatement().execute(String.format(CREATE_CHILD_VIEW_LEVEL_3_DDL,
+ CHILD_VIEW_LEVEL_3_SCHEMA, level3ViewName1,
+ CHILD_VIEW_LEVEL_2_SCHEMA, level2ViewName2));
+ expectedLegitChildViewsListForParent2.add(new TableInfo(tenantId.getBytes(),
+ CHILD_VIEW_LEVEL_3_SCHEMA.getBytes(), level3ViewName1.getBytes()));
+ }
+ /*
+ After this setup, SYSTEM.CHILD_LINK parent->child linking rows will look like this:
+ parent1->level1view1
+ parent2->level1view1 (orphan)
+ parent2->level1view2
+
+ level1view1->t001.level2view1
+ level1view1->t001.level2view2 (orphan)
+ level1view2->t001.level2view2
+
+ t001.level2view2->t001.level3view1
+ */
+
+ try (Connection conn = DriverManager.getConnection(getUrl())) {
+ ConnectionQueryServices cqs = conn.unwrap(PhoenixConnection.class).getQueryServices();
+ try (Table childLinkTable = cqs.getTable(SchemaUtil.getPhysicalName(
+ SYSTEM_LINK_HBASE_TABLE_NAME.toBytes(), cqs.getProps()).getName())) {
+ Pair<List<PTable>, List<TableInfo>> allDescendants =
+ ViewUtil.findAllDescendantViews(childLinkTable, cqs.getConfiguration(),
+ EMPTY_BYTE_ARRAY, BASE_TABLE_SCHEMA.getBytes(),
+ parent1TableName.getBytes(), HConstants.LATEST_TIMESTAMP, false);
+ List<PTable> legitChildViews = allDescendants.getFirst();
+ List<TableInfo> orphanViews = allDescendants.getSecond();
+ // All of the orphan links are legit views of the other parent so they don't count
+ // as orphan views for this parent
+ assertTrue(orphanViews.isEmpty());
+ assertLegitChildViews(expectedLegitChildViewsListForParent1, legitChildViews);
+
+ allDescendants = ViewUtil.findAllDescendantViews(childLinkTable,
+ cqs.getConfiguration(), EMPTY_BYTE_ARRAY, BASE_TABLE_SCHEMA.getBytes(),
+ parent2TableName.getBytes(), HConstants.LATEST_TIMESTAMP, false);
+ legitChildViews = allDescendants.getFirst();
+ orphanViews = allDescendants.getSecond();
+ // All of the orphan links are legit views of the other parent so they don't count
+ // as orphan views for this parent
+ assertTrue(orphanViews.isEmpty());
+ assertLegitChildViews(expectedLegitChildViewsListForParent2, legitChildViews);
+
+ // Drop the legitimate level 1 view that was on top of parent1
Review comment:
Sure, I can add more global views.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org