You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by GitBox <gi...@apache.org> on 2021/05/03 23:54:13 UTC
[GitHub] [phoenix] ankitjain64 opened a new pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
ankitjain64 opened a new pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219
…stemCatalogWALEntryFilter
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ankitjain64 commented on a change in pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
ankitjain64 commented on a change in pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#discussion_r625451166
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
##########
@@ -103,18 +104,20 @@ private boolean isTenantRowCellSystemChildLink(final Cell cell) {
if (!isTenantRowCell) {
boolean isChildLink = CellUtil.matchingQualifier(
cell, PhoenixDatabaseMetaData.LINK_TYPE_BYTES);
- if (isChildLink) {
- if (CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) {
+ // Check if cell is of type LINK_TYPE with value 4 or DeleteFamily
+ if ((isChildLink && CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) ||
+ CellUtil.isDeleteFamily(cell) ) {
Review comment:
To replicate a delete marker we are using two checks.
1. The cell is of type `DeleteFamily`.
2. The length for `COLUMN_NAME` is greater than `0`. (For tenant views column_name is populated with tenant_id which is not the case for global views.)
One may argue that the above two checks are also true for delete markers of column rows and we may over-replicate those rows. But, we are avoiding all those scenarios by doing the above filtering only for the mutations belonging to `SYSTEM.CHILD_LINK` table. With 4.15+ we are assured that `SYSTEM.CHILD_LINK` table only stores parent-child linking rows with `LINK_TYPE=4`, so the only thing we need to differentiate here is parent-child link delete markers for tenant vs non-tenant view. Let me know if you still feel we are missing any scenario. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ankitjain64 commented on pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
ankitjain64 commented on pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#issuecomment-831606695
With this PR we are fixing the bug where delete markers for parent-child link rows do not get replicated via SystemCatalogWalEntry Filter for tenant views.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] gjacoby126 merged pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
gjacoby126 merged pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] xcangCRM commented on a change in pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
xcangCRM commented on a change in pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#discussion_r625448015
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilterIT.java
##########
@@ -281,4 +347,68 @@ private int getAndAssertTenantCountInEdit(WAL.Entry entry) {
}
return entry;
}
+
+ public WAL.Entry getEntry(TableName tableName, Scan scan, boolean addIndexedKeyValueCell)
Review comment:
Does this have to be public?
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
##########
@@ -103,18 +104,20 @@ private boolean isTenantRowCellSystemChildLink(final Cell cell) {
if (!isTenantRowCell) {
boolean isChildLink = CellUtil.matchingQualifier(
cell, PhoenixDatabaseMetaData.LINK_TYPE_BYTES);
- if (isChildLink) {
- if (CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) {
+ // Check if cell is of type LINK_TYPE with value 4 or DeleteFamily
+ if ((isChildLink && CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) ||
+ CellUtil.isDeleteFamily(cell) ) {
Review comment:
this logic means you will replicate whenever it's a DeleteFamily cell, regardless of childlink or not. Is this the intention?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ankitjain64 commented on a change in pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
ankitjain64 commented on a change in pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#discussion_r625451166
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
##########
@@ -103,18 +104,20 @@ private boolean isTenantRowCellSystemChildLink(final Cell cell) {
if (!isTenantRowCell) {
boolean isChildLink = CellUtil.matchingQualifier(
cell, PhoenixDatabaseMetaData.LINK_TYPE_BYTES);
- if (isChildLink) {
- if (CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) {
+ // Check if cell is of type LINK_TYPE with value 4 or DeleteFamily
+ if ((isChildLink && CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) ||
+ CellUtil.isDeleteFamily(cell) ) {
Review comment:
To replicate a delete marker we are using two checks.
1. The cell is of type `DeleteFamily`.
2. The length for `COLUMN_NAME` is greater than `0`. (For tenant views column_name is populated with tenant_id which is not the case for global views.)
One may argue that the above two checks are also true for delete markers of column rows and we may over-replicate those rows. But, we are avoiding all those scenarios by doing the above filtering only for the mutations belonging to `SYSTEM.CHILD_LINK` table. With 4.15+ we are assured that `SYSTEM.CHILD_LINK` table only stores parent-child linking rows with `LINK_TYPE=4`. Let me know if you still feel we are missing any scenario. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] stoty commented on pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
stoty commented on pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#issuecomment-831917190
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Comment |
|:----:|----------:|--------:|:--------|
| +0 :ok: | reexec | 1m 36s | Docker mode activated. |
||| _ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. |
| +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. |
| +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. |
| -1 :x: | test4tests | 0m 0s | The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. |
||| _ 4.16 Compile Tests _ |
| +1 :green_heart: | mvninstall | 15m 24s | 4.16 passed |
| +1 :green_heart: | compile | 1m 9s | 4.16 passed |
| +1 :green_heart: | checkstyle | 0m 30s | 4.16 passed |
| +1 :green_heart: | javadoc | 0m 49s | 4.16 passed |
| +0 :ok: | spotbugs | 3m 23s | phoenix-core in 4.16 has 945 extant spotbugs warnings. |
||| _ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 6m 50s | the patch passed |
| +1 :green_heart: | compile | 1m 10s | the patch passed |
| +1 :green_heart: | javac | 1m 10s | the patch passed |
| -1 :x: | checkstyle | 0m 31s | phoenix-core: The patch generated 6 new + 8 unchanged - 0 fixed = 14 total (was 8) |
| +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. |
| +1 :green_heart: | javadoc | 0m 51s | the patch passed |
| +1 :green_heart: | spotbugs | 3m 40s | the patch passed |
||| _ Other Tests _ |
| +1 :green_heart: | unit | 199m 44s | phoenix-core in the patch passed. |
| +1 :green_heart: | asflicense | 0m 10s | The patch does not generate ASF License warnings. |
| | | 236m 36s | |
| Subsystem | Report/Notes |
|----------:|:-------------|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1219/2/artifact/yetus-general-check/output/Dockerfile |
| GITHUB PR | https://github.com/apache/phoenix/pull/1219 |
| JIRA Issue | PHOENIX-6437 |
| Optional Tests | dupname asflicense javac javadoc unit spotbugs hbaseanti checkstyle compile |
| uname | Linux b2b42220db37 4.15.0-101-generic #102-Ubuntu SMP Mon May 11 10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev/phoenix-personality.sh |
| git revision | 4.16 / e3c8ac0 |
| Default Java | Private Build-1.8.0_242-8u242-b08-0ubuntu3~16.04-b08 |
| checkstyle | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1219/2/artifact/yetus-general-check/output/diff-checkstyle-phoenix-core.txt |
| Test Results | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1219/2/testReport/ |
| Max. process+thread count | 4919 (vs. ulimit of 30000) |
| modules | C: phoenix-core U: phoenix-core |
| Console output | https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1219/2/console |
| versions | git=2.7.4 maven=3.3.9 spotbugs=4.1.3 |
| Powered by | Apache Yetus 0.12.0 https://yetus.apache.org |
This message was automatically generated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ankitjain64 commented on a change in pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
ankitjain64 commented on a change in pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#discussion_r625451166
##########
File path: phoenix-core/src/main/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilter.java
##########
@@ -103,18 +104,20 @@ private boolean isTenantRowCellSystemChildLink(final Cell cell) {
if (!isTenantRowCell) {
boolean isChildLink = CellUtil.matchingQualifier(
cell, PhoenixDatabaseMetaData.LINK_TYPE_BYTES);
- if (isChildLink) {
- if (CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) {
+ // Check if cell is of type LINK_TYPE with value 4 or DeleteFamily
+ if ((isChildLink && CellUtil.matchingValue(cell, CHILD_TABLE_BYTES)) ||
+ CellUtil.isDeleteFamily(cell) ) {
Review comment:
To replicate a delete marker we are using two checks.
1. The cell is of type `DeleteFamily`.
2. The length for `COLUMN_NAME` is greater than `0`. (For tenant views column_name is populated with tenant_id which is not the case for global views.)
One may argue that the above two checks are also true for delete markers of column rows and we may over-replicate that as well. But, we are avoiding all those scenarios by doing the above filtering only for the mutations belonging to `SYSTEM.CHILD_LINK` table. With 4.15+ we are assured that `SYSTEM.CHILD_LINK` table only stores parent-child linking rows with `LINK_TYPE=4`. Let me know if you still feel we are missing any scenario. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [phoenix] ankitjain64 commented on a change in pull request #1219: PHOENIX-6437: Parent-Child Delete marker should get replicated via Sy…
Posted by GitBox <gi...@apache.org>.
ankitjain64 commented on a change in pull request #1219:
URL: https://github.com/apache/phoenix/pull/1219#discussion_r625452537
##########
File path: phoenix-core/src/it/java/org/apache/phoenix/replication/SystemCatalogWALEntryFilterIT.java
##########
@@ -281,4 +347,68 @@ private int getAndAssertTenantCountInEdit(WAL.Entry entry) {
}
return entry;
}
+
+ public WAL.Entry getEntry(TableName tableName, Scan scan, boolean addIndexedKeyValueCell)
Review comment:
Updated in latest commit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org