You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@phoenix.apache.org by "tkhurana (via GitHub)" <gi...@apache.org> on 2024/05/28 19:52:08 UTC
[PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
tkhurana opened a new pull request, #1897:
URL: https://github.com/apache/phoenix/pull/1897
(no comment)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "kadirozde (via GitHub)" <gi...@apache.org>.
kadirozde commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2136641699
@tkhurana, the PR is good to me. There is one test failure related to this PR which is OnDuplicateKeyIT.testAtomicUpdate. I will approve the PR if that is resolved.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2138652103
Based on this build result, perhaps CDC feature might have caused this flakes https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/679/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2138650239
@kadirozde @tkhurana I am able to reproduce `GlobalImmutableTxIndexIT#testLastDDLTimestampOnAsyncIndexes` test failure locally:
```
java.lang.AssertionError: PTable DDL Timestamp not in the right range!
at org.junit.Assert.fail(Assert.java:89)
at org.junit.Assert.assertTrue(Assert.java:42)
at org.apache.phoenix.end2end.CreateTableIT.verifyLastDDLTimestamp(CreateTableIT.java:1765)
at org.apache.phoenix.end2end.index.BaseIndexIT.testLastDDLTimestampOnAsyncIndexes(BaseIndexIT.java:1517)
...
...
```
So something else on master branch might have broken this test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "tkhurana (via GitHub)" <gi...@apache.org>.
tkhurana commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2140469374
The latest run only had test failures due to known flappers. Merging
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "tkhurana (via GitHub)" <gi...@apache.org>.
tkhurana merged PR #1897:
URL: https://github.com/apache/phoenix/pull/1897
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "ranganathg (via GitHub)" <gi...@apache.org>.
ranganathg commented on code in PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#discussion_r1618276259
##########
phoenix-core-server/src/main/java/org/apache/phoenix/hbase/index/IndexRegionObserver.java:
##########
@@ -857,11 +864,30 @@ private void getCurrentRowStates(ObserverContext<RegionCoprocessorEnvironment> c
if (keys.isEmpty()) {
return;
}
- Scan scan = new Scan();
- ScanRanges scanRanges = ScanRanges.createPointLookup(new ArrayList<KeyRange>(keys));
- scanRanges.initializeScan(scan);
- SkipScanFilter skipScanFilter = scanRanges.getSkipScanFilter();
- scan.setFilter(skipScanFilter);
+
+ if (this.useBloomFilter) {
+ for (KeyRange key : keys) {
+ // Scan.java usage alters scan instances, safer to create scan instance per usage
+ Scan scan = new Scan();
+ // create a scan with same start/stop row key scan#isGetScan()
+ // for bloom filters scan should be a get
+ scan.withStartRow(key.getLowerRange(), true);
+ scan.withStopRow(key.getLowerRange(), true);
Review Comment:
start and stop row is the same ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "kadirozde (via GitHub)" <gi...@apache.org>.
kadirozde commented on code in PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#discussion_r1618278521
##########
phoenix-core-server/src/main/java/org/apache/phoenix/hbase/index/IndexRegionObserver.java:
##########
@@ -857,11 +864,30 @@ private void getCurrentRowStates(ObserverContext<RegionCoprocessorEnvironment> c
if (keys.isEmpty()) {
return;
}
- Scan scan = new Scan();
- ScanRanges scanRanges = ScanRanges.createPointLookup(new ArrayList<KeyRange>(keys));
- scanRanges.initializeScan(scan);
- SkipScanFilter skipScanFilter = scanRanges.getSkipScanFilter();
- scan.setFilter(skipScanFilter);
+
+ if (this.useBloomFilter) {
+ for (KeyRange key : keys) {
+ // Scan.java usage alters scan instances, safer to create scan instance per usage
+ Scan scan = new Scan();
+ // create a scan with same start/stop row key scan#isGetScan()
+ // for bloom filters scan should be a get
+ scan.withStartRow(key.getLowerRange(), true);
+ scan.withStopRow(key.getLowerRange(), true);
Review Comment:
Yes, it means point lookup for HBase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "virajjasani (via GitHub)" <gi...@apache.org>.
virajjasani commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2138646315
Triggered another build: https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-PreCommit-GitHub-PR/job/PR-1897/3/
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] PHOENIX-7319 Leverage Bloom Filters to improve performance on write path [phoenix]
Posted by "tkhurana (via GitHub)" <gi...@apache.org>.
tkhurana commented on PR #1897:
URL: https://github.com/apache/phoenix/pull/1897#issuecomment-2140466779
> Based on this build result, perhaps CDC feature might have caused this flakes https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/679/
You are right @virajjasani . The CDC PR broke the `testLastDDLTimestampOnAsyncIndexes` test
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@phoenix.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org