You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "sekikn (via GitHub)" <gi...@apache.org> on 2023/03/29 06:57:53 UTC
[GitHub] [camel] sekikn opened a new pull request, #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
sekikn opened a new pull request, #9672:
URL: https://github.com/apache/camel/pull/9672
# Description
Currently, the scan operation of the Kudu producer scans entire table and sets the whole results to the message body, but it's inefficient or not practical if the table is large. This PR adds predicate support to the scan operation so that users can filter the results.
# Target
- [x] I checked that the commit is targeting the correct branch (note that Camel 3 uses `camel-3.x`, whereas Camel 4 uses the `main` branch)
# Tracking
- [x] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
# Apache Camel coding standards and style
- [x] I checked that each commit in the pull request has a meaningful subject line and body.
- [x] I formatted the code using `mvn -Pformat,fastinstall install && mvn -Psourcecheck`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus merged pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #9672:
URL: https://github.com/apache/camel/pull/9672
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9672:
URL: https://github.com/apache/camel/pull/9672#issuecomment-1488100707
### Components tested:
| Total | Tested | Failed :x: | Passed :white_check_mark: |
| --- | --- | --- | --- |
| 1 | 1 | 0 | 1 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] orpiske commented on a diff in pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #9672:
URL: https://github.com/apache/camel/pull/9672#discussion_r1151490014
##########
components/camel-kudu/src/test/java/org/apache/camel/component/kudu/KuduScanTest.java:
##########
@@ -107,6 +111,34 @@ public void scan() throws InterruptedException {
}
+ @Test
+ public void scanWithPredicate() throws InterruptedException {
+ errorEndpoint.expectedMessageCount(0);
+ successEndpoint.expectedMessageCount(2);
+
+ // without predicate
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(KuduConstants.CAMEL_KUDU_SCAN_PREDICATE, null);
+ sendBody("direct:scan", null, headers);
+ List<Map<String, Object>> results = (List<Map<String, Object>>) successEndpoint.getReceivedExchanges()
+ .get(0).getIn().getBody(List.class);
+ // two records with id=1 and id=2 are expected to be returned
+ assertEquals(2, results.size());
Review Comment:
Just a minor suggestion ... maybe use the comment in the `assertEquals` so that:
` assertEquals(2, results.size(), "two records with id=1 and id=2 are expected to be returned");`
That way we don't have to dig into the code to read the expectations and the errors on CI are easier to read.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9672:
URL: https://github.com/apache/camel/pull/9672#issuecomment-1488042348
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:warning: Please note that the changes on this PR may be **tested automatically**.
If necessary Apache Camel Committers may access logs and test results in the job summaries!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] sekikn commented on a diff in pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "sekikn (via GitHub)" <gi...@apache.org>.
sekikn commented on code in PR #9672:
URL: https://github.com/apache/camel/pull/9672#discussion_r1151818491
##########
components/camel-kudu/src/test/java/org/apache/camel/component/kudu/KuduScanTest.java:
##########
@@ -107,6 +111,34 @@ public void scan() throws InterruptedException {
}
+ @Test
+ public void scanWithPredicate() throws InterruptedException {
+ errorEndpoint.expectedMessageCount(0);
+ successEndpoint.expectedMessageCount(2);
+
+ // without predicate
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(KuduConstants.CAMEL_KUDU_SCAN_PREDICATE, null);
+ sendBody("direct:scan", null, headers);
+ List<Map<String, Object>> results = (List<Map<String, Object>>) successEndpoint.getReceivedExchanges()
+ .get(0).getIn().getBody(List.class);
+ // two records with id=1 and id=2 are expected to be returned
+ assertEquals(2, results.size());
Review Comment:
Thanks for the review! Will update the PR soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #9672:
URL: https://github.com/apache/camel/pull/9672#issuecomment-1488532680
### Components tested:
| Total | Tested | Failed :x: | Passed :white_check_mark: |
| --- | --- | --- | --- |
| 1 | 1 | 0 | 1 |
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] orpiske commented on a diff in pull request #9672: CAMEL-19216: camel-kudu - add predicate support to the scan operation
Posted by "orpiske (via GitHub)" <gi...@apache.org>.
orpiske commented on code in PR #9672:
URL: https://github.com/apache/camel/pull/9672#discussion_r1151490261
##########
components/camel-kudu/src/test/java/org/apache/camel/component/kudu/KuduScanTest.java:
##########
@@ -107,6 +111,34 @@ public void scan() throws InterruptedException {
}
+ @Test
+ public void scanWithPredicate() throws InterruptedException {
+ errorEndpoint.expectedMessageCount(0);
+ successEndpoint.expectedMessageCount(2);
+
+ // without predicate
+ Map<String, Object> headers = new HashMap<>();
+ headers.put(KuduConstants.CAMEL_KUDU_SCAN_PREDICATE, null);
+ sendBody("direct:scan", null, headers);
+ List<Map<String, Object>> results = (List<Map<String, Object>>) successEndpoint.getReceivedExchanges()
+ .get(0).getIn().getBody(List.class);
+ // two records with id=1 and id=2 are expected to be returned
+ assertEquals(2, results.size());
+
+ // with predicate
+ ColumnSchema schema = new ColumnSchema.ColumnSchemaBuilder("id", Type.INT32).build();
+ KuduPredicate predicate = KuduPredicate.newComparisonPredicate(schema, KuduPredicate.ComparisonOp.EQUAL, 2);
+ headers.put(KuduConstants.CAMEL_KUDU_SCAN_PREDICATE, predicate);
+ sendBody("direct:scan", null, headers);
+ results = (List<Map<String, Object>>) successEndpoint.getReceivedExchanges()
+ .get(1).getIn().getBody(List.class);
+ // only one record with id=2 is expected to be returned
+ assertEquals(1, results.size());
Review Comment:
Same note about the assert.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org