You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@camel.apache.org by "GarridoKevin (via GitHub)" <gi...@apache.org> on 2023/06/19 14:01:24 UTC
[GitHub] [camel] GarridoKevin opened a new pull request, #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
GarridoKevin opened a new pull request, #10436:
URL: https://github.com/apache/camel/pull/10436
… consumer.
# Description
Added fullDocument option to mongodb endpoint to attach the updated document to changeStreams event. Possible values are 'default' and 'updateLookup' in mongo version < 6.0.
# Target
- [ ] I checked that the commit is targeting the correct branch (note that Camel 3 uses `camel-3.x`, whereas Camel 4 uses the `main` branch)
# Tracking
- [ ] If this is a large change, bug fix, or code improvement, I checked there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for the change (usually before you start working on it).
<!--
# *Note*: trivial changes like, typos, minor documentation fixes and other small items do not require a JIRA issue. In this case your pull request should address just this issue, without pulling in other changes.
-->
# Apache Camel coding standards and style
- [ ] I checked that each commit in the pull request has a meaningful subject line and body.
<!--
If you're unsure, you can format the pull request title like `[CAMEL-XXX] Fixes bug in camel-file component`, where you replace `CAMEL-XXX` with the appropriate JIRA issue.
-->
- [ ] I formatted the code using `mvn -Pformat,fastinstall install && mvn -Psourcecheck`
<!--
You can run the aforementioned command in your module so that the build auto-formats your code and the source check verifies that is complies with our coding style. This will also be verified as part of the checks and your PR may be rejected if the checkstyle does not pass.
You can learn more about the contribution guidelines at https://github.com/apache/camel/blob/main/CONTRIBUTING.md
-->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] github-actions[bot] commented on pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #10436:
URL: https://github.com/apache/camel/pull/10436#issuecomment-1597247361
:star2: Thank you for your contribution to the Apache Camel project! :star2:
:camel: Maintainers, please note that first-time contributors *require manual approval* for the GitHub Actions to run.
:warning: Please note that the changes on this PR may be **tested automatically** if they change components.
:robot: Use the command `/component-test (camel-)component-name1 (camel-)component-name2..` to request a test from the test bot.
If necessary Apache Camel Committers may access logs and test results in the job summaries!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] GarridoKevin commented on pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "GarridoKevin (via GitHub)" <gi...@apache.org>.
GarridoKevin commented on PR #10436:
URL: https://github.com/apache/camel/pull/10436#issuecomment-1597276208
I will push another commit later for fixes for changes with camel-4.x
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] GarridoKevin commented on a diff in pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "GarridoKevin (via GitHub)" <gi...@apache.org>.
GarridoKevin commented on code in PR #10436:
URL: https://github.com/apache/camel/pull/10436#discussion_r1234185185
##########
components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/integration/MongoDbChangeStreamsConsumerIT.java:
##########
@@ -109,6 +109,36 @@ public void filterTest() throws Exception {
@Order(3)
@Test
+ public void updateWithFullDocumentTest() throws Exception {
+ assertEquals(0, mongoCollection.countDocuments());
+ MockEndpoint mock = contextExtension.getMockEndpoint("mock:test");
+ mock.expectedMessageCount(1);
+
+ String consumerRouteId = "updateWithFullDocumentConsumer";
+ context.getRouteController().startRoute(consumerRouteId);
+
+ ObjectId objectId1 = new ObjectId();
+ ObjectId objectId2 = new ObjectId();
+ Executors.newSingleThreadExecutor().submit(() -> {
Review Comment:
I'll change the other tests as well then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] essobedo commented on pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "essobedo (via GitHub)" <gi...@apache.org>.
essobedo commented on PR #10436:
URL: https://github.com/apache/camel/pull/10436#issuecomment-1597281877
> I will push another commit later for fixes for changes with [camel-4](https://issues.apache.org/jira/browse/CAMEL-4).x
Just to be clear, If the target branch of your PR is `main` then the fix will go in Camel 4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] GarridoKevin commented on pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "GarridoKevin (via GitHub)" <gi...@apache.org>.
GarridoKevin commented on PR #10436:
URL: https://github.com/apache/camel/pull/10436#issuecomment-1597315398
> > I will push another commit later for fixes for changes with [camel-4](https://issues.apache.org/jira/browse/CAMEL-4).x
>
> Just to be clear, If the target branch of your PR is `main` then the fix will go in Camel 4
Yes, there is another separate PR for camel-3x. I opened this one without checking for change in the new version.
It should be good now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] essobedo commented on a diff in pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "essobedo (via GitHub)" <gi...@apache.org>.
essobedo commented on code in PR #10436:
URL: https://github.com/apache/camel/pull/10436#discussion_r1234167731
##########
components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/integration/MongoDbChangeStreamsConsumerIT.java:
##########
@@ -109,6 +109,36 @@ public void filterTest() throws Exception {
@Order(3)
@Test
+ public void updateWithFullDocumentTest() throws Exception {
+ assertEquals(0, mongoCollection.countDocuments());
+ MockEndpoint mock = contextExtension.getMockEndpoint("mock:test");
+ mock.expectedMessageCount(1);
+
+ String consumerRouteId = "updateWithFullDocumentConsumer";
+ context.getRouteController().startRoute(consumerRouteId);
+
+ ObjectId objectId1 = new ObjectId();
+ ObjectId objectId2 = new ObjectId();
+ Executors.newSingleThreadExecutor().submit(() -> {
Review Comment:
If you don't mind, I would rather prefer to use the common pool instead by calling `CompletableFuture.runAsync` like in this commit https://github.com/apache/camel/commit/8a33263dee0d6265a970835f1417a6c166838df9
##########
components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbChangeStreamsThread.java:
##########
@@ -34,10 +35,13 @@
class MongoDbChangeStreamsThread extends MongoAbstractConsumerThread {
private List<BsonDocument> bsonFilter;
private BsonDocument resumeToken;
+ private FullDocument fullDocument;
- MongoDbChangeStreamsThread(MongoDbEndpoint endpoint, MongoDbChangeStreamsConsumer consumer, List<BsonDocument> bsonFilter) {
+ MongoDbChangeStreamsThread(MongoDbEndpoint endpoint, MongoDbChangeStreamsConsumer consumer,
+ List<BsonDocument> bsonFilter, FullDocument fullDocument) {
Review Comment:
You don't need to add `fullDocument` in the constructor since it is already available from the endpoint
##########
components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbEndpoint.java:
##########
@@ -107,6 +107,8 @@ public class MongoDbEndpoint extends DefaultEndpoint {
private String tailTrackIncreasingField;
@UriParam(label = "consumer,changeStream")
private String streamFilter;
+ @UriParam(label = "consumer,changeStream", enums = "default,updateLookup", defaultValue = "default")
+ private String fullDocument = "default";
Review Comment:
Why not use the enum type here instead of String?
##########
components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbChangeStreamsThread.java:
##########
@@ -48,8 +52,8 @@ protected void init() {
@Override
protected MongoCursor initializeCursor() {
ChangeStreamIterable<Document> iterable = bsonFilter != null
- ? dbCol.watch(bsonFilter)
- : dbCol.watch();
+ ? dbCol.watch(bsonFilter).fullDocument(fullDocument)
+ : dbCol.watch().fullDocument(fullDocument);
Review Comment:
Or simply call it only once after, I mean something like `iterable = iterable.fullDocument(fullDocument);`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] davsclaus merged pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "davsclaus (via GitHub)" <gi...@apache.org>.
davsclaus merged PR #10436:
URL: https://github.com/apache/camel/pull/10436
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] essobedo commented on a diff in pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "essobedo (via GitHub)" <gi...@apache.org>.
essobedo commented on code in PR #10436:
URL: https://github.com/apache/camel/pull/10436#discussion_r1234167731
##########
components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/integration/MongoDbChangeStreamsConsumerIT.java:
##########
@@ -109,6 +109,36 @@ public void filterTest() throws Exception {
@Order(3)
@Test
+ public void updateWithFullDocumentTest() throws Exception {
+ assertEquals(0, mongoCollection.countDocuments());
+ MockEndpoint mock = contextExtension.getMockEndpoint("mock:test");
+ mock.expectedMessageCount(1);
+
+ String consumerRouteId = "updateWithFullDocumentConsumer";
+ context.getRouteController().startRoute(consumerRouteId);
+
+ ObjectId objectId1 = new ObjectId();
+ ObjectId objectId2 = new ObjectId();
+ Executors.newSingleThreadExecutor().submit(() -> {
Review Comment:
If you don't mind, I would rather prefer to use the common pool instead by calling `CompletableFuture.runAsync` like in this commit https://github.com/apache/camel/commit/8a33263dee0d6265a970835f1417a6c166838df9. This way you don't create any thread pool.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [camel] essobedo commented on a diff in pull request #10436: CAMEL-19470 camel-mongodb : add option fullDocument for changeStreams...
Posted by "essobedo (via GitHub)" <gi...@apache.org>.
essobedo commented on code in PR #10436:
URL: https://github.com/apache/camel/pull/10436#discussion_r1234221826
##########
components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbEndpoint.java:
##########
@@ -664,15 +665,15 @@ public void setStreamFilter(String streamFilter) {
this.streamFilter = streamFilter;
}
- public String getFullDocument() {
+ public FullDocument getFullDocument() {
return fullDocument;
}
/**
* Specifies whether changeStream consumer include a copy of the full document when modified by update operations.
* Possible values are default and updateLookup.
Review Comment:
Seems to be out dated with the latest changes
##########
components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbEndpoint.java:
##########
@@ -580,9 +581,9 @@ public String getTailTrackIncreasingField() {
public MongoDbTailTrackingConfig getTailTrackingConfig() {
if (tailTrackingConfig == null) {
tailTrackingConfig = new MongoDbTailTrackingConfig(
- persistentTailTracking, tailTrackIncreasingField, tailTrackDb == null ? database : tailTrackDb,
- tailTrackCollection,
- tailTrackField, getPersistentId());
+ persistentTailTracking, tailTrackIncreasingField,
+ tailTrackDb == null ? database : tailTrackDb, tailTrackCollection, tailTrackField,
+ getPersistentId());
Review Comment:
I don't see any changes, only reformatting, so please revert
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@camel.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org