You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "liferoad (via GitHub)" <gi...@apache.org> on 2025/08/24 03:09:13 UTC
[PR] [Test-only] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
liferoad opened a new pull request, #35946:
URL: https://github.com/apache/beam/pull/35946
Update MongoDB Java driver from 3.12.11 to 5.5.0 and refactor code to use new API Add mongo-bson dependency required by new driver version Replace deprecated MongoClient with MongoClients and update GridFS implementation
**MongoDB Java driver 5.5.0 supports MongoDB >=6.0**
https://www.mongodb.com/docs/drivers/java/sync/current/reference/compatibility/
------------------------
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit [https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more information about GitHub Actions CI or the [workflows README](https://github.com/apache/beam/blob/master/.github/workflows/README.md) to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3228246018
> Thanks!
>
> Consider add an item to CHANGES.md since this is a major version bump?
Let me do this later. I want to merge this first and then use SNAPSHOT to test Dataflow Templates to get more testing coverage.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Test-only] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "gemini-code-assist[bot] (via GitHub)" <gi...@apache.org>.
gemini-code-assist[bot] commented on code in PR #35946:
URL: https://github.com/apache/beam/pull/35946#discussion_r2296802126
##########
sdks/java/io/mongodb/build.gradle:
##########
@@ -28,13 +28,14 @@ dependencies {
implementation project(path: ":sdks:java:core", configuration: "shadow")
implementation library.java.joda_time
implementation library.java.mongo_java_driver
+ implementation library.java.mongo_bson
+ implementation "org.mongodb:mongodb-driver-core:5.5.0"
Review Comment:
To maintain consistency and ease of dependency management, it's better to use the library alias defined in `buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy` instead of a hardcoded version string. The `mongodb_driver_core` is already defined there.
```
implementation library.java.mongodb_driver_core
```
##########
sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbGridFSIO.java:
##########
@@ -197,12 +202,25 @@ static ConnectionConfiguration create(
}
MongoClient setupMongo() {
- return uri() == null ? new MongoClient() : new MongoClient(new MongoClientURI(uri()));
+ if (uri() == null) {
+ return MongoClients.create();
+ } else {
+ String uriString = uri();
+ if (uriString == null) {
+ return MongoClients.create();
+ }
+ MongoClientSettings settings =
+ MongoClientSettings.builder()
+ .applyConnectionString(new ConnectionString(uriString))
+ .build();
+ return MongoClients.create(settings);
+ }
}
Review Comment:
The logic in `setupMongo()` can be simplified. The check `if (uriString == null)` is redundant because `uri()` is already checked for nullability at the beginning of the method. If `uri()` is not null, `uriString` will not be null either.
```java
MongoClient setupMongo() {
if (uri() == null) {
return MongoClients.create();
}
MongoClientSettings settings =
MongoClientSettings.builder()
.applyConnectionString(new ConnectionString(uri()))
.build();
return MongoClients.create(settings);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on code in PR #35946:
URL: https://github.com/apache/beam/pull/35946#discussion_r2301368369
##########
sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbGridFSIO.java:
##########
@@ -331,12 +344,18 @@ public void teardown() {
@ProcessElement
public void processElement(final ProcessContext c) throws IOException {
- Preconditions.checkStateNotNull(gridfs);
+ GridFSBucket bucket = Preconditions.checkStateNotNull(gridFSBucket);
ObjectId oid = c.element();
- GridFSDBFile file = gridfs.find(oid);
+ GridFSDownloadStream downloadStream = bucket.openDownloadStream(oid);
+ GridFSFile gridFSFile =
+ bucket.find(com.mongodb.client.model.Filters.eq("_id", oid)).first();
+ if (gridFSFile == null) {
Review Comment:
Is it fine to early return if not found?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on code in PR #35946:
URL: https://github.com/apache/beam/pull/35946#discussion_r2301415241
##########
sdks/java/io/mongodb/src/main/java/org/apache/beam/sdk/io/mongodb/MongoDbGridFSIO.java:
##########
@@ -331,12 +344,18 @@ public void teardown() {
@ProcessElement
public void processElement(final ProcessContext c) throws IOException {
- Preconditions.checkStateNotNull(gridfs);
+ GridFSBucket bucket = Preconditions.checkStateNotNull(gridFSBucket);
ObjectId oid = c.element();
- GridFSDBFile file = gridfs.find(oid);
+ GridFSDownloadStream downloadStream = bucket.openDownloadStream(oid);
+ GridFSFile gridFSFile =
+ bucket.find(com.mongodb.client.model.Filters.eq("_id", oid)).first();
+ if (gridFSFile == null) {
Review Comment:
If not found, we do not output anything. So I think it is fine to just return. Do you see any issue here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad merged PR #35946:
URL: https://github.com/apache/beam/pull/35946
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3218437855
Assigning reviewers:
R: @ahmedabu98 for label java.
R: @Abacn for label build.
Note: If you would like to opt out of this review, comment `assign to next reviewer`.
Available commands:
- `stop reviewer notifications` - opt out of the automated review tooling
- `remind me after tests pass` - tag the comment author after tests pass
- `waiting on author` - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)
The PR bot will only process comments in the main thread (not review comments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3235262229
https://github.com/apache/beam/pull/36004 to check whether we can fix the SQL postcommit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Test-only] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3218372571
A lot of changes. If we think these are good, I can update CHANGES.md. We also need to consider Dataflow Templates.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3235098087
Looks like this caused PostCommit SQL test failing (#35514): https://github.com/apache/beam/runs/49030465739
```
java.lang.AssertionError:
Expected: <Document{{$or=[Document{{c_varchar=varchar}}, Document{{c_varchar=Document{{$not=Document{{$eq=fakeString}}}}}}], c_boolean=true, c_integer=2147483647}}>
but: was <Document{{$and=[Document{{$or=[Document{{c_varchar=varchar}}, Document{{c_varchar=Document{{$not=Document{{$eq=fakeString}}}}}}]}}, Document{{c_boolean=true}}, Document{{c_integer=2147483647}}]}}>
at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
```
need to fix assertion after checking whether it's a test issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] [Test-only] feat(mongodb): upgrade MongoDB Java driver to version 5.5.0 [beam]
Posted by "liferoad (via GitHub)" <gi...@apache.org>.
liferoad commented on PR #35946:
URL: https://github.com/apache/beam/pull/35946#issuecomment-3218367469
/gemini review
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org