You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@pulsar.apache.org by "LucasEby (via GitHub)" <gi...@apache.org> on 2025/10/09 22:41:03 UTC
[I] [Bug] Nondeterministic ordering in MongoSourceTest.testWriteBadMessage [pulsar]
LucasEby opened a new issue, #24834:
URL: https://github.com/apache/pulsar/issues/24834
### Search before reporting
- [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Read release policy
- [x] I understand that [unsupported versions](https://pulsar.apache.org/contribute/release-policy/#supported-versions) don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker.
### User environment
Broker version: 4.2.0-SNAPSHOT
Broker Operating system and hardware type: Linux x86_64 GNU/Linux
Broker Java version: 17.0.16
### Issue Description
The test was written with the incorrect assumption that the compared strings have a deterministic order. Since the original data was stored with [HashMaps](https://docs.oracle.com/javase/8/docs/api/java/util/HashMap.html), which makes no guarantees to the order of it's map, the resulting strings that were compared could potentially have reordered contents. The ordering can change due to different environments producing the contents in different orders despite the logical contents being the same. Since the raw strings/trees were compared "as-is", harmless re-ordering could flip the test from pass to fail despite the data being semantically the same.
I discovered the problem with the [NonDex](https://github.com/TestingResearchIllinois/NonDex) tool. NonDex systematically detects incorrect tests that rely on non-deterministic behaviors in Java APIs—like assuming order of the contents of HashMaps — by exploring all specification-allowed outcomes. It does this by instrumenting undetermined APIs and randomizing the returned order/behavior within what the spec allows. Failures it exposes reliably indicate flawed assumptions in order that were never promised and should be fixed.
### Error messages
```text
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.apache.pulsar.io.mongodb.MongoSourceTest
[ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 3.958 s <<< FAILURE! -- in org.apache.pulsar.io.mongodb.MongoSourceTest
[ERROR] org.apache.pulsar.io.mongodb.MongoSourceTest.testWriteBadMessage -- Time elapsed: 0.017 s <<< FAILURE!
java.lang.AssertionError: expected [{"fullDocument":{"hello":"pulsar"},"ns":{"databaseName":"hello","collectionName":"pulsar","fullName":"hello.pulsar"},"operation":"INSERT"}] but found [{"fullDocument":{"hello":"pulsar"},"operation":"INSERT","ns":{"fullName":"hello.pulsar","collectionName":"pulsar","databaseName":"hello"}}]
at org.testng.Assert.fail(Assert.java:110)
at org.testng.Assert.failNotEquals(Assert.java:1577)
at org.testng.Assert.assertEqualsImpl(Assert.java:149)
at org.testng.Assert.assertEquals(Assert.java:131)
at org.testng.Assert.assertEquals(Assert.java:655)
at org.testng.Assert.assertEquals(Assert.java:665)
at org.apache.pulsar.io.mongodb.MongoSourceTest.testWriteBadMessage(MongoSourceTest.java:131)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:139)
at org.testng.internal.invokers.InvokeMethodRunnable.runOne(InvokeMethodRunnable.java:47)
at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:76)
at org.testng.internal.invokers.InvokeMethodRunnable.call(InvokeMethodRunnable.java:11)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:840)
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] org.apache.pulsar.io.mongodb.MongoSourceTest.testWriteBadMessage
[INFO] Run 1: PASS
[ERROR] Run 2: MongoSourceTest.testWriteBadMessage:131 expected [{"fullDocument":{"hello":"pulsar"},"ns":{"databaseName":"hello","collectionName":"pulsar","fullName":"hello.pulsar"},"operation":"INSERT"}] but found [{"fullDocument":{"hello":"pulsar"},"operation":"INSERT","ns":{"fullName":"hello.pulsar","collectionName":"pulsar","databaseName":"hello"}}]
[INFO]
[INFO]
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0
```
### Reproducing the issue
You can replicate the problem with the [NonDex](https://github.com/TestingResearchIllinois/NonDex) tool command:
```mvn -pl pulsar-io/mongo -Dtest=org.apache.pulsar.io.mongodb.MongoSourceTest#testWriteBadMessage -DforkCount=1 -DnondexRuns=1 -DnondexSeed=1016066 -DreuseForks=false edu.illinois:nondex-maven-plugin:2.2.1:nondex```
### Additional information
_No response_
### Are you willing to submit a PR?
- [x] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Bug] Nondeterministic ordering in MongoSourceTest.testWriteBadMessage [pulsar]
Posted by "lhotari (via GitHub)" <gi...@apache.org>.
lhotari closed issue #24834: [Bug] Nondeterministic ordering in MongoSourceTest.testWriteBadMessage
URL: https://github.com/apache/pulsar/issues/24834
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org