You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by "NightOwl888 (via GitHub)" <gi...@apache.org> on 2026/09/01 03:57:17 UTC
[I] Investigate `TestDirectoryTaxonomy.TestConcurrency()` rare test failure [lucenenet]
NightOwl888 opened a new issue, #1444:
URL: https://github.com/apache/lucenenet/issues/1444
### Is there an existing issue for this?
- [x] I have searched the existing issues
### Describe the bug
This test failed 1 out of 30 runs when merging `master` @ e7e18b275abafbb9c010a32b844f0c04592a4fac+ #1373 + #1311.
```
mismatch number of categories
(Test: Lucene.Net.Facet.Taxonomy.Directory.TestDirectoryTaxonomyWriter.TestConcurrency)
To reproduce this test result:
Option 1:
Apply the following assembly-level attributes:
[assembly: Lucene.Net.Util.RandomSeed("0x82f9bca80d16aa84:0xffeaa58b2329e4e9")]
[assembly: NUnit.Framework.SetCulture("en")]
Option 2:
Use the following .runsettings file:
<RunSettings>
<TestRunParameters>
<Parameter name="tests:seed" value="0x82f9bca80d16aa84:0xffeaa58b2329e4e9" />
<Parameter name="tests:culture" value="en" />
</TestRunParameters>
</RunSettings>
Option 3:
Create the following lucene.testsettings.json file somewhere between the test assembly and the root of your drive:
{
"tests": {
"seed": "0x82f9bca80d16aa84:0xffeaa58b2329e4e9",
"culture": "en"
}
}
Fixture Test Values
Random Seed: 0x82f9bca80d16aa84:0xffeaa58b2329e4e9
Culture: en
Time Zone: (UTC+01:00) Amsterdam, Berlin, Bern, Rome, Stockholm, Vienna
Default Codec: Lucene41 (Lucene41RWCodec)
Default Similarity: RandomSimilarityProvider(queryNorm=True,coord=yes): {}
System Properties
Nightly: False
Weekly: False
Slow: True
Awaits Fix: False
Directory: random
Verbose: False
Random Multiplier: 1
Stack trace
at Lucene.Net.Facet.Taxonomy.Directory.TestDirectoryTaxonomyWriter.TestConcurrency() in /_/src/Lucene.Net.Tests.Facet/Taxonomy/Directory/TestDirectoryTaxonomyWriter.cs:line 325
1) at Lucene.Net.Facet.Taxonomy.Directory.TestDirectoryTaxonomyWriter.TestConcurrency() in /_/src/Lucene.Net.Tests.Facet/Taxonomy/Directory/TestDirectoryTaxonomyWriter.cs:line 325
```
I was unable to reproduce it more than the one time, so I am opening this issue to track it in case it happens again after those 2 PRs are merged. I attempted to reproduce it on all 3 upstream branches without success (even when I placed `[Repeat(25)]` on it).
Since the combination of those 3 branches is much more stable than any of them on their own, it seems like merging them should take priority over tracking this issue down.
### Expected Behavior
_No response_
### Steps To Reproduce
_No response_
### Exceptions (if any)
_No response_
### Lucene.NET Version
master @ e7e18b275abafbb9c010a32b844f0c04592a4fac+ #1373 + #1311
### .NET Version
net8.0
### Operating System
Windows
### Anything else?
x86
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `TestDirectoryTaxonomy.TestConcurrency()` rare test failure [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin commented on issue #1444:
URL: https://github.com/apache/lucenenet/issues/1444#issuecomment-6298768070
An update on my earlier comment. The stress testing confirmed that the issue was fixed (it ran 2,400 times with zero failures, compared to failing at the 347th run without the fix), however, I was second-guessing it. If the .NET team intentionally made hash codes random so that we don't depend on them, maybe we should fix the real root cause of this problem rather than patching our way around it by using stable hash codes across processes?
I went back to the drawing board and found that this was fixed upstream in Lucene 7.1 via LUCENE-7891. The description of that issue includes this relevant bit:
> LruTaxonomyWriterCache uses LRUType.LRU_HASHED by default. This has a very small but non-zero chance of producing incorrect results due to collisions in the longHashCode of FacetLabel. If such a collision occurs, then an affected document will get an incorrect facet.
That's exactly what we ran into here. So the better fix is to backport this change instead of trying to make the hash codes stable across processes. PR incoming that will do just that and backport the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `TestDirectoryTaxonomy.TestConcurrency()` rare test failure [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin commented on issue #1444:
URL: https://github.com/apache/lucenenet/issues/1444#issuecomment-6295039934
Found the cause with the help of Claude Code. FacetLabel and CategoryPath have `Int64HashCode()` methods that call `string.GetHashCode()`, which in modern .NET is randomized per process. Can reliably reproduce by forcing the test to pick the LruTaxonomyWriterCache (which normally is picked only ~30% of the time) and running repeated runs in a loop (failed at run 347 on my machine). It fails when the last component's 32-bit string hash collides, which is a birthday problem with a random failure rate of ~3.5% per process, and label `a` has to still be cached when `b` is looked up. Since `string.GetHashCode()`'s randomization is not affected by our random seed, we can't control that behavior with a fixed seed to reproduce.
Switching the methods to use J2N's `CharSequenceComparer.Ordinal.GetHashCode(string)` results in a stable hash code, fixing the issue.
PR incoming once I stress-test this to ensure there are no other failures.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Investigate `TestDirectoryTaxonomy.TestConcurrency()` rare test failure [lucenenet]
Posted by "paulirwin (via GitHub)" <gi...@apache.org>.
paulirwin closed issue #1444: Investigate `TestDirectoryTaxonomy.TestConcurrency()` rare test failure
URL: https://github.com/apache/lucenenet/issues/1444
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@lucenenet.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org