You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by GitBox <gi...@apache.org> on 2020/06/09 20:17:13 UTC
[GitHub] [lucenenet] NightOwl888 opened a new issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
NightOwl888 opened a new issue #295:
URL: https://github.com/apache/lucenenet/issues/295
As part of #261, here is a list of known tests in the `Lucene.Net.Util` namespace that are taking significantly longer in .NET than they did in Java. The percentage is the amount of time that .NET takes compared to Java (for example, if the .NET test is running 10x longer the percentage will be 1000% of the Java test).
These tests need to be investigated to determine what is causing the slowness. Fixing these issues will likely ripple across the entire project reducing the time the tests take to run as well as impacting end users.
## Slow Running Tests
- [ ] Lucene.Net.Util.TestDocIdBitSet.TestAgainstBitSet (11583%)
- [ ] Lucene.Net.Util.TestMathUtil.TestAcoshMethod (7400%)
- [ ] Lucene.Net.Util.TestAttributeSource.TestCaptureState (4850%)
- [ ] Lucene.Net.Util.TestBroadWord.TestPerfSelectAllBitsBroad (4267%)
- [ ] Lucene.Net.Util.TestFixedBitSet.TestAgainstBitSet (2999%)
- [ ] Lucene.Net.Util.Automaton.TestBasicOperations.TestEmptyLanguageConcatenate (2750%)
- [ ] Lucene.Net.Util.TestRecyclingByteBlockAllocator.TestAllocate (2333%)
- [ ] Lucene.Net.Util.TestPackedInts.TestAppendingLongBuffer (1597%)
- [ ] Lucene.Net.Util.TestIdentityHashSet.TestCheck (1176%)
- [ ] Lucene.Net.Util.TestPagedBytes.TestDataInputOutput (1015%)
- [ ] Lucene.Net.Util.TestByteBlockPool.TestReadAndWrite (1008%)
- [ ] Lucene.Net.Util.TestWAH8DocIdSet.TestAgainstBitSet (895%)
- [ ] Lucene.Net.Util.TestPagedBytes.TestDataInputOutput2 (849%)
- [ ] Lucene.Net.Util.TestBasicOperations.TestEmptySingletonConcatenate (775%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestSize (762%)
- [ ] Lucene.Net.Util.TestOpenBitSet.TestAgainstBitSet (720%)
- [ ] Lucene.Net.Util.TestNumericUtils.TestRandomSplit (708%)
- [ ] Lucene.Net.Util.Fst.TestBytesStore.TestRandom (681%)
- [ ] Lucene.Net.Util.Packed.TestEliasFanoSequence.TestAdvanceToAndBackToMultiples (555%)
- [ ] Lucene.Net.Util.TestOfflineSorter.TestSmallRandom (555%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestSort (414%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestGet (343%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestAddByPoolOffset (331%)
- [ ] Lucene.Net.Util.TestUnicodeUtil.TestCodePointCount (294%)
- [ ] Lucene.Net.Util.Fst.TestFsts.TestRandomWords (289%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestFind (284%)
- [ ] Lucene.Net.Util.TestBytesRefHash.TestCompact (273%)
- [ ] Lucene.Net.Util.TestUnicodeUtil.TestUTF8toUTF32 (256%)
- [ ] Lucene.Net.Util.TestOfflineSorter.TestIntermediateMerges (242%)
- [ ] Lucene.Net.Util.TestOpenBitSet.TestSmall (188%)
---------------
> These tests were not apples-apples comparisons, the Java tests were run on an older and slower machine that was outclassed in RAM, CPU, and disk read/write speeds by the machine that was used to test in .NET. The percentage is the approximate value that was measured by the test frameworks, but is probably significantly lower than the difference using a proper benchmark on the same machine.
### BitSet vs BitArray
The `TestAgainstBitSet` test is slow in almost every test that utilizes it.
One potential reason for this is that we are using the .NET `BitArray` class and in Java Lucene was using a `BitSet`. While remapping to .NET APIs is the preferred way of porting, in this case we have classes with completely different behaviors. `BitArray` is a fixed size, and `BitSet` automatically expands as needed. To make `BitArray` expand, we need to reallocate it, and those extra allocations may be contributing to these tests being slow.
Note that J2N has [a C# implementation of `BitSet`](https://github.com/NightOwl888/J2N/blob/1ace2b3ad498501f9ea36bc277b353b54ddfc588/src/J2N/Collections/BitSet.cs) that would be acceptable to use instead of `BitArray` if this is indeed one of the contributors.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 closed issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
NightOwl888 closed issue #295:
URL: https://github.com/apache/lucenenet/issues/295
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 closed issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
NightOwl888 closed issue #295:
URL: https://github.com/apache/lucenenet/issues/295
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] bongohrtech edited a comment on issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
bongohrtech edited a comment on issue #295:
URL: https://github.com/apache/lucenenet/issues/295#issuecomment-648122779
Tests in T-U seem to have been resolved where there is the most delay.
Adding the following line will force the tests to use the overloaded methods.
using Assert = Lucene.Net.TestFramework.Assert;
assertEquals in LuceneTestCase has also been modified to use the same equivalence checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] bongohrtech commented on issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
bongohrtech commented on issue #295:
URL: https://github.com/apache/lucenenet/issues/295#issuecomment-648122779
Tests in T-U seem to have been resolved where there is the most delay.
Adding the following line will force the tests to use the overloaded methods.
using Debug = Lucene.Net.Diagnostics.Debug;
assertEquals in LuceneTestCase has also been modified to use the same equivalence checks
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 commented on issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
NightOwl888 commented on issue #295:
URL: https://github.com/apache/lucenenet/issues/295#issuecomment-648164251
@bongohrtech - Good find. There are several tests that are passing value types through `Assert.AreEqual(object, object)` in a tight loop, causing lots of box/unbox operations. And of course, this problem isn't limited to the `Lucene.Net.Util` namespace.
I have confirmed that changing this gets both T-U and U-Z projects to run in under 3.5 minutes, which were more like 11 minutes before.
We should aim to apply this fix on all of the projects starting with the longest running ones:
- `Lucene.Net.Tests._E-I`
- `Lucene.Net.Tests.Analysis.Common`
- `Lucene.Net.Tests.Codecs`
- `Lucene.Net.Tests._A-D`
- `Lucene.Net.Tests._J-S`
We can probably also consolidate some of the 5 Lucene.Net test projects that were broken up, which should help to speed up the overall time in Azure DevOps.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] bongohrtech commented on issue #295: Investigate Slow Tests in Lucene.Net.Util Namespace
Posted by GitBox <gi...@apache.org>.
bongohrtech commented on issue #295:
URL: https://github.com/apache/lucenenet/issues/295#issuecomment-648119529
I believe NUnit should have an overload for numeric types but when comparing type long this seems to take a very long time to resolve. Using an overload a == b seems to result in much faster performance. The same is true for byte comparison.
This as been added to the Assert.cs file which overloads AreEqual for long and byte
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org