You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by GitBox <gi...@apache.org> on 2021/01/28 21:04:22 UTC
[GitHub] [lucenenet] rclabo opened a new pull request #404: Implements [LUCENE-5889]
rclabo opened a new pull request #404:
URL: https://github.com/apache/lucenenet/pull/404
Implements [LUCENE-5889](https://github.com/apache/lucene-solr/commit/4a485d68771e5e123139cc12e9ea2733a922baea#diff-96baf6cc4649d2c8d0aa5423587cf3aa3bf82adca5c953d1c9fbbba2575aca6f)
A version 5.0 feature later backported to 4.11.0
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 commented on a change in pull request #404: Implements [LUCENE-5889]
Posted by GitBox <gi...@apache.org>.
NightOwl888 commented on a change in pull request #404:
URL: https://github.com/apache/lucenenet/pull/404#discussion_r566487555
##########
File path: src/Lucene.Net.Suggest/Suggest/Analyzing/AnalyzingInfixSuggester.cs
##########
@@ -246,6 +276,16 @@ public override void Build(IInputEnumerator enumerator)
}
}
+ //LUCENENET specific -Support for LUCENE - 5889.
+ public void Commit()
+ {
+ if (writer == null)
+ {
+ throw new InvalidOperationException("Cannot commit on an closed writer. Add documents first");
Review comment:
Please indent 4 spaces (and update `.editorconfig` if it is possible to enforce this).
##########
File path: CHANGES.txt
##########
@@ -1,4 +1,11 @@
-=================== Release 4.8.0-beta00006 =====================
+=================== Release 4.8.0-beta00014 =====================
Review comment:
Please revert the changes to this file.
Since 4.8.0-beta00007, we have been using [GitHub releases](https://github.com/apache/lucenenet/releases) officially (using Markdown rather than plain text). This file is basically a carry-over from Java that is intentionally no longer being kept up to date.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -50,7 +51,7 @@ public void TestBasic()
};
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3);
+ using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -93,14 +94,14 @@ public void TestAfterLoad()
DirectoryInfo tempDir = CreateTempDir("AnalyzingInfixSuggesterTest");
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -93,14 +94,14 @@ public void TestAfterLoad()
DirectoryInfo tempDir = CreateTempDir("AnalyzingInfixSuggesterTest");
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
try
{
suggester.Build(new InputArrayEnumerator(keys));
assertEquals(2, suggester.Count);
suggester.Dispose();
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Suggest/Suggest/Analyzing/AnalyzingInfixSuggester.cs
##########
@@ -282,6 +322,29 @@ protected override TokenStreamComponents WrapComponents(string fieldName, TokenS
}
}
+ //LUCENENET specific -Support for LUCENE - 5889.
+ //use of lock as substitute for Java's synchronized keyword on method
+ private void EnsureOpen()
+ {
+ if (writer != null)
+ return;
+
+ lock (openLock)
+ {
+ if (writer == null)
+ {
+ if (m_searcherMgr != null)
+ {
+ m_searcherMgr.Dispose();
+ m_searcherMgr = null;
+ }
+ writer = new IndexWriter(dir, GetIndexWriterConfig(matchVersion, GetGramAnalyzer(), OpenMode.CREATE));
+ m_searcherMgr = new SearcherManager(writer, true, null);
+ }
+ }
+ }
+ private object openLock = new object();
Review comment:
It is a good thing to use the best practice of locking on a privately declared object. However, keep in mind that the `synchronized` keyword in Java locks on `this` so any additional future uses of the keyword will need to share the same lock.
Therefore, please name this lock object generically (we have been using the name `syncLock` by convention) and move it to the top of the class where private fields are declared.
Also, please declare the field `readonly`.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -328,7 +329,7 @@ public void TestRandomMinPrefixLength()
// Make sure things still work after close and reopen:
suggester.Dispose();
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixLength);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixLength); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -143,7 +144,7 @@ public override string ToString()
internal class TestHighlightAnalyzingInfixSuggester : AnalyzingInfixSuggester
{
public TestHighlightAnalyzingInfixSuggester(AnalyzingInfixSuggesterTest outerInstance, Analyzer a)
- : base(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3)
+ : base(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3) //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -252,7 +253,7 @@ public void TestRandomMinPrefixLength()
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
int minPrefixLength = Random.nextInt(10);
- AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixLength);
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixLength); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -346,7 +347,7 @@ public void TestHighlight()
};
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3);
+ using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -607,7 +608,7 @@ public void TestRandomNRT()
Console.WriteLine(" minPrefixChars=" + minPrefixChars);
}
- AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixChars);
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixChars); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -379,7 +380,7 @@ public void TestHighlightCaseChange()
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, true);
IList<Lookup.LookupResult> results;
- using (AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3))
+ using (AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3)) //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -357,7 +358,7 @@ internal class TestHighlightChangeCaseAnalyzingInfixSuggester : AnalyzingInfixSu
{
private readonly AnalyzingInfixSuggesterTest outerInstance;
public TestHighlightChangeCaseAnalyzingInfixSuggester(AnalyzingInfixSuggesterTest outerInstance, Analyzer a)
- : base(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3)
+ : base(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3) //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -481,7 +482,7 @@ public void TestEmptyAtStart()
public void TestBothExactAndPrefix()
{
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3);
+ using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -443,7 +444,7 @@ public void TestSuggestStopFilter()
public void TestEmptyAtStart()
{
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3);
+ using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -710,7 +711,7 @@ public void TestRandomNRT()
lookupThread.Join();
Assert.Null(error[0], "Unexpcted exception at retry : \n" + stackTraceStr(error[0]));
suggester.Dispose();
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixChars);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, minPrefixChars); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/BlendedInfixSuggesterTest.cs
##########
@@ -184,8 +187,9 @@ public void TestTrying()
// if factor is small, we don't get the expected element
BlendedInfixSuggester suggester = new BlendedInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a,
- AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS, BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL,
- BlendedInfixSuggester.DEFAULT_NUM_FACTOR);
+ AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
+ BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL,
+ BlendedInfixSuggester.DEFAULT_NUM_FACTOR); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -917,7 +918,7 @@ public void TestBasicNRT()
};
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
- using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3);
+ using AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewDirectory(), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/BlendedInfixSuggesterTest.cs
##########
@@ -49,7 +49,7 @@ public void TestBlendedSort()
BlendedInfixSuggester suggester = new BlendedInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a,
AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
BlendedInfixSuggester.BlenderType.POSITION_LINEAR,
- BlendedInfixSuggester.DEFAULT_NUM_FACTOR);
+ BlendedInfixSuggester.DEFAULT_NUM_FACTOR); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/BlendedInfixSuggesterTest.cs
##########
@@ -132,7 +133,8 @@ public void TestRequiresMore()
// if factor is small, we don't get the expected element
BlendedInfixSuggester suggester = new BlendedInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a,
- AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS, BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 1);
+ AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
+ BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 1); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/BlendedInfixSuggesterTest.cs
##########
@@ -99,7 +99,8 @@ public void TestBlendingType()
// BlenderType.RECIPROCAL is using 1/(1+p) * w where w is weight and p the position of the word
suggester = new BlendedInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a,
- AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS, BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 1);
+ AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
+ BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 1); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
Review comment:
Please move the curly bracket to the following line.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/BlendedInfixSuggesterTest.cs
##########
@@ -151,7 +153,8 @@ public void TestRequiresMore()
// if we increase the factor we have it
suggester = new BlendedInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a,
- AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS, BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 2);
+ AnalyzingInfixSuggester.DEFAULT_MIN_PREFIX_CHARS,
+ BlendedInfixSuggester.BlenderType.POSITION_RECIPROCAL, 2); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ string key = keys[i];
+ multiAddThreads[i] = new Thread(() => IndexDocument(suggester, key)); // LUCENENET specific: use of closure rather than object.
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ multiAddThreads[i].Start();
+ }
+
+ //Make sure all threads have completed indexing
+ for(int i=0; i<10; i++) {
Review comment:
Please space out the for block properly (usually cutting and pasting the entire contents of the file will fix it).
For example `for (int i = 0; i < 10; i++)`
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -1006,13 +1071,13 @@ public void TestBasicContext()
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
if (iter == 0)
{
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
suggester.Build(new InputArrayEnumerator(keys));
}
else
{
// Test again, after close/reopen:
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
Review comment:
Please move the curly bracket to the following line.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
Review comment:
Please fix the build warning on unused variable `e`. In general, we just suppress them in tests but in production code we comment or remove the variable.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ string key = keys[i];
+ multiAddThreads[i] = new Thread(() => IndexDocument(suggester, key)); // LUCENENET specific: use of closure rather than object.
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ multiAddThreads[i].Start();
+ }
+
+ //Make sure all threads have completed indexing
+ for(int i=0; i<10; i++) {
+ multiAddThreads[i].Join();
+ }
+
+ suggester.Refresh();
+ IList<LookupResult> results = suggester.DoLookup(TestUtil.StringToCharSequence("python", Random).ToString(), 10, true, false);
+ assertEquals(1, results.size());
+ assertEquals("python", results[0].Key);
+
+ //Test if the index is getting persisted correctly and can be reopened.
+ suggester.Commit();
+ suggester.Dispose();
+
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ results = suggester.DoLookup(TestUtil.StringToCharSequence("python", Random).ToString(), 10, true, false);
+ assertEquals(1, results.size());
+ assertEquals("python", results[0].Key);
+
+ suggester.Dispose();
+ }
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature. Use of method not class due to Thread parameter needs in .Net.
+ public void IndexDocument(AnalyzingInfixSuggester suggester, String key) {
+ try
+ {
+ suggester.Add(new BytesRef(key), null, 10, null);
+ }
+ catch (IOException e) {
Review comment:
Please fix the build warning on unused variable `e`. In general, we just suppress them in tests but in production code we comment or remove the variable.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ string key = keys[i];
+ multiAddThreads[i] = new Thread(() => IndexDocument(suggester, key)); // LUCENENET specific: use of closure rather than object.
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ multiAddThreads[i].Start();
+ }
+
+ //Make sure all threads have completed indexing
+ for(int i=0; i<10; i++) {
Review comment:
Please move the curly bracket to the following line.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -1006,13 +1071,13 @@ public void TestBasicContext()
Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
if (iter == 0)
{
- suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3);
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3); //TODO: add extra false param at version 4.11.0
Review comment:
Since Lucene uses the convention `// TODO:` when commenting lines, we are using `// LUCENENET TODO:` to keep them separate. Please change this.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ string key = keys[i];
+ multiAddThreads[i] = new Thread(() => IndexDocument(suggester, key)); // LUCENENET specific: use of closure rather than object.
+ }
+
+ for(int i=0; i<10; i++)
Review comment:
Please space out the for block properly (usually cutting and pasting the entire contents of the file will fix it).
For example `for (int i = 0; i < 10; i++)`
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ string key = keys[i];
+ multiAddThreads[i] = new Thread(() => IndexDocument(suggester, key)); // LUCENENET specific: use of closure rather than object.
+ }
+
+ for(int i=0; i<10; i++)
+ {
+ multiAddThreads[i].Start();
+ }
+
+ //Make sure all threads have completed indexing
+ for(int i=0; i<10; i++) {
+ multiAddThreads[i].Join();
+ }
+
+ suggester.Refresh();
+ IList<LookupResult> results = suggester.DoLookup(TestUtil.StringToCharSequence("python", Random).ToString(), 10, true, false);
+ assertEquals(1, results.size());
+ assertEquals("python", results[0].Key);
+
+ //Test if the index is getting persisted correctly and can be reopened.
+ suggester.Commit();
+ suggester.Dispose();
+
+ suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ results = suggester.DoLookup(TestUtil.StringToCharSequence("python", Random).ToString(), 10, true, false);
+ assertEquals(1, results.size());
+ assertEquals("python", results[0].Key);
+
+ suggester.Dispose();
+ }
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature. Use of method not class due to Thread parameter needs in .Net.
+ public void IndexDocument(AnalyzingInfixSuggester suggester, String key) {
+ try
+ {
+ suggester.Add(new BytesRef(key), null, 10, null);
+ }
+ catch (IOException e) {
Review comment:
Please move the curly bracket to the next line.
##########
File path: src/Lucene.Net.Tests.Suggest/Suggest/Analyzing/AnalyzingInfixSuggesterTest.cs
##########
@@ -976,6 +977,70 @@ public void TestBasicNRT()
assertEquals(new BytesRef("foobaz"), results[1].Payload);
}
+
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature.
+ [Test]
+ public void TestNRTWithParallelAdds(){
+ String[] keys = new String[] {"python", "java", "c", "scala", "ruby", "clojure", "erlang", "go", "swift", "lisp"};
+ Analyzer a = new MockAnalyzer(Random, MockTokenizer.WHITESPACE, false);
+ DirectoryInfo tempDir = CreateTempDir("AIS_NRT_PERSIST_TEST");
+ AnalyzingInfixSuggester suggester = new AnalyzingInfixSuggester(TEST_VERSION_CURRENT, NewFSDirectory(tempDir), a, a, 3, false);
+ Thread[] multiAddThreads = new Thread[10];
+
+ try
+ {
+ suggester.Refresh();
+ fail("Cannot call refresh on an suggester when no docs are added to the index");
+ }
+ catch(InvalidOperationException e) {
+ //Expected
+ }
+
+ for(int i=0; i<10; i++)
Review comment:
Please space out the for block properly (usually cutting and pasting the entire contents of the file will fix it).
For example `for (int i = 0; i < 10; i++)`
##########
File path: src/Lucene.Net.Suggest/Suggest/Analyzing/AnalyzingInfixSuggester.cs
##########
@@ -133,8 +134,33 @@ public AnalyzingInfixSuggester(LuceneVersion matchVersion, Directory dir, Analyz
/// Prefixes shorter than this are indexed as character
/// ngrams (increasing index size but making lookups
/// faster). </param>
- public AnalyzingInfixSuggester(LuceneVersion matchVersion, Directory dir, Analyzer indexAnalyzer,
+ // LUCENENET specific - LUCENE-5889, a 4.11.0 feature. calls new constructor with extra param.
+ // TODO: Remove method at version 4.11.0. Was retained for perfect 4.8 compatibility
+ public AnalyzingInfixSuggester(LuceneVersion matchVersion, Directory dir, Analyzer indexAnalyzer,
Analyzer queryAnalyzer, int minPrefixChars)
+ : this(matchVersion, dir, indexAnalyzer, queryAnalyzer, minPrefixChars, commitOnBuild:false)
Review comment:
Please add a space between `commitOnBuild:` and `false`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] rclabo commented on a change in pull request #404: Implements [LUCENE-5889]
Posted by GitBox <gi...@apache.org>.
rclabo commented on a change in pull request #404:
URL: https://github.com/apache/lucenenet/pull/404#discussion_r566830965
##########
File path: CHANGES.txt
##########
@@ -1,4 +1,11 @@
-=================== Release 4.8.0-beta00006 =====================
+=================== Release 4.8.0-beta00014 =====================
Review comment:
Good to know. Should I move my release note to the GitHub releases and if so where do I do that? Or is that done when you create the new release by looking through the commits for the release and summarizing them then?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 commented on a change in pull request #404: Implements [LUCENE-5889]
Posted by GitBox <gi...@apache.org>.
NightOwl888 commented on a change in pull request #404:
URL: https://github.com/apache/lucenenet/pull/404#discussion_r566912377
##########
File path: CHANGES.txt
##########
@@ -1,4 +1,11 @@
-=================== Release 4.8.0-beta00006 =====================
+=================== Release 4.8.0-beta00014 =====================
Review comment:
If you are curious, the [release procedure](https://lucenenet.apache.org/contributing/make-release.html#release-steps) describes the whole process. Basically, we go through the commit log and PRs to summarize the release at that time.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [lucenenet] NightOwl888 merged pull request #404: Implements [LUCENE-5889]
Posted by GitBox <gi...@apache.org>.
NightOwl888 merged pull request #404:
URL: https://github.com/apache/lucenenet/pull/404
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org