You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@lucenenet.apache.org by ni...@apache.org on 2019/08/09 19:50:01 UTC
[lucenenet] branch master updated (885d5b9 -> 7441677)
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git.
from 885d5b9 Lucene.Net.Analysis.Common.Analysis.Synonym.FSTSynonymFilterFactory: Commented unused exception variable declaration to suppress the warning.
new 9c4a3c3 Lucene.Net.ICU.PostingsHighlight.PostingsHighlighter: Changed default BreakIterator implementation to be the ICU default rather than using a customized RuleBasedBreakIterator that was built just to make the tests pass. Renamed ICUPostingsFormatter to reflect the change and be consistent with other ICU classes. Confirmed against Java that this is the correct default behavior and created separate ICU-specific tests. Moved the JdkBreakIterator to Lucene.Net.ICU.Tests becaus [...]
new 50623b6 Lucene.Net.Tests.ICU.Search.VectorHighlight.BreakIteratorBoundaryScannerTest: Modified original TestSentenceBoundary test to skip east Asian languages because the mock doesn't override them. Added TestICUWordBoundary and TestICUSentenceBoundary tests to confirm that the boundary scanner works the same with ICU4N as it does with ICU4J with default settings.
new bf9771e Lucene.Net.Highlighter.PostingsHighlight.LoadFieldValues: Changed return type from string[][] to IList<string[]> (cleaner API)
new fde5bdc build/Dependencies.props: Added reminder to lock down the ICU4N package dependency version when it is stable. The Lucene.Net.ICU resources are not compatible with versions of ICU other than 60.1.
new 870a366 Added CHANGES.txt, CONTRIBUTING.md, LICENSE.txt, NOTICE.txt, and README.md to solution items
new 3606b91 Updated README with lucene-cli info
new dbdadab Added missing license headers
new 48460f0 Added NOTICE.txt to each NuGet package as per http://www.apache.org/legal/release-policy.html#licensing-documentation
new 7441677 Upgraded ICU4N to 60.1.0-alpha.53
The 9 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
Lucene.Net.sln | 5 +
README.md | 21 +-
build/Dependencies.props | 13 +-
build/NuGet.props | 2 +
.../Support/TagSoup/HTMLSchema.Generated.cs | 22 +-
.../Support/TagSoup/HTMLSchema.tt | 4 +-
.../PostingsHighlight/MultiTermHighlighting.cs | 2 +-
.../PostingsHighlight/PassageFormatter.cs | 2 +-
.../PostingsHighlight/PassageScorer.cs | 2 +-
.../PostingsHighlight/PostingsHighlighter.cs | 33 +--
.../PostingsHighlight/TestMultiTermHighlighting.cs | 161 +--------------
.../PostingsHighlight/TestPostingsHighlighter.cs | 19 +-
.../TestPostingsHighlighterRanking.cs | 44 +---
.../BreakIteratorBoundaryScannerTest.cs | 60 +++++-
src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln | 18 ++
.../LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs | 21 +-
.../LuceneDocsPlugins/LuceneDocsPlugins.csproj | 20 ++
.../LuceneDocsPlugins/LuceneNoteBlockRule.cs | 19 ++
.../LuceneDocsPlugins/LuceneNoteBlockToken.cs | 19 ++
.../LuceneRendererPartProvider.cs | 21 +-
.../LuceneDocsPlugins/LuceneTokenRendererPart.cs | 19 ++
.../LuceneDocsPlugins/Properties/AssemblyInfo.cs | 21 +-
.../LuceneDocsPlugins/packages.config | 20 ++
.../Lucene.Net.Tests.ICU.csproj | 4 +
.../TestICUMultiTermHighlighting.cs} | 222 ++++-----------------
.../TestICUPostingsHighlighter.cs} | 143 +++++++------
.../TestICUPostingsHighlighterRanking.cs} | 61 ++----
.../Support}/Data/jdk7/jdksent.rbbi | 0
.../Support}/Data/jdk7/jdkword.rbbi | 0
.../Support/JDKBreakIterator.cs | 20 +-
.../Support/PostingsHighlighter.cs} | 30 +--
.../Support/TestJDKBreakIterator.cs | 27 ++-
.../Support/jdksent.brk | Bin
.../Support/jdkword.brk | Bin
.../Formatters/CodeLinkReplacer.cs | 21 +-
.../Formatters/DocTypeReplacer.cs | 21 +-
.../Formatters/ExtraHtmlElementReplacer.cs | 21 +-
.../Formatters/IReplacer.cs | 21 +-
.../Formatters/JavaDocFormatters.cs | 21 +-
.../Formatters/PatternReplacer.cs | 21 +-
.../Formatters/RepoLinkReplacer.cs | 21 +-
.../JavaDocToMarkdownConverter/StringExtensions.cs | 21 +-
src/dotnet/tools/lucene-cli/docs/index.md | 4 +-
websites/apidocs/lucenetemplate/styles/main.css | 19 ++
websites/apidocs/lucenetemplate/styles/main.js | 19 ++
websites/apidocs/lucenetemplate/web.config | 20 ++
websites/site/lucenetemplate/styles/main.css | 19 ++
websites/site/lucenetemplate/styles/site.css | 19 ++
websites/site/lucenetemplate/web.config | 20 ++
49 files changed, 826 insertions(+), 537 deletions(-)
copy src/{Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs => dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUMultiTermHighlighting.cs} (81%)
copy src/{Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs => dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs} (91%)
copy src/{Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs => dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighterRanking.cs} (87%)
rename src/dotnet/{Lucene.Net.ICU => Lucene.Net.Tests.ICU/Support}/Data/jdk7/jdksent.rbbi (100%)
rename src/dotnet/{Lucene.Net.ICU => Lucene.Net.Tests.ICU/Support}/Data/jdk7/jdkword.rbbi (100%)
rename src/dotnet/{Lucene.Net.ICU => Lucene.Net.Tests.ICU}/Support/JDKBreakIterator.cs (84%)
copy src/{Lucene.Net.Benchmark/ByTask/Tasks/ClearStatsTask.cs => dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs} (54%)
rename src/dotnet/{Lucene.Net.ICU => Lucene.Net.Tests.ICU}/Support/jdksent.brk (100%)
rename src/dotnet/{Lucene.Net.ICU => Lucene.Net.Tests.ICU}/Support/jdkword.brk (100%)
[lucenenet] 09/09: Upgraded ICU4N to 60.1.0-alpha.53
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 74416777808ec50351a1c49920fe4d4fd7b4b795
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Sat Aug 10 02:47:07 2019 +0700
Upgraded ICU4N to 60.1.0-alpha.53
---
build/Dependencies.props | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/build/Dependencies.props b/build/Dependencies.props
index b7a63b7..edde3a8 100644
--- a/build/Dependencies.props
+++ b/build/Dependencies.props
@@ -32,7 +32,7 @@
https://github.com/apache/lucene-solr/tree/31d7ec7bbfdcd2c4cc61d9d35e962165410b65fe/lucene/analysis/icu/src/data/utr30
Just make sure they are adjusted to the right version of ICU/Lucene.
<ICU4NPackageVersion>[60.1,60.2)</ICU4NPackageVersion> -->
- <ICU4NPackageVersion>60.1.0-alpha.52</ICU4NPackageVersion>
+ <ICU4NPackageVersion>60.1.0-alpha.53</ICU4NPackageVersion>
<ICU4NCollationPackageVersion>$(ICU4NPackageVersion)</ICU4NCollationPackageVersion>
<ICU4NCurrencyDataPackageVersion>$(ICU4NPackageVersion)</ICU4NCurrencyDataPackageVersion>
<ICU4NLanguageDataPackageVersion>$(ICU4NPackageVersion)</ICU4NLanguageDataPackageVersion>
[lucenenet] 04/09: build/Dependencies.props: Added reminder to lock
down the ICU4N package dependency version when it is stable. The
Lucene.Net.ICU resources are not compatible with versions of ICU other than
60.1.
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit fde5bdc2664f551a1df5fefbfc9d650bab2eb7d6
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 19:54:06 2019 +0700
build/Dependencies.props: Added reminder to lock down the ICU4N package dependency version when it is stable. The Lucene.Net.ICU resources are not compatible with versions of ICU other than 60.1.
---
build/Dependencies.props | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/build/Dependencies.props b/build/Dependencies.props
index 397eea7..b7a63b7 100644
--- a/build/Dependencies.props
+++ b/build/Dependencies.props
@@ -21,6 +21,17 @@
<Project>
<PropertyGroup Label="NuGet Package Reference Versions">
<Antlr3RuntimePackageVersion>3.5.1</Antlr3RuntimePackageVersion>
+ <!-- LUCENENET TODO: When ICU4N is released to production,
+ be sure to lock down the version range below. The resource
+ files in Lucene.Net.ICU are not compatible with any other
+ ICU version than 60.1!! This should only change if ICU4N
+ is upgraded AND the resources in Lucene.Net.ICU are upgraded to match.
+ Note that some of the files need to be generated.
+ The resource files to sync include .nrm files, .brk files, and .rbbi files.
+ They can be generated based off of the files here:
+ https://github.com/apache/lucene-solr/tree/31d7ec7bbfdcd2c4cc61d9d35e962165410b65fe/lucene/analysis/icu/src/data/utr30
+ Just make sure they are adjusted to the right version of ICU/Lucene.
+ <ICU4NPackageVersion>[60.1,60.2)</ICU4NPackageVersion> -->
<ICU4NPackageVersion>60.1.0-alpha.52</ICU4NPackageVersion>
<ICU4NCollationPackageVersion>$(ICU4NPackageVersion)</ICU4NCollationPackageVersion>
<ICU4NCurrencyDataPackageVersion>$(ICU4NPackageVersion)</ICU4NCurrencyDataPackageVersion>
[lucenenet] 05/09: Added CHANGES.txt, CONTRIBUTING.md, LICENSE.txt,
NOTICE.txt, and README.md to solution items
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 870a366f5c396843a7971c45bbf6a0723c7d1875
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 20:06:43 2019 +0700
Added CHANGES.txt, CONTRIBUTING.md, LICENSE.txt, NOTICE.txt, and README.md to solution items
---
Lucene.Net.sln | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Lucene.Net.sln b/Lucene.Net.sln
index bde08f5..709b0ff 100644
--- a/Lucene.Net.sln
+++ b/Lucene.Net.sln
@@ -69,8 +69,13 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "apidocs", "apidocs", "{58FD
EndProject
Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Solution Items", "Solution Items", "{4DF0A2A1-B9C7-4EE5-BAF0-BEEF53E34220}"
ProjectSection(SolutionItems) = preProject
+ CHANGES.txt = CHANGES.txt
+ CONTRIBUTING.md = CONTRIBUTING.md
Directory.Build.props = Directory.Build.props
Directory.Build.targets = Directory.Build.targets
+ LICENSE.txt = LICENSE.txt
+ NOTICE.txt = NOTICE.txt
+ README.md = README.md
EndProjectSection
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "Lucene.Net", "src\Lucene.Net\Lucene.Net.csproj", "{3A0AA37E-2B7B-4416-B528-DA4E0E6A6706}"
[lucenenet] 06/09: Updated README with lucene-cli info
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 3606b914b8fbec7c3d546c298721e6141605eea2
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 21:12:15 2019 +0700
Updated README with lucene-cli info
---
README.md | 21 ++++++++++++++++-----
src/dotnet/tools/lucene-cli/docs/index.md | 4 ++--
2 files changed, 18 insertions(+), 7 deletions(-)
diff --git a/README.md b/README.md
index 4ce4667..ff17d89 100644
--- a/README.md
+++ b/README.md
@@ -86,12 +86,10 @@ PM> Install-Package Lucene.Net -Pre
- [Lucene.Net.Replicator](https://www.nuget.org/packages/Lucene.Net.Replicator/) Files replication utility
- [Lucene.Net.Sandbox](https://www.nuget.org/packages/Lucene.Net.Sandbox/) - Various third party contributions and new ideas
- [Lucene.Net.Spatial](https://www.nuget.org/packages/Lucene.Net.Spatial/) - Geospatial search
-- [Lucene.Net.Suggest](https://www.nuget.org/packages/Lucene.Net.Suggest/) - Auto-suggest and Spellchecking support
+- [Lucene.Net.Suggest](https://www.nuget.org/packages/Lucene.Net.Suggest/) - Auto-suggest and Spell-checking support
## Documentation
-[Lucene.Net WIKI](https://cwiki.apache.org/confluence/display/LUCENENET/Lucene.Net)
-
We now have some preliminary documentation for Lucene.Net 4.8.0 [on the Lucene.Net Website](https://lucenenet.apache.org/).
The API is similar to Java [Lucene 4.8.0](https://lucene.apache.org/core/4_8_0/), which you may also find helpful to review.
@@ -103,10 +101,23 @@ The API is similar to Java [Lucene 4.8.0](https://lucene.apache.org/core/4_8_0/)
- [Lucene.Net 3.0.3 API Documentation](http://incubator.apache.org/lucene.net/docs/3.0.3/Index.html)
- [Lucene.Net 2.9.4 API Documentation](http://incubator.apache.org/lucene.net/docs/2.9.4/Index.html)
-## Demos
+## Demos & Tools
There are several demos implemented as simple console applications that can be copied and pasted into Visual Studio or compiled on the command line in the [Lucene.Net.Demo project](https://github.com/apache/lucenenet/tree/master/src/Lucene.Net.Demo).
+<!-- TO BE ADDED WHEN RELEASED
+
+There is also a dotnet command line tool available on NuGet. It contains all of the demos as well as tools maintaining your Lucene.Net index, containing such operations as splitting, merging, listing segment info, fixing, deleting segments, upgrading, etc. Always be sure to back up your index before running any commands against it!
+
+```
+dotnet tool install lucene-cli -g --version 4.8.0-beta00006
+```
+
+Once installed, you can explore what commands and options are available by entering the command `lucene`.
+
+[lucene-cli Documentation](https://github.com/apache/lucenenet/blob/master/src/dotnet/tools/lucene-cli/docs/index.md)
+-->
+
## How to Contribute
Lucene.Net is a very large project (over 400,000 executable lines of code and nearly 1,000,000 lines of text total) and we welcome any and all help to maintain such an effort. Read our [Contribution Guide](https://github.com/apache/lucenenet/blob/master/CONTRIBUTING.md) or read on for ways that you can help.
@@ -149,7 +160,7 @@ If you plan to submit multiple pull requests, please submit an [Individual Contr
### Command Line
-Building on the Command Line is only supported on Windows.
+Building on the Command Line is currently only supported on Windows.
##### Prerequisites
diff --git a/src/dotnet/tools/lucene-cli/docs/index.md b/src/dotnet/tools/lucene-cli/docs/index.md
index 8dc35f6..06f3971 100644
--- a/src/dotnet/tools/lucene-cli/docs/index.md
+++ b/src/dotnet/tools/lucene-cli/docs/index.md
@@ -1,4 +1,4 @@
-# Lucene.Net command line interface (CLI) tools
+# Lucene.Net command line interface (CLI) tools
The Lucene.Net command line interface (CLI) is a new cross-platform toolchain with utilities for maintaining Lucene.Net and demos for learning basic Lucene.Net functionality.
@@ -11,7 +11,7 @@ The Lucene.Net command line interface (CLI) is a new cross-platform toolchain wi
Perform a one-time install of the lucene-cli tool using the following dotnet CLI command:
```
-dotnet tool install -g lucene-cli
+dotnet tool install lucene-cli -g --version 4.8.0-beta00006
```
You may then use the lucene-cli tool to analyze and update Lucene.Net indexes and use its demos.
[lucenenet] 03/09:
Lucene.Net.Highlighter.PostingsHighlight.LoadFieldValues: Changed return
type from string[][] to IList (cleaner API)
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit bf9771e5ee921c28a27e43b1b91c44a90fa1ad2d
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 17:41:52 2019 +0700
Lucene.Net.Highlighter.PostingsHighlight.LoadFieldValues: Changed return type from string[][] to IList<string[]> (cleaner API)
---
src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs | 6 +++---
.../PostingsHighlight/TestPostingsHighlighter.cs | 2 +-
.../Search/PostingsHighlight/TestICUPostingsHighlighter.cs | 2 +-
3 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs b/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
index 58a5d7e..e8e20f3 100644
--- a/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
+++ b/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
@@ -408,7 +408,7 @@ namespace Lucene.Net.Search.PostingsHighlight
new InPlaceMergeSorterAnonymousHelper(fields, maxPassages).Sort(0, fields.Length);
// pull stored data:
- string[][] contents = LoadFieldValues(searcher, fields, docids, maxLength);
+ IList<string[]> contents = LoadFieldValues(searcher, fields, docids, maxLength);
IDictionary<string, object[]> highlights = new Dictionary<string, object[]>();
for (int i = 0; i < fields.Length; i++)
@@ -447,7 +447,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// and fill all values. The returned strings must be
/// identical to what was indexed.
/// </summary>
- protected virtual string[][] LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
+ protected virtual IList<string[]> LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
{
string[][] contents = RectangularArrays.ReturnRectangularArray<string>(fields.Length, docids.Length);
char[] valueSeparators = new char[fields.Length];
@@ -461,7 +461,7 @@ namespace Lucene.Net.Search.PostingsHighlight
searcher.Doc(docids[i], visitor);
for (int j = 0; j < fields.Length; j++)
{
- contents[j][i] = visitor.GetValue(j).ToString();
+ contents[j][i] = visitor.GetValue(j); // LUCENENET: No point in doing ToString() on a string
}
visitor.Reset();
}
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
index 1b24a8c..9235b1a 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
+++ b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
@@ -755,7 +755,7 @@ namespace Lucene.Net.Search.PostingsHighlight
this.text = text;
}
- protected override string[][] LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
+ protected override IList<string[]> LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
{
Debug.Assert(fields.Length == 1);
Debug.Assert(docids.Length == 1);
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
index b5e1f47..242f96e 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
@@ -755,7 +755,7 @@ namespace Lucene.Net.Search.PostingsHighlight
this.text = text;
}
- protected override string[][] LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
+ protected override IList<string[]> LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
{
Debug.Assert( fields.Length == 1);
Debug.Assert( docids.Length == 1);
[lucenenet] 07/09: Added missing license headers
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit dbdadabbc38ebe742d8ec8162ca034fab92c935c
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 21:42:41 2019 +0700
Added missing license headers
---
.../Support/TagSoup/HTMLSchema.Generated.cs | 22 +++++++++++++++++++++-
.../Support/TagSoup/HTMLSchema.tt | 4 ++--
src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln | 18 ++++++++++++++++++
.../LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs | 21 ++++++++++++++++++++-
.../LuceneDocsPlugins/LuceneDocsPlugins.csproj | 20 ++++++++++++++++++++
.../LuceneDocsPlugins/LuceneNoteBlockRule.cs | 19 +++++++++++++++++++
.../LuceneDocsPlugins/LuceneNoteBlockToken.cs | 19 +++++++++++++++++++
.../LuceneRendererPartProvider.cs | 21 ++++++++++++++++++++-
.../LuceneDocsPlugins/LuceneTokenRendererPart.cs | 19 +++++++++++++++++++
.../LuceneDocsPlugins/Properties/AssemblyInfo.cs | 21 ++++++++++++++++++++-
.../LuceneDocsPlugins/packages.config | 20 ++++++++++++++++++++
.../Support/JDKBreakIterator.cs | 17 +++++++++++++++++
.../Support/PostingsHighlighter.cs | 17 +++++++++++++++++
.../Support/TestJDKBreakIterator.cs | 17 +++++++++++++++++
.../Formatters/CodeLinkReplacer.cs | 21 ++++++++++++++++++++-
.../Formatters/DocTypeReplacer.cs | 21 ++++++++++++++++++++-
.../Formatters/ExtraHtmlElementReplacer.cs | 21 ++++++++++++++++++++-
.../Formatters/IReplacer.cs | 21 ++++++++++++++++++++-
.../Formatters/JavaDocFormatters.cs | 21 ++++++++++++++++++++-
.../Formatters/PatternReplacer.cs | 21 ++++++++++++++++++++-
.../Formatters/RepoLinkReplacer.cs | 21 ++++++++++++++++++++-
.../JavaDocToMarkdownConverter/StringExtensions.cs | 21 ++++++++++++++++++++-
websites/apidocs/lucenetemplate/styles/main.css | 19 +++++++++++++++++++
websites/apidocs/lucenetemplate/styles/main.js | 19 +++++++++++++++++++
websites/apidocs/lucenetemplate/web.config | 20 ++++++++++++++++++++
websites/site/lucenetemplate/styles/main.css | 19 +++++++++++++++++++
websites/site/lucenetemplate/styles/site.css | 19 +++++++++++++++++++
websites/site/lucenetemplate/web.config | 20 ++++++++++++++++++++
28 files changed, 525 insertions(+), 14 deletions(-)
diff --git a/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.Generated.cs b/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.Generated.cs
index 6f9c539..21a20d3 100644
--- a/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.Generated.cs
+++ b/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.Generated.cs
@@ -1,4 +1,24 @@
-//------------------------------------------------------------------------------
+/*
+# -----------------------------------------------------------------------------------
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements. See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the ""License""); you may not use this file except in compliance with
+# the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an ""AS IS"" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+# -----------------------------------------------------------------------------------
+*/
+//------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
//
diff --git a/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.tt b/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.tt
index 5f4a839..a4253de 100644
--- a/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.tt
+++ b/src/Lucene.Net.Benchmark/Support/TagSoup/HTMLSchema.tt
@@ -3,7 +3,7 @@
<#@ import namespace="System.IO" #>
<#@ import namespace="System.Xml.Xsl" #>
<#@ output extension=".Generated.cs" #>
-<# /*
+/*
# -----------------------------------------------------------------------------------
#
# Licensed to the Apache Software Foundation (ASF) under one or more
@@ -22,7 +22,7 @@
# limitations under the License.
#
# -----------------------------------------------------------------------------------
-*/ #>
+*/
//------------------------------------------------------------------------------
// <auto-generated>
// This code was generated by a tool.
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln
index 99a8fbb..b2979b1 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins.sln
@@ -1,6 +1,24 @@
Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio 15
+
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
VisualStudioVersion = 15.0.26430.14
MinimumVisualStudioVersion = 10.0.40219.1
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "LuceneDocsPlugins", "LuceneDocsPlugins\LuceneDocsPlugins.csproj", "{D5D1C256-4A5A-4C57-949D-E9A1FFB6A5D1}"
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs
index 3da552a..f21c2b1 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDfmEngineCustomizer.cs
@@ -1,4 +1,23 @@
-using System;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System;
using System.Collections.Generic;
using System.Composition;
using System.Diagnostics;
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDocsPlugins.csproj b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDocsPlugins.csproj
index 2c771f8..44e90c8 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDocsPlugins.csproj
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneDocsPlugins.csproj
@@ -1,4 +1,24 @@
<?xml version="1.0" encoding="utf-8"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\packages\Microsoft.Net.Compilers.2.2.0\build\Microsoft.Net.Compilers.props" Condition="Exists('..\packages\Microsoft.Net.Compilers.2.2.0\build\Microsoft.Net.Compilers.props')" />
<Import Project="$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props" Condition="Exists('$(MSBuildExtensionsPath)\$(MSBuildToolsVersion)\Microsoft.Common.props')" />
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockRule.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockRule.cs
index 28af227..6a8c8fd 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockRule.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockRule.cs
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
using System.Text.RegularExpressions;
using Microsoft.DocAsCode.MarkdownLite;
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockToken.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockToken.cs
index 310ac80..3c80706 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockToken.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneNoteBlockToken.cs
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
using Microsoft.DocAsCode.MarkdownLite;
namespace LuceneDocsPlugins
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneRendererPartProvider.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneRendererPartProvider.cs
index 5ca2407..70bffda 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneRendererPartProvider.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneRendererPartProvider.cs
@@ -1,4 +1,23 @@
-using System.Collections.Generic;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System.Collections.Generic;
using System.Composition;
using Microsoft.DocAsCode.Dfm;
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneTokenRendererPart.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneTokenRendererPart.cs
index 19140fe..c89965f 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneTokenRendererPart.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/LuceneTokenRendererPart.cs
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
using Microsoft.DocAsCode.Dfm;
using Microsoft.DocAsCode.MarkdownLite;
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/Properties/AssemblyInfo.cs b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/Properties/AssemblyInfo.cs
index 7c430c6..a2dbaa4 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/Properties/AssemblyInfo.cs
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/Properties/AssemblyInfo.cs
@@ -1,4 +1,23 @@
-using System.Reflection;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
diff --git a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/packages.config b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/packages.config
index 848589b..2f5f59f 100644
--- a/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/packages.config
+++ b/src/docs/LuceneDocsPlugins/LuceneDocsPlugins/packages.config
@@ -1,4 +1,24 @@
<?xml version="1.0" encoding="utf-8"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<packages>
<package id="HtmlAgilityPack" version="1.4.9" targetFramework="net46" />
<package id="Microsoft.Composition" version="1.0.31" targetFramework="net461" />
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
index 169b04c..249f400 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
@@ -5,6 +5,23 @@ using System.Reflection;
namespace Lucene.Net.Support
{
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/// <summary>
/// Static methods to create <see cref="BreakIterator"/> instances that behave (somewhat) like the JDK.
/// It is recommended to either use the default ICU <see cref="BreakIterator"/> methods instead of these
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs
index 71bf5c3..71c6372 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs
@@ -4,6 +4,23 @@ using System.Globalization;
namespace Lucene.Net.Search.PostingsHighlight
{
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/// <summary>
/// Mock of the original Lucene <see cref="PostingsHighlighter"/> that is backed
/// by a <see cref="JdkBreakIterator"/> with custom rules to act
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
index 1e5efbe..aea820d 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
@@ -6,6 +6,23 @@ using System;
namespace Lucene.Net.Tests.ICU.Support
{
+ /*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
/// <summary>
/// Tests to ensure compatibility with JDK 7
/// </summary>
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/CodeLinkReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/CodeLinkReplacer.cs
index 0d3cce2..a20ddc6 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/CodeLinkReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/CodeLinkReplacer.cs
@@ -1,4 +1,23 @@
-using Html2Markdown.Replacement;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using Html2Markdown.Replacement;
using System.Globalization;
using System.Text.RegularExpressions;
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/DocTypeReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/DocTypeReplacer.cs
index d488cbf..7e75dcc 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/DocTypeReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/DocTypeReplacer.cs
@@ -1,4 +1,23 @@
-using Html2Markdown.Replacement;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using Html2Markdown.Replacement;
using System.Text.RegularExpressions;
namespace JavaDocToMarkdownConverter.Formatters
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/ExtraHtmlElementReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/ExtraHtmlElementReplacer.cs
index 25677f7..044c7e5 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/ExtraHtmlElementReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/ExtraHtmlElementReplacer.cs
@@ -1,4 +1,23 @@
-using Html2Markdown.Replacement;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using Html2Markdown.Replacement;
using HtmlAgilityPack;
using System.IO;
using System.Text;
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/IReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/IReplacer.cs
index 79d89ad..728bad7 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/IReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/IReplacer.cs
@@ -1,4 +1,23 @@
-namespace JavaDocToMarkdownConverter.Formatters
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+namespace JavaDocToMarkdownConverter.Formatters
{
//This is exposed in the newer version of Html2Markdown but the later versions don't parse correctly so we have
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/JavaDocFormatters.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/JavaDocFormatters.cs
index c3fbd10..609d6d9 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/JavaDocFormatters.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/JavaDocFormatters.cs
@@ -1,4 +1,23 @@
-using System;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System;
using System.Collections.Generic;
namespace JavaDocToMarkdownConverter.Formatters
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/PatternReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/PatternReplacer.cs
index 27a7e35..609de9c 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/PatternReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/PatternReplacer.cs
@@ -1,4 +1,23 @@
-using System.Text.RegularExpressions;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System.Text.RegularExpressions;
namespace JavaDocToMarkdownConverter.Formatters
{
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/RepoLinkReplacer.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/RepoLinkReplacer.cs
index 24ff096..1ccd5c5 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/RepoLinkReplacer.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/Formatters/RepoLinkReplacer.cs
@@ -1,4 +1,23 @@
-using Html2Markdown.Replacement;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using Html2Markdown.Replacement;
using System.Text.RegularExpressions;
namespace JavaDocToMarkdownConverter.Formatters
diff --git a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/StringExtensions.cs b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/StringExtensions.cs
index eeda50b..d60d1f6 100644
--- a/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/StringExtensions.cs
+++ b/src/dotnet/tools/JavaDocToMarkdownConverter/JavaDocToMarkdownConverter/StringExtensions.cs
@@ -1,4 +1,23 @@
-using System.Collections.Generic;
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
diff --git a/websites/apidocs/lucenetemplate/styles/main.css b/websites/apidocs/lucenetemplate/styles/main.css
index 252cdb1..f98614d 100644
--- a/websites/apidocs/lucenetemplate/styles/main.css
+++ b/websites/apidocs/lucenetemplate/styles/main.css
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
/* .navbar-inverse {
background: #4a95da;
background: rgb(44, 95, 163);
diff --git a/websites/apidocs/lucenetemplate/styles/main.js b/websites/apidocs/lucenetemplate/styles/main.js
index ad4722e..634b5c6 100644
--- a/websites/apidocs/lucenetemplate/styles/main.js
+++ b/websites/apidocs/lucenetemplate/styles/main.js
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
$(function () {
renderAlerts();
diff --git a/websites/apidocs/lucenetemplate/web.config b/websites/apidocs/lucenetemplate/web.config
index f646909..51f0d98 100644
--- a/websites/apidocs/lucenetemplate/web.config
+++ b/websites/apidocs/lucenetemplate/web.config
@@ -1,4 +1,24 @@
<?xml version="1.0"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<configuration>
<system.webServer>
diff --git a/websites/site/lucenetemplate/styles/main.css b/websites/site/lucenetemplate/styles/main.css
index 812bf28..06e4d9e 100644
--- a/websites/site/lucenetemplate/styles/main.css
+++ b/websites/site/lucenetemplate/styles/main.css
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
/* .navbar-inverse {
background: #4a95da;
background: rgb(44, 95, 163);
diff --git a/websites/site/lucenetemplate/styles/site.css b/websites/site/lucenetemplate/styles/site.css
index b4fe7f7..aa5f52a 100644
--- a/websites/site/lucenetemplate/styles/site.css
+++ b/websites/site/lucenetemplate/styles/site.css
@@ -1,3 +1,22 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
/* START From hugo academic css */
#homepage section {
font-family: 'Merriweather', serif;
diff --git a/websites/site/lucenetemplate/web.config b/websites/site/lucenetemplate/web.config
index f646909..51f0d98 100644
--- a/websites/site/lucenetemplate/web.config
+++ b/websites/site/lucenetemplate/web.config
@@ -1,4 +1,24 @@
<?xml version="1.0"?>
+
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements. See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership. The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied. See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
<configuration>
<system.webServer>
[lucenenet] 01/09:
Lucene.Net.ICU.PostingsHighlight.PostingsHighlighter: Changed default
BreakIterator implementation to be the ICU default rather than using a
customized RuleBasedBreakIterator that was built just to make the tests
pass. Renamed ICUPostingsFormatter to reflect the change and be consistent
with other ICU classes. Confirmed against Java that this is the correct
default behavior and created separate ICU-specific tests. Moved the
JdkBreakIterator to Lucene.Net.ICU.Tests because it is just a test [...]
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 9c4a3c38a9f744ae5993a88a9a80933a63aafc0c
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 10:49:18 2019 +0700
Lucene.Net.ICU.PostingsHighlight.PostingsHighlighter: Changed default BreakIterator implementation to be the ICU default rather than using a customized RuleBasedBreakIterator that was built just to make the tests pass. Renamed ICUPostingsFormatter to reflect the change and be consistent with other ICU classes. Confirmed against Java that this is the correct default behavior and created separate ICU-specific tests. Moved the JdkBreakIterator to Lucene.Net.ICU.Tests because it is just a [...]
---
.../PostingsHighlight/MultiTermHighlighting.cs | 2 +-
.../PostingsHighlight/PassageFormatter.cs | 2 +-
.../PostingsHighlight/PassageScorer.cs | 2 +-
.../PostingsHighlight/PostingsHighlighter.cs | 27 ++-
.../PostingsHighlight/TestMultiTermHighlighting.cs | 161 +--------------
.../PostingsHighlight/TestPostingsHighlighter.cs | 17 +-
.../TestPostingsHighlighterRanking.cs | 44 +---
.../Lucene.Net.Tests.ICU.csproj | 4 +
.../TestICUMultiTermHighlighting.cs} | 222 ++++-----------------
.../TestICUPostingsHighlighter.cs} | 141 +++++++------
.../TestICUPostingsHighlighterRanking.cs} | 61 ++----
.../Support}/Data/jdk7/jdksent.rbbi | 0
.../Support}/Data/jdk7/jdkword.rbbi | 0
.../Support/JDKBreakIterator.cs | 3 +-
.../Support/PostingsHighlighter.cs | 29 +++
.../Support/TestJDKBreakIterator.cs | 2 +-
.../Support/jdksent.brk | Bin
.../Support/jdkword.brk | Bin
18 files changed, 226 insertions(+), 491 deletions(-)
diff --git a/src/Lucene.Net.Highlighter/PostingsHighlight/MultiTermHighlighting.cs b/src/Lucene.Net.Highlighter/PostingsHighlight/MultiTermHighlighting.cs
index 482716c..4d0c46a 100644
--- a/src/Lucene.Net.Highlighter/PostingsHighlight/MultiTermHighlighting.cs
+++ b/src/Lucene.Net.Highlighter/PostingsHighlight/MultiTermHighlighting.cs
@@ -221,7 +221,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// Returns a "fake" <see cref="DocsAndPositionsEnum"/> over the tokenstream, returning offsets where <paramref name="matchers"/>
/// matches tokens.
/// <para/>
- /// This is solely used internally by <see cref="PostingsHighlighter"/>: <b>DO NOT USE THIS METHOD!</b>
+ /// This is solely used internally by <see cref="ICUPostingsHighlighter"/>: <b>DO NOT USE THIS METHOD!</b>
/// </summary>
internal static DocsAndPositionsEnum GetDocsEnum(TokenStream ts, CharacterRunAutomaton[] matchers)
{
diff --git a/src/Lucene.Net.Highlighter/PostingsHighlight/PassageFormatter.cs b/src/Lucene.Net.Highlighter/PostingsHighlight/PassageFormatter.cs
index 770a6fa..80f0e09 100644
--- a/src/Lucene.Net.Highlighter/PostingsHighlight/PassageFormatter.cs
+++ b/src/Lucene.Net.Highlighter/PostingsHighlight/PassageFormatter.cs
@@ -36,7 +36,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// <param name="content">content for the field.</param>
/// <returns>
/// formatted highlight. Note that for the
- /// non-expert APIs in <see cref="PostingsHighlighter"/> that
+ /// non-expert APIs in <see cref="ICUPostingsHighlighter"/> that
/// return <see cref="string"/>, the <see cref="object.ToString()"/> method on the <see cref="object"/>
/// returned by this method is used to compute the string.
/// </returns>
diff --git a/src/Lucene.Net.Highlighter/PostingsHighlight/PassageScorer.cs b/src/Lucene.Net.Highlighter/PostingsHighlight/PassageScorer.cs
index fc64a70..ef23b20 100644
--- a/src/Lucene.Net.Highlighter/PostingsHighlight/PassageScorer.cs
+++ b/src/Lucene.Net.Highlighter/PostingsHighlight/PassageScorer.cs
@@ -21,7 +21,7 @@ namespace Lucene.Net.Search.PostingsHighlight
*/
/// <summary>
- /// Ranks passages found by <see cref="PostingsHighlighter"/>.
+ /// Ranks passages found by <see cref="ICUPostingsHighlighter"/>.
/// <para/>
/// Each passage is scored as a miniature document within the document.
/// The final score is computed as <c>norm</c> * ∑ (<c>weight</c> * <c>tf</c>).
diff --git a/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs b/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
index 274b44f..58a5d7e 100644
--- a/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
+++ b/src/Lucene.Net.Highlighter/PostingsHighlight/PostingsHighlighter.cs
@@ -1,7 +1,6 @@
#if FEATURE_BREAKITERATOR
using ICU4N.Text;
using Lucene.Net.Analysis;
-using Lucene.Net.ICU.Support;
using Lucene.Net.Index;
using Lucene.Net.Support;
using Lucene.Net.Util;
@@ -37,10 +36,11 @@ namespace Lucene.Net.Search.PostingsHighlight
/// Simple highlighter that does not analyze fields nor use
/// term vectors. Instead it requires
/// <see cref="IndexOptions.DOCS_AND_FREQS_AND_POSITIONS_AND_OFFSETS"/>.
- /// <para/>
+ /// </summary>
+ /// <remarks>
/// PostingsHighlighter treats the single original document as the whole corpus, and then scores individual
/// passages as if they were documents in this corpus. It uses a <see cref="BreakIterator"/> to find
- /// passages in the text; by default it breaks using <see cref="JdkBreakIterator.GetSentenceInstance(CultureInfo)"/> (for sentence breaking).
+ /// passages in the text; by default it breaks using <see cref="BreakIterator.GetSentenceInstance(CultureInfo)"/> (for sentence breaking).
/// It then iterates in parallel (merge sorting by offset) through
/// the positions of all terms from the query, coalescing those hits that occur in a single passage
/// into a <see cref="Passage"/>, and then scores each Passage using a separate <see cref="PassageScorer"/>.
@@ -64,16 +64,25 @@ namespace Lucene.Net.Search.PostingsHighlight
/// Field body = new Field("body", "foobar", offsetsType);
///
/// // retrieve highlights at query time
- /// PostingsHighlighter highlighter = new PostingsHighlighter();
+ /// ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
/// Query query = new TermQuery(new Term("body", "highlighting"));
/// TopDocs topDocs = searcher.Search(query, n);
/// string highlights[] = highlighter.Highlight("body", query, searcher, topDocs);
/// </code>
/// <para/>
/// This is thread-safe, and can be used across different readers.
+ /// <para/>
+ /// Note that the .NET implementation differs from the <c>PostingsHighlighter</c> in Lucene in
+ /// that it is backed by an ICU <see cref="RuleBasedBreakIterator"/>, which differs slightly in default behavior
+ /// than the one in the JDK. However, the ICU <see cref="RuleBasedBreakIterator"/> behavior can be customized
+ /// to meet a lot of scenarios that the one in the JDK cannot. See the ICU documentation at
+ /// <a href="http://userguide.icu-project.org/boundaryanalysis/break-rules">http://userguide.icu-project.org/boundaryanalysis/break-rules</a>
+ /// for more information how to pass custom rules to an ICU <see cref="RuleBasedBreakIterator"/>.
+ /// <para/>
/// @lucene.experimental
- /// </summary>
- public class PostingsHighlighter
+ /// </remarks>
+ [ExceptionToClassNameConvention]
+ public class ICUPostingsHighlighter // LUCENENET specific - renamed ICUPostingsHighlighter to reflect the change in default behavior
{
// TODO: maybe allow re-analysis for tiny fields? currently we require offsets,
// but if the analyzer is really fast and the field is tiny, this might really be
@@ -104,7 +113,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// <summary>
/// Creates a new highlighter with <see cref="DEFAULT_MAX_LENGTH"/>.
/// </summary>
- public PostingsHighlighter()
+ public ICUPostingsHighlighter()
: this(DEFAULT_MAX_LENGTH)
{
}
@@ -114,7 +123,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// </summary>
/// <param name="maxLength">maximum content size to process.</param>
/// <exception cref="ArgumentException">if <paramref name="maxLength"/> is negative or <c>int.MaxValue</c></exception>
- public PostingsHighlighter(int maxLength)
+ public ICUPostingsHighlighter(int maxLength)
{
if (maxLength < 0 || maxLength == int.MaxValue)
{
@@ -133,7 +142,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/// </summary>
protected virtual BreakIterator GetBreakIterator(string field)
{
- return JdkBreakIterator.GetSentenceInstance(CultureInfo.InvariantCulture);
+ return BreakIterator.GetSentenceInstance(CultureInfo.InvariantCulture);
}
/// <summary>
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs
index eeb6d47..87841f1 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs
+++ b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs
@@ -38,6 +38,17 @@ namespace Lucene.Net.Search.PostingsHighlight
/// Some tests that override <see cref="PostingsHighlighter.GetIndexAnalyzer(string)"/> to
/// highlight wilcard, fuzzy, etc queries.
/// </summary>
+ /// <remarks>
+ /// LUCENENET specific - These are the original tests from Lucene. They are only here as proof that we
+ /// can customize the <see cref="ICUPostingsHighlighter"/> to act like the PostingsHighlighter in Lucene,
+ /// which has slightly different default behavior than that of ICU because Lucene uses
+ /// the RuleBasedBreakIterator from the JDK, not that of ICU4J.
+ /// <para/>
+ /// These tests use a mock <see cref="PostingsHighlighter"/>, which is backed by an ICU
+ /// <see cref="ICU4N.Text.RuleBasedBreakIterator"/> that is customized a bit to act (sort of)
+ /// like the one in the JDK. However, this customized implementation is not a logical default for
+ /// the <see cref="ICUPostingsHighlighter"/>.
+ /// </remarks>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
public class TestMultiTermHighlighting : LuceneTestCase
{
@@ -82,13 +93,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- // PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- // {
- // return analyzer;
- // }
- //};
Query query = new WildcardQuery(new Term("body", "te*"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -138,13 +142,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
Query query = new PrefixQuery(new Term("body", "te"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -194,13 +191,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
Query query = new RegexpQuery(new Term("body", "te.*"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -250,13 +240,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
Query query = new FuzzyQuery(new Term("body", "tets"), 1);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -315,13 +298,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
Query query = TermRangeQuery.NewStringRange("body", "ta", "tf", true, true);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -429,13 +405,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
BooleanQuery query = new BooleanQuery();
query.Add(new WildcardQuery(new Term("body", "te*")), Occur.SHOULD);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -486,13 +455,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
DisjunctionMaxQuery query = new DisjunctionMaxQuery(0);
query.Add(new WildcardQuery(new Term("body", "te*")));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -532,13 +494,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
Query query = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -577,13 +532,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanOrQuery(new SpanQuery[] { childQuery });
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -623,13 +571,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanNearQuery(new SpanQuery[] { childQuery }, 0, true);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -669,13 +610,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
SpanQuery include = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
SpanQuery exclude = new SpanTermQuery(new Term("body", "bogus"));
Query query = new SpanNotQuery(include, exclude);
@@ -716,13 +650,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanFirstQuery(childQuery, 1000000);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -823,13 +750,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
BooleanQuery query = new BooleanQuery();
query.Add(new WildcardQuery(new Term("body", "te*")), Occur.SHOULD);
query.Add(new WildcardQuery(new Term("body", "one")), Occur.SHOULD);
@@ -846,65 +766,6 @@ namespace Lucene.Net.Search.PostingsHighlight
// matching term's text into the result:
highlighter = new PostingsHighlighterAnalyzerAndFormatterHelper(analyzer, new PassageFormatterHelper());
- //highlighter = new PostingsHighlighter()
- //{
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
-
- //@Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // return new PassageFormatter() {
-
- // @Override
- // public Object format(Passage passages[], String content)
- //{
- // // Copied from DefaultPassageFormatter, but
- // // tweaked to include the matched term:
- // StringBuilder sb = new StringBuilder();
- // int pos = 0;
- // for (Passage passage : passages)
- // {
- // // don't add ellipsis if its the first one, or if its connected.
- // if (passage.startOffset > pos && pos > 0)
- // {
- // sb.append("... ");
- // }
- // pos = passage.startOffset;
- // for (int i = 0; i < passage.numMatches; i++)
- // {
- // int start = passage.matchStarts[i];
- // int end = passage.matchEnds[i];
- // // its possible to have overlapping terms
- // if (start > pos)
- // {
- // sb.append(content, pos, start);
- // }
- // if (end > pos)
- // {
- // sb.append("<b>");
- // sb.append(content, Math.max(pos, start), end);
- // sb.append('(');
- // sb.append(passage.getMatchTerms()[i].utf8ToString());
- // sb.append(')');
- // sb.append("</b>");
- // pos = end;
- // }
- // }
- // // its possible a "term" from the analyzer could span a sentence boundary.
- // sb.append(content, pos, Math.max(pos, passage.endOffset));
- // pos = passage.endOffset;
- // }
- // return sb.toString();
- //}
- // };
- // }
- // };
-
-
assertEquals(1, topDocs.TotalHits);
snippets = highlighter.Highlight("body", query, searcher, topDocs);
assertEquals(1, snippets.Length);
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
index b96efea..1b24a8c 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
+++ b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
@@ -33,6 +33,17 @@ namespace Lucene.Net.Search.PostingsHighlight
* limitations under the License.
*/
+ /// <summary>
+ /// LUCENENET specific - These are the original tests from Lucene. They are only here as proof that we
+ /// can customize the <see cref="ICUPostingsHighlighter"/> to act like the PostingsHighlighter in Lucene,
+ /// which has slightly different default behavior than that of ICU because Lucene uses
+ /// the RuleBasedBreakIterator from the JDK, not that of ICU4J.
+ /// <para/>
+ /// These tests use a mock <see cref="PostingsHighlighter"/>, which is backed by an ICU
+ /// <see cref="ICU4N.Text.RuleBasedBreakIterator"/> that is customized a bit to act (sort of)
+ /// like the one in the JDK. However, this customized implementation is not a logical default for
+ /// the <see cref="ICUPostingsHighlighter"/>.
+ /// </summary>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
public class TestPostingsHighlighter : LuceneTestCase
{
@@ -746,8 +757,8 @@ namespace Lucene.Net.Search.PostingsHighlight
protected override string[][] LoadFieldValues(IndexSearcher searcher, string[] fields, int[] docids, int maxLength)
{
- Debug.Assert( fields.Length == 1);
- Debug.Assert( docids.Length == 1);
+ Debug.Assert(fields.Length == 1);
+ Debug.Assert(docids.Length == 1);
String[][] contents = RectangularArrays.ReturnRectangularArray<string>(1, 1); //= new String[1][1];
contents[0][0] = text;
return contents;
@@ -1150,7 +1161,7 @@ namespace Lucene.Net.Search.PostingsHighlight
{
protected override char GetMultiValuedSeparator(string field)
{
- Debug.Assert( field.Equals("body", StringComparison.Ordinal));
+ Debug.Assert(field.Equals("body", StringComparison.Ordinal));
return '\u2029';
}
}
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs
index 6d94c24..0243ed6 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs
+++ b/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs
@@ -30,6 +30,17 @@ namespace Lucene.Net.Search.PostingsHighlight
* limitations under the License.
*/
+ /// <summary>
+ /// LUCENENET specific - These are the original tests from Lucene. They are only here as proof that we
+ /// can customize the <see cref="ICUPostingsHighlighter"/> to act like the PostingsHighlighter in Lucene,
+ /// which has slightly different default behavior than that of ICU because Lucene uses
+ /// the RuleBasedBreakIterator from the JDK, not that of ICU4J.
+ /// <para/>
+ /// These tests use a mock <see cref="PostingsHighlighter"/>, which is backed by an ICU
+ /// <see cref="ICU4N.Text.RuleBasedBreakIterator"/> that is customized a bit to act (sort of)
+ /// like the one in the JDK. However, this customized implementation is not a logical default for
+ /// the <see cref="ICUPostingsHighlighter"/>.
+ /// </summary>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
public class TestPostingsHighlighterRanking : LuceneTestCase
{
@@ -122,25 +133,6 @@ namespace Lucene.Net.Search.PostingsHighlight
{
CheckQueryPostingsHighlighter p1 = new CheckQueryPostingsHighlighter(int.MaxValue - 1);
CheckQueryPostingsHighlighter p2 = new CheckQueryPostingsHighlighter(int.MaxValue - 1);
- // FakePassageFormatter f1 = new FakePassageFormatter();
- // PostingsHighlighter p1 = new PostingsHighlighter(int.MaxValue - 1) {
- // @Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // assertEquals("body", field);
- // return f1;
- //}
- // };
-
- // FakePassageFormatter f2 = new FakePassageFormatter();
- //PostingsHighlighter p2 = new PostingsHighlighter(Integer.MAX_VALUE - 1) {
- // @Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // assertEquals("body", field);
- // return f2;
- //}
- // };
BooleanQuery bq = new BooleanQuery(false);
bq.Add(query, Occur.MUST);
@@ -300,13 +292,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new CustomBPostingsHighlighter();
- //PostingsHighlighter highlighter = new PostingsHighlighter(10000) {
- // @Override
- // protected PassageScorer getScorer(String field)
- //{
- // return new PassageScorer(1.2f, 0, 87);
- //}
- // };
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -351,13 +336,6 @@ namespace Lucene.Net.Search.PostingsHighlight
IndexSearcher searcher = NewSearcher(ir);
PostingsHighlighter highlighter = new CustomK1PostingsHighlighter();
- //PostingsHighlighter highlighter = new PostingsHighlighter(10000) {
- // @Override
- // protected PassageScorer getScorer(String field)
- //{
- // return new PassageScorer(0, 0.75f, 87);
- //}
- // };
BooleanQuery query = new BooleanQuery();
query.Add(new TermQuery(new Term("body", "foo")), Occur.SHOULD);
query.Add(new TermQuery(new Term("body", "bar")), Occur.SHOULD);
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj b/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
index cc2a743..eb5d448 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Lucene.Net.Tests.ICU.csproj
@@ -40,7 +40,11 @@
<EmbeddedResource Include="..\..\Lucene.Net.Tests.Analysis.ICU\Analysis\Icu\Segmentation\*.rbbi" LinkBase="Analysis\Icu\Segmentation" />
<Compile Include="..\..\Lucene.Net.Tests.Analysis.ICU\Collation\**\*.cs" LinkBase="Collation" />
<Compile Include="..\..\Lucene.Net.Tests.Highlighter\PostingsHighlight\**\*.cs" LinkBase="Search\PostingsHighlight" />
+ <None Remove="Support\jdksent.brk" />
+ <None Remove="Support\jdkword.brk" />
<EmbeddedResource Include="..\..\Lucene.Net.Tests.Highlighter\PostingsHighlight\CambridgeMA.utf8" Link="Search\PostingsHighlight\CambridgeMA.utf8" />
+ <EmbeddedResource Include="Support\jdksent.brk" />
+ <EmbeddedResource Include="Support\jdkword.brk" />
<Compile Include="..\..\Lucene.Net.Tests.Highlighter\VectorHighlight\BreakIteratorBoundaryScannerTest.cs" Link="Search\VectorHighlight\BreakIteratorBoundaryScannerTest.cs" />
</ItemGroup>
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUMultiTermHighlighting.cs
similarity index 81%
copy from src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs
copy to src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUMultiTermHighlighting.cs
index eeb6d47..2849b81 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestMultiTermHighlighting.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUMultiTermHighlighting.cs
@@ -1,18 +1,15 @@
#if FEATURE_BREAKITERATOR
using Lucene.Net.Analysis;
+using Lucene.Net.Attributes;
using Lucene.Net.Documents;
using Lucene.Net.Index;
-using Lucene.Net.Search;
using Lucene.Net.Search.Spans;
using Lucene.Net.Store;
using Lucene.Net.Support;
using Lucene.Net.Util;
using NUnit.Framework;
using System;
-using System.Collections.Generic;
-using System.Linq;
using System.Text;
-using System.Threading.Tasks;
using IndexOptions = Lucene.Net.Index.IndexOptions;
namespace Lucene.Net.Search.PostingsHighlight
@@ -35,13 +32,22 @@ namespace Lucene.Net.Search.PostingsHighlight
*/
/// <summary>
- /// Some tests that override <see cref="PostingsHighlighter.GetIndexAnalyzer(string)"/> to
+ /// Some tests that override <see cref="ICUPostingsHighlighter.GetIndexAnalyzer(string)"/> to
/// highlight wilcard, fuzzy, etc queries.
/// </summary>
+ /// <remarks>
+ /// LUCENENET specific - Modified the behavior of the PostingsHighlighter in Java to return the
+ /// org.ibm.icu.BreakIterator version 60.1 instead of java.text.BreakIterator and modified the original Lucene
+ /// tests to pass, then ported to .NET. There are no changes in this class from that of Lucene 4.8.1.
+ /// <para/>
+ /// Although the ICU <see cref="ICU4N.Text.BreakIterator"/> acts slightly different than the JDK's verision, using the default
+ /// behavior of the ICU <see cref="ICU4N.Text.BreakIterator"/> is the most logical default to use in .NET. It is the same
+ /// default that was chosen in Apache Harmony.
+ /// </remarks>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
- public class TestMultiTermHighlighting : LuceneTestCase
+ public class TestICUMultiTermHighlighting : LuceneTestCase
{
- internal class PostingsHighlighterAnalyzerHelper : PostingsHighlighter
+ internal class PostingsHighlighterAnalyzerHelper : ICUPostingsHighlighter
{
private readonly Analyzer analyzer;
@@ -56,7 +62,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestWildcards()
{
Directory dir = NewDirectory();
@@ -81,14 +87,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- // PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- // {
- // return analyzer;
- // }
- //};
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = new WildcardQuery(new Term("body", "te*"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -112,7 +111,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestOnePrefix()
{
Directory dir = NewDirectory();
@@ -137,14 +136,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = new PrefixQuery(new Term("body", "te"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -168,7 +160,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestOneRegexp()
{
Directory dir = NewDirectory();
@@ -193,14 +185,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = new RegexpQuery(new Term("body", "te.*"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -224,7 +209,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestOneFuzzy()
{
Directory dir = NewDirectory();
@@ -249,14 +234,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = new FuzzyQuery(new Term("body", "tets"), 1);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -289,7 +267,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestRanges()
{
Directory dir = NewDirectory();
@@ -314,14 +292,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = TermRangeQuery.NewStringRange("body", "ta", "tf", true, true);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -403,7 +374,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestWildcardInBoolean()
{
Directory dir = NewDirectory();
@@ -428,14 +399,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
BooleanQuery query = new BooleanQuery();
query.Add(new WildcardQuery(new Term("body", "te*")), Occur.SHOULD);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -460,7 +424,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestWildcardInDisjunctionMax()
{
Directory dir = NewDirectory();
@@ -485,14 +449,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
DisjunctionMaxQuery query = new DisjunctionMaxQuery(0);
query.Add(new WildcardQuery(new Term("body", "te*")));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -506,7 +463,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpanWildcard()
{
Directory dir = NewDirectory();
@@ -531,14 +488,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
Query query = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -551,7 +501,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpanOr()
{
Directory dir = NewDirectory();
@@ -576,14 +526,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanOrQuery(new SpanQuery[] { childQuery });
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -597,7 +540,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpanNear()
{
Directory dir = NewDirectory();
@@ -622,14 +565,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanNearQuery(new SpanQuery[] { childQuery }, 0, true);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -643,7 +579,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpanNot()
{
Directory dir = NewDirectory();
@@ -668,14 +604,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
SpanQuery include = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
SpanQuery exclude = new SpanTermQuery(new Term("body", "bogus"));
Query query = new SpanNotQuery(include, exclude);
@@ -690,7 +619,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpanPositionCheck()
{
Directory dir = NewDirectory();
@@ -715,14 +644,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
SpanQuery childQuery = new SpanMultiTermQueryWrapper<WildcardQuery>(new WildcardQuery(new Term("body", "te*")));
Query query = new SpanFirstQuery(childQuery, 1000000);
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -799,7 +721,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/** Runs a query with two MTQs and confirms the formatter
* can tell which query matched which hit. */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestWhichMTQMatched()
{
Directory dir = NewDirectory();
@@ -822,14 +744,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
- //PostingsHighlighter highlighter = new PostingsHighlighter() {
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
- // };
+ ICUPostingsHighlighter highlighter = new PostingsHighlighterAnalyzerHelper(analyzer);
BooleanQuery query = new BooleanQuery();
query.Add(new WildcardQuery(new Term("body", "te*")), Occur.SHOULD);
query.Add(new WildcardQuery(new Term("body", "one")), Occur.SHOULD);
@@ -846,65 +761,6 @@ namespace Lucene.Net.Search.PostingsHighlight
// matching term's text into the result:
highlighter = new PostingsHighlighterAnalyzerAndFormatterHelper(analyzer, new PassageFormatterHelper());
- //highlighter = new PostingsHighlighter()
- //{
- // @Override
- // protected Analyzer getIndexAnalyzer(String field)
- //{
- // return analyzer;
- //}
-
- //@Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // return new PassageFormatter() {
-
- // @Override
- // public Object format(Passage passages[], String content)
- //{
- // // Copied from DefaultPassageFormatter, but
- // // tweaked to include the matched term:
- // StringBuilder sb = new StringBuilder();
- // int pos = 0;
- // for (Passage passage : passages)
- // {
- // // don't add ellipsis if its the first one, or if its connected.
- // if (passage.startOffset > pos && pos > 0)
- // {
- // sb.append("... ");
- // }
- // pos = passage.startOffset;
- // for (int i = 0; i < passage.numMatches; i++)
- // {
- // int start = passage.matchStarts[i];
- // int end = passage.matchEnds[i];
- // // its possible to have overlapping terms
- // if (start > pos)
- // {
- // sb.append(content, pos, start);
- // }
- // if (end > pos)
- // {
- // sb.append("<b>");
- // sb.append(content, Math.max(pos, start), end);
- // sb.append('(');
- // sb.append(passage.getMatchTerms()[i].utf8ToString());
- // sb.append(')');
- // sb.append("</b>");
- // pos = end;
- // }
- // }
- // // its possible a "term" from the analyzer could span a sentence boundary.
- // sb.append(content, pos, Math.max(pos, passage.endOffset));
- // pos = passage.endOffset;
- // }
- // return sb.toString();
- //}
- // };
- // }
- // };
-
-
assertEquals(1, topDocs.TotalHits);
snippets = highlighter.Highlight("body", query, searcher, topDocs);
assertEquals(1, snippets.Length);
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
similarity index 91%
copy from src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
copy to src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
index b96efea..b5e1f47 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighter.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighter.cs
@@ -1,6 +1,7 @@
#if FEATURE_BREAKITERATOR
using ICU4N.Text;
using Lucene.Net.Analysis;
+using Lucene.Net.Attributes;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Support;
@@ -33,10 +34,20 @@ namespace Lucene.Net.Search.PostingsHighlight
* limitations under the License.
*/
+ /// <summary>
+ /// LUCENENET specific - Modified the behavior of the PostingsHighlighter in Java to return the
+ /// org.ibm.icu.BreakIterator version 60.1 instead of java.text.BreakIterator and modified the original Lucene
+ /// tests to pass, then ported to .NET. The only change required was that of the TestEmptyHighlights method
+ /// which breaks the sentence in a different place than in the JDK.
+ /// <para/>
+ /// Although the ICU <see cref="BreakIterator"/> acts slightly different than the JDK's verision, using the default
+ /// behavior of the ICU <see cref="BreakIterator"/> is the most logical default to use in .NET. It is the same
+ /// default that was chosen in Apache Harmony.
+ /// </summary>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
- public class TestPostingsHighlighter : LuceneTestCase
+ public class TestICUPostingsHighlighter : LuceneTestCase
{
- [Test]
+ [Test, LuceneNetSpecific]
public void TestBasics()
{
Directory dir = NewDirectory();
@@ -59,7 +70,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -72,7 +83,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFormatWithMatchExceedingContentLength2()
{
@@ -85,7 +96,7 @@ namespace Lucene.Net.Search.PostingsHighlight
assertEquals("123 <b>TEST</b> 01234 TE", snippets[0]);
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFormatWithMatchExceedingContentLength3()
{
@@ -98,7 +109,7 @@ namespace Lucene.Net.Search.PostingsHighlight
assertEquals("123 5678 01234 TE", snippets[0]);
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFormatWithMatchExceedingContentLength()
{
@@ -143,7 +154,7 @@ namespace Lucene.Net.Search.PostingsHighlight
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
- PostingsHighlighter highlighter = new PostingsHighlighter(maxLength);
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter(maxLength);
String[] snippets = highlighter.Highlight("body", query, searcher, topDocs);
@@ -153,7 +164,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
// simple test highlighting last word.
- [Test]
+ [Test, LuceneNetSpecific]
public void TestHighlightLastWord()
{
Directory dir = NewDirectory();
@@ -174,7 +185,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -187,7 +198,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
// simple test with one sentence documents.
- [Test]
+ [Test, LuceneNetSpecific]
public void TestOneSentence()
{
Directory dir = NewDirectory();
@@ -211,7 +222,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -225,7 +236,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
// simple test with multiple values that make a result longer than maxLength.
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMaxLengthWithMultivalue()
{
Directory dir = NewDirectory();
@@ -251,7 +262,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter(40);
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter(40);
Query query = new TermQuery(new Term("body", "field"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -264,7 +275,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMultipleFields()
{
Directory dir = NewDirectory();
@@ -291,7 +302,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
BooleanQuery query = new BooleanQuery();
query.Add(new TermQuery(new Term("body", "highlighting")), Occur.SHOULD);
query.Add(new TermQuery(new Term("title", "best")), Occur.SHOULD);
@@ -307,7 +318,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMultipleTerms()
{
Directory dir = NewDirectory();
@@ -330,7 +341,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
BooleanQuery query = new BooleanQuery();
query.Add(new TermQuery(new Term("body", "highlighting")), Occur.SHOULD);
query.Add(new TermQuery(new Term("body", "just")), Occur.SHOULD);
@@ -346,7 +357,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMultiplePassages()
{
Directory dir = NewDirectory();
@@ -369,7 +380,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -382,7 +393,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestUserFailedToIndexOffsets()
{
Directory dir = NewDirectory();
@@ -409,7 +420,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -440,7 +451,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestBuddhism()
{
String text = "This eight-volume set brings together seminal papers in Buddhist studies from a vast " +
@@ -470,7 +481,7 @@ namespace Lucene.Net.Search.PostingsHighlight
query.Add(new Term("body", "origins"));
TopDocs topDocs = searcher.Search(query, 10);
assertEquals(1, topDocs.TotalHits);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
String[] snippets = highlighter.Highlight("body", query, searcher, topDocs, 2);
assertEquals(1, snippets.Length);
assertTrue(snippets[0].Contains("<b>Buddhist</b> <b>origins</b>"));
@@ -478,7 +489,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCuriousGeorge()
{
String text = "It’s the formula for success for preschoolers—Curious George and fire trucks! " +
@@ -502,7 +513,7 @@ namespace Lucene.Net.Search.PostingsHighlight
query.Add(new Term("body", "george"));
TopDocs topDocs = searcher.Search(query, 10);
assertEquals(1, topDocs.TotalHits);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
String[] snippets = highlighter.Highlight("body", query, searcher, topDocs, 2);
assertEquals(1, snippets.Length);
assertFalse(snippets[0].Contains("<b>Curious</b>Curious"));
@@ -510,7 +521,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCambridgeMA()
{
String text;
@@ -537,7 +548,7 @@ namespace Lucene.Net.Search.PostingsHighlight
query.Add(new TermQuery(new Term("body", "massachusetts")), Occur.SHOULD);
TopDocs topDocs = searcher.Search(query, 10);
assertEquals(1, topDocs.TotalHits);
- PostingsHighlighter highlighter = new PostingsHighlighter(int.MaxValue - 1);
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter(int.MaxValue - 1);
String[] snippets = highlighter.Highlight("body", query, searcher, topDocs, 2);
assertEquals(1, snippets.Length);
assertTrue(snippets[0].Contains("<b>Square</b>"));
@@ -546,7 +557,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestPassageRanking()
{
Directory dir = NewDirectory();
@@ -567,7 +578,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -579,7 +590,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestBooleanMustNot()
{
Directory dir = NewDirectory();
@@ -601,7 +612,7 @@ namespace Lucene.Net.Search.PostingsHighlight
query2.Add(new TermQuery(new Term("body", "both")), Occur.MUST_NOT);
TopDocs topDocs = searcher.Search(query, 10);
assertEquals(1, topDocs.TotalHits);
- PostingsHighlighter highlighter = new PostingsHighlighter(int.MaxValue - 1);
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter(int.MaxValue - 1);
String[] snippets = highlighter.Highlight("body", query, searcher, topDocs, 2);
assertEquals(1, snippets.Length);
assertFalse(snippets[0].Contains("<b>both</b>"));
@@ -609,7 +620,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestHighlightAllText()
{
Directory dir = NewDirectory();
@@ -630,7 +641,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new WholeBreakIteratorPostingsHighlighter(10000);
+ ICUPostingsHighlighter highlighter = new WholeBreakIteratorPostingsHighlighter(10000);
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -642,7 +653,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class WholeBreakIteratorPostingsHighlighter : PostingsHighlighter
+ internal class WholeBreakIteratorPostingsHighlighter : ICUPostingsHighlighter
{
public WholeBreakIteratorPostingsHighlighter()
: base()
@@ -660,7 +671,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestSpecificDocIDs()
{
Directory dir = NewDirectory();
@@ -683,7 +694,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(2, topDocs.TotalHits);
@@ -700,7 +711,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCustomFieldValueSource()
{
Directory dir = NewDirectory();
@@ -721,7 +732,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new LoadFieldValuesPostingsHighlighter(10000, text);
+ ICUPostingsHighlighter highlighter = new LoadFieldValuesPostingsHighlighter(10000, text);
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -756,7 +767,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/** Make sure highlighter returns first N sentences if
* there were no hits. */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestEmptyHighlights()
{
Directory dir = NewDirectory();
@@ -776,12 +787,12 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
int[] docIDs = new int[] { 0 };
String[] snippets = highlighter.HighlightFields(new String[] { "body" }, query, searcher, docIDs, new int[] { 2 })["body"];
assertEquals(1, snippets.Length);
- assertEquals("test this is. another sentence this test has. ", snippets[0]);
+ assertEquals("test this is. another sentence this test has. far away is that planet.", snippets[0]);
ir.Dispose();
dir.Dispose();
@@ -789,7 +800,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/** Make sure highlighter we can customize how emtpy
* highlight is returned. */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCustomEmptyHighlights()
{
Directory dir = NewDirectory();
@@ -809,7 +820,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new GetEmptyHighlightPostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new GetEmptyHighlightPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
int[] docIDs = new int[] { 0 };
String[] snippets = highlighter.HighlightFields(new String[] { "body" }, query, searcher, docIDs, new int[] { 2 })["body"];
@@ -820,7 +831,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class GetEmptyHighlightPostingsHighlighter : PostingsHighlighter
+ internal class GetEmptyHighlightPostingsHighlighter : ICUPostingsHighlighter
{
protected override Passage[] GetEmptyHighlight(string fieldName, BreakIterator bi, int maxPassages)
{
@@ -830,7 +841,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/** Make sure highlighter returns whole text when there
* are no hits and BreakIterator is null. */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestEmptyHighlightsWhole()
{
Directory dir = NewDirectory();
@@ -850,7 +861,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new WholeBreakIteratorPostingsHighlighter(10000);
+ ICUPostingsHighlighter highlighter = new WholeBreakIteratorPostingsHighlighter(10000);
Query query = new TermQuery(new Term("body", "highlighting"));
int[] docIDs = new int[] { 0 };
String[] snippets = highlighter.HighlightFields(new String[] { "body" }, query, searcher, docIDs, new int[] { 2 })["body"];
@@ -863,7 +874,7 @@ namespace Lucene.Net.Search.PostingsHighlight
/** Make sure highlighter is OK with entirely missing
* field. */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFieldIsMissing()
{
Directory dir = NewDirectory();
@@ -883,7 +894,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("bogus", "highlighting"));
int[] docIDs = new int[] { 0 };
String[] snippets = highlighter.HighlightFields(new String[] { "bogus" }, query, searcher, docIDs, new int[] { 2 })["bogus"];
@@ -894,7 +905,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFieldIsJustSpace()
{
Directory dir = NewDirectory();
@@ -918,7 +929,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
int docID = searcher.Search(new TermQuery(new Term("id", "id")), 1).ScoreDocs[0].Doc;
Query query = new TermQuery(new Term("body", "highlighting"));
@@ -932,7 +943,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestFieldIsEmptyString()
{
Directory dir = NewDirectory();
@@ -956,7 +967,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
int docID = searcher.Search(new TermQuery(new Term("id", "id")), 1).ScoreDocs[0].Doc;
Query query = new TermQuery(new Term("body", "highlighting"));
@@ -970,7 +981,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMultipleDocs()
{
Directory dir = NewDirectory();
@@ -1004,7 +1015,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
Query query = new TermQuery(new Term("body", "answer"));
TopDocs hits = searcher.Search(query, numDocs);
assertEquals(numDocs, hits.TotalHits);
@@ -1027,7 +1038,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestMultipleSnippetSizes()
{
Directory dir = NewDirectory();
@@ -1051,7 +1062,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new PostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ICUPostingsHighlighter();
BooleanQuery query = new BooleanQuery();
query.Add(new TermQuery(new Term("body", "test")), Occur.SHOULD);
query.Add(new TermQuery(new Term("title", "test")), Occur.SHOULD);
@@ -1064,7 +1075,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- [Test]
+ [Test, LuceneNetSpecific]
public void TestEncode()
{
Directory dir = NewDirectory();
@@ -1085,7 +1096,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new GetFormatterPostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new GetFormatterPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -1097,7 +1108,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class GetFormatterPostingsHighlighter : PostingsHighlighter
+ internal class GetFormatterPostingsHighlighter : ICUPostingsHighlighter
{
protected override PassageFormatter GetFormatter(string field)
{
@@ -1106,7 +1117,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
/** customizing the gap separator to force a sentence break */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestGapSeparator()
{
Directory dir = NewDirectory();
@@ -1133,7 +1144,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new GetMultiValuedSeparatorPostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new GetMultiValuedSeparatorPostingsHighlighter();
Query query = new TermQuery(new Term("body", "field"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -1146,7 +1157,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class GetMultiValuedSeparatorPostingsHighlighter : PostingsHighlighter
+ internal class GetMultiValuedSeparatorPostingsHighlighter : ICUPostingsHighlighter
{
protected override char GetMultiValuedSeparator(string field)
{
@@ -1156,7 +1167,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
// LUCENE-4906
- [Test]
+ [Test, LuceneNetSpecific]
public void TestObjectFormatter()
{
Directory dir = NewDirectory();
@@ -1177,7 +1188,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new ObjectFormatterPostingsHighlighter();
+ ICUPostingsHighlighter highlighter = new ObjectFormatterPostingsHighlighter();
Query query = new TermQuery(new Term("body", "highlighting"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
@@ -1193,7 +1204,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class ObjectFormatterPostingsHighlighter : PostingsHighlighter
+ internal class ObjectFormatterPostingsHighlighter : ICUPostingsHighlighter
{
protected override PassageFormatter GetFormatter(string field)
{
diff --git a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighterRanking.cs
similarity index 87%
copy from src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs
copy to src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighterRanking.cs
index 6d94c24..1f73a12 100644
--- a/src/Lucene.Net.Tests.Highlighter/PostingsHighlight/TestPostingsHighlighterRanking.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Search/PostingsHighlight/TestICUPostingsHighlighterRanking.cs
@@ -1,5 +1,6 @@
#if FEATURE_BREAKITERATOR
using Lucene.Net.Analysis;
+using Lucene.Net.Attributes;
using Lucene.Net.Documents;
using Lucene.Net.Index;
using Lucene.Net.Store;
@@ -30,15 +31,24 @@ namespace Lucene.Net.Search.PostingsHighlight
* limitations under the License.
*/
+ /// <summary>
+ /// LUCENENET specific - Modified the behavior of the PostingsHighlighter in Java to return the
+ /// org.ibm.icu.BreakIterator version 60.1 instead of java.text.BreakIterator and modified the original Lucene
+ /// tests to pass, then ported to .NET. There are no changes in this class from that of Lucene 4.8.1.
+ /// <para/>
+ /// Although the ICU <see cref="ICU4N.Text.BreakIterator"/> acts slightly different than the JDK's verision, using the default
+ /// behavior of the ICU <see cref="ICU4N.Text.BreakIterator"/> is the most logical default to use in .NET. It is the same
+ /// default that was chosen in Apache Harmony.
+ /// </summary>
[SuppressCodecs("MockFixedIntBlock", "MockVariableIntBlock", "MockSep", "MockRandom", "Lucene3x")]
- public class TestPostingsHighlighterRanking : LuceneTestCase
+ public class TestICUPostingsHighlighterRanking : LuceneTestCase
{
/// <summary>
/// indexes a bunch of gibberish, and then highlights top(n).
/// asserts that top(n) highlights is a subset of top(n+1) up to some max N
/// </summary>
// TODO: this only tests single-valued fields. we should also index multiple values per field!
- [Test]
+ [Test, LuceneNetSpecific]
public void TestRanking()
{
// number of documents: we will check each one
@@ -100,7 +110,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
}
- internal class CheckQueryPostingsHighlighter : PostingsHighlighter
+ internal class CheckQueryPostingsHighlighter : ICUPostingsHighlighter
{
internal FakePassageFormatter f = new FakePassageFormatter();
@@ -122,25 +132,6 @@ namespace Lucene.Net.Search.PostingsHighlight
{
CheckQueryPostingsHighlighter p1 = new CheckQueryPostingsHighlighter(int.MaxValue - 1);
CheckQueryPostingsHighlighter p2 = new CheckQueryPostingsHighlighter(int.MaxValue - 1);
- // FakePassageFormatter f1 = new FakePassageFormatter();
- // PostingsHighlighter p1 = new PostingsHighlighter(int.MaxValue - 1) {
- // @Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // assertEquals("body", field);
- // return f1;
- //}
- // };
-
- // FakePassageFormatter f2 = new FakePassageFormatter();
- //PostingsHighlighter p2 = new PostingsHighlighter(Integer.MAX_VALUE - 1) {
- // @Override
- // protected PassageFormatter getFormatter(String field)
- //{
- // assertEquals("body", field);
- // return f2;
- //}
- // };
BooleanQuery bq = new BooleanQuery(false);
bq.Add(query, Occur.MUST);
@@ -277,7 +268,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
/** sets b=0 to disable passage length normalization */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCustomB()
{
Directory dir = NewDirectory();
@@ -299,14 +290,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new CustomBPostingsHighlighter();
- //PostingsHighlighter highlighter = new PostingsHighlighter(10000) {
- // @Override
- // protected PassageScorer getScorer(String field)
- //{
- // return new PassageScorer(1.2f, 0, 87);
- //}
- // };
+ ICUPostingsHighlighter highlighter = new CustomBPostingsHighlighter();
Query query = new TermQuery(new Term("body", "test"));
TopDocs topDocs = searcher.Search(query, null, 10, Sort.INDEXORDER);
assertEquals(1, topDocs.TotalHits);
@@ -318,7 +302,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class CustomBPostingsHighlighter : PostingsHighlighter
+ internal class CustomBPostingsHighlighter : ICUPostingsHighlighter
{
protected override PassageScorer GetScorer(string field)
{
@@ -327,7 +311,7 @@ namespace Lucene.Net.Search.PostingsHighlight
}
/** sets k1=0 for simple coordinate-level match (# of query terms present) */
- [Test]
+ [Test, LuceneNetSpecific]
public void TestCustomK1()
{
Directory dir = NewDirectory();
@@ -350,14 +334,7 @@ namespace Lucene.Net.Search.PostingsHighlight
iw.Dispose();
IndexSearcher searcher = NewSearcher(ir);
- PostingsHighlighter highlighter = new CustomK1PostingsHighlighter();
- //PostingsHighlighter highlighter = new PostingsHighlighter(10000) {
- // @Override
- // protected PassageScorer getScorer(String field)
- //{
- // return new PassageScorer(0, 0.75f, 87);
- //}
- // };
+ ICUPostingsHighlighter highlighter = new CustomK1PostingsHighlighter();
BooleanQuery query = new BooleanQuery();
query.Add(new TermQuery(new Term("body", "foo")), Occur.SHOULD);
query.Add(new TermQuery(new Term("body", "bar")), Occur.SHOULD);
@@ -371,7 +348,7 @@ namespace Lucene.Net.Search.PostingsHighlight
dir.Dispose();
}
- internal class CustomK1PostingsHighlighter : PostingsHighlighter
+ internal class CustomK1PostingsHighlighter : ICUPostingsHighlighter
{
public CustomK1PostingsHighlighter()
: base(10000)
diff --git a/src/dotnet/Lucene.Net.ICU/Data/jdk7/jdksent.rbbi b/src/dotnet/Lucene.Net.Tests.ICU/Support/Data/jdk7/jdksent.rbbi
similarity index 100%
rename from src/dotnet/Lucene.Net.ICU/Data/jdk7/jdksent.rbbi
rename to src/dotnet/Lucene.Net.Tests.ICU/Support/Data/jdk7/jdksent.rbbi
diff --git a/src/dotnet/Lucene.Net.ICU/Data/jdk7/jdkword.rbbi b/src/dotnet/Lucene.Net.Tests.ICU/Support/Data/jdk7/jdkword.rbbi
similarity index 100%
rename from src/dotnet/Lucene.Net.ICU/Data/jdk7/jdkword.rbbi
rename to src/dotnet/Lucene.Net.Tests.ICU/Support/Data/jdk7/jdkword.rbbi
diff --git a/src/dotnet/Lucene.Net.ICU/Support/JDKBreakIterator.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
similarity index 98%
rename from src/dotnet/Lucene.Net.ICU/Support/JDKBreakIterator.cs
rename to src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
index 266b36a..169b04c 100644
--- a/src/dotnet/Lucene.Net.ICU/Support/JDKBreakIterator.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/JDKBreakIterator.cs
@@ -1,10 +1,9 @@
using ICU4N.Text;
-using Lucene.Net.Support;
using System.Globalization;
using System.IO;
using System.Reflection;
-namespace Lucene.Net.ICU.Support
+namespace Lucene.Net.Support
{
/// <summary>
/// Static methods to create <see cref="BreakIterator"/> instances that behave (somewhat) like the JDK.
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs
new file mode 100644
index 0000000..71bf5c3
--- /dev/null
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/PostingsHighlighter.cs
@@ -0,0 +1,29 @@
+using ICU4N.Text;
+using Lucene.Net.Support;
+using System.Globalization;
+
+namespace Lucene.Net.Search.PostingsHighlight
+{
+ /// <summary>
+ /// Mock of the original Lucene <see cref="PostingsHighlighter"/> that is backed
+ /// by a <see cref="JdkBreakIterator"/> with custom rules to act
+ /// (sort of) like the JDK. This is just to verify we can make the behavior work
+ /// similar to the implementation in Lucene by customizing the <see cref="BreakIterator"/>.
+ /// </summary>
+ public class PostingsHighlighter : ICUPostingsHighlighter
+ {
+ public PostingsHighlighter()
+ : base()
+ {
+ }
+
+ public PostingsHighlighter(int maxLength)
+ : base(maxLength)
+ {
+ }
+ protected override BreakIterator GetBreakIterator(string field)
+ {
+ return JdkBreakIterator.GetSentenceInstance(CultureInfo.InvariantCulture);
+ }
+ }
+}
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
index 04a09b7..ea8651c 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
@@ -1,6 +1,6 @@
using ICU4N.Text;
using Lucene.Net.Attributes;
-using Lucene.Net.ICU.Support;
+using Lucene.Net.Support;
using NUnit.Framework;
using System;
diff --git a/src/dotnet/Lucene.Net.ICU/Support/jdksent.brk b/src/dotnet/Lucene.Net.Tests.ICU/Support/jdksent.brk
similarity index 100%
rename from src/dotnet/Lucene.Net.ICU/Support/jdksent.brk
rename to src/dotnet/Lucene.Net.Tests.ICU/Support/jdksent.brk
diff --git a/src/dotnet/Lucene.Net.ICU/Support/jdkword.brk b/src/dotnet/Lucene.Net.Tests.ICU/Support/jdkword.brk
similarity index 100%
rename from src/dotnet/Lucene.Net.ICU/Support/jdkword.brk
rename to src/dotnet/Lucene.Net.Tests.ICU/Support/jdkword.brk
[lucenenet] 02/09:
Lucene.Net.Tests.ICU.Search.VectorHighlight.BreakIteratorBoundaryScannerTest:
Modified original TestSentenceBoundary test to skip east Asian languages
because the mock doesn't override them. Added TestICUWordBoundary and
TestICUSentenceBoundary tests to confirm that the boundary scanner works
the same with ICU4N as it does with ICU4J with default settings.
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 50623b6f84f8d05bf9e6c99539d67ee56047b38c
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 13:50:02 2019 +0700
Lucene.Net.Tests.ICU.Search.VectorHighlight.BreakIteratorBoundaryScannerTest: Modified original TestSentenceBoundary test to skip east Asian languages because the mock doesn't override them. Added TestICUWordBoundary and TestICUSentenceBoundary tests to confirm that the boundary scanner works the same with ICU4N as it does with ICU4J with default settings.
---
.../BreakIteratorBoundaryScannerTest.cs | 60 +++++++++++++++++++++-
.../Support/TestJDKBreakIterator.cs | 8 +--
2 files changed, 63 insertions(+), 5 deletions(-)
diff --git a/src/Lucene.Net.Tests.Highlighter/VectorHighlight/BreakIteratorBoundaryScannerTest.cs b/src/Lucene.Net.Tests.Highlighter/VectorHighlight/BreakIteratorBoundaryScannerTest.cs
index 6210fa8..be527c8 100644
--- a/src/Lucene.Net.Tests.Highlighter/VectorHighlight/BreakIteratorBoundaryScannerTest.cs
+++ b/src/Lucene.Net.Tests.Highlighter/VectorHighlight/BreakIteratorBoundaryScannerTest.cs
@@ -1,6 +1,6 @@
#if FEATURE_BREAKITERATOR
using ICU4N.Text;
-using Lucene.Net.ICU.Support;
+using Lucene.Net.Attributes;
using Lucene.Net.Support;
using Lucene.Net.Util;
using NUnit.Framework;
@@ -50,10 +50,30 @@ namespace Lucene.Net.Search.VectorHighlight
assertEquals(start, scanner.FindEndOffset(text, start));
}
+ // LUCENENET specific - Confirmed that ICU4J 60.1 behaves like this by default...
+ [Test, LuceneNetSpecific]
+ public void TestICUWordBoundary()
+ {
+ StringBuilder text = new StringBuilder(TEXT);
+ BreakIterator bi = BreakIterator.GetWordInstance(CultureInfo.InvariantCulture);
+ IBoundaryScanner scanner = new BreakIteratorBoundaryScanner(bi);
+
+ int start = TEXT.IndexOf("formance", StringComparison.Ordinal);
+ int expected = TEXT.IndexOf("performance", StringComparison.Ordinal);
+ TestFindStartOffset(text, start, expected, scanner);
+
+ expected = TEXT.IndexOf(", full", StringComparison.Ordinal);
+ TestFindEndOffset(text, start, expected, scanner);
+ }
+
+ // LUCENENET specific - this is the original Lucene test with a mock BreakIterator that
+ // is intended to act (sort of) like the JDK
[Test]
public void TestWordBoundary()
{
StringBuilder text = new StringBuilder(TEXT);
+ // LUCENENET specific - using a mock of the JDK BreakIterator class, which is just
+ // an ICU BreakIterator with custom rules applied.
BreakIterator bi = JdkBreakIterator.GetWordInstance(CultureInfo.InvariantCulture);
IBoundaryScanner scanner = new BreakIteratorBoundaryScanner(bi);
@@ -65,11 +85,49 @@ namespace Lucene.Net.Search.VectorHighlight
TestFindEndOffset(text, start, expected, scanner);
}
+ // LUCENENET specific - Confirmed that ICU4J 60.1 behaves like this by default...
+ [Test, LuceneNetSpecific]
+ public void TestICUSentenceBoundary()
+ {
+ StringBuilder text = new StringBuilder(TEXT);
+ // we test this with default locale, its randomized by LuceneTestCase
+ BreakIterator bi = BreakIterator.GetSentenceInstance(CultureInfo.CurrentCulture);
+ IBoundaryScanner scanner = new BreakIteratorBoundaryScanner(bi);
+
+ int start = TEXT.IndexOf("any application");
+ int expected = TEXT.IndexOf("It is a");
+ TestFindStartOffset(text, start, expected, scanner);
+
+ expected = TEXT.IndexOf("application that requires") + "application that requires\n".Length;
+ TestFindEndOffset(text, start, expected, scanner);
+ }
+
+ // LUCENENET specific - this is the original Lucene test with a mock BreakIterator that
+ // is intended to act (sort of) like the JDK
[Test]
public void TestSentenceBoundary()
{
+ // LUCENENET specific - using a mock of the JDK BreakIterator class, which is just
+ // an ICU BreakIterator with custom rules applied. East Asian
+ // languages are skipped because the DictionaryBasedBreakIterator is not overridden by the rules.
+ switch (CultureInfo.CurrentCulture.TwoLetterISOLanguageName)
+ {
+ case "th": // Thai
+ case "lo": // Lao
+ case "my": // Burmese
+ case "km": // Khmer
+ case "ja": // Japanese
+ case "ko": // Korean
+ case "zh": // Chinese
+ Assume.That(false, "This test does not apply to East Asian languages.");
+ break;
+ }
+
StringBuilder text = new StringBuilder(TEXT);
// we test this with default locale, its randomized by LuceneTestCase
+
+ // LUCENENET specific - using a mock of the JDK BreakIterator class, which is just
+ // an ICU BreakIterator with custom rules applied.
BreakIterator bi = JdkBreakIterator.GetSentenceInstance(CultureInfo.CurrentCulture);
IBoundaryScanner scanner = new BreakIteratorBoundaryScanner(bi);
diff --git a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
index ea8651c..1e5efbe 100644
--- a/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
+++ b/src/dotnet/Lucene.Net.Tests.ICU/Support/TestJDKBreakIterator.cs
@@ -262,9 +262,9 @@ namespace Lucene.Net.Tests.ICU.Support
Assert.AreEqual(400, bi.Last());
}
- // LUCENENET TODO: This test doesn't pass. We need to customize line iteration in order to get it to. However,
- // none of the defaults set in lucene use line iteration, so this is low priority. Hopefully, someone works it
- // out and makes a contribution.
+ // NOTE: This test doesn't pass. We need to customize line iteration in order to get it to. However,
+ // none of the defaults set in lucene use line iteration, so this is low priority. Leaving in place
+ // in case we need to make JDK style line breaks in the future.
static readonly String LINE_TEXT =
"Apache\tLucene(TM) is a high-\nperformance, full-featured text search engine library written entirely in Java.";
@@ -274,7 +274,7 @@ namespace Lucene.Net.Tests.ICU.Support
}
[Test]
- [Ignore("LUCENENET TODO: Setup JDK style line iteration (see jdksent.rbbi and jdkword.rbbi)")]
+ [Ignore("Not required to confirm compatibility with Java, as this is not required by Lucene's tests.")]
public void TestLineIteration()
{
BreakIterator bi = GetLineInstance(System.Globalization.CultureInfo.InvariantCulture);
[lucenenet] 08/09: Added NOTICE.txt to each NuGet package as per
http://www.apache.org/legal/release-policy.html#licensing-documentation
Posted by ni...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
nightowl888 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/lucenenet.git
commit 48460f0ad6d4295953d2cf35f80e9aa14ca91c1e
Author: Shad Storhaug <sh...@shadstorhaug.com>
AuthorDate: Fri Aug 9 22:17:16 2019 +0700
Added NOTICE.txt to each NuGet package as per http://www.apache.org/legal/release-policy.html#licensing-documentation
---
build/NuGet.props | 2 ++
1 file changed, 2 insertions(+)
diff --git a/build/NuGet.props b/build/NuGet.props
index 237a84d..7e4a421 100644
--- a/build/NuGet.props
+++ b/build/NuGet.props
@@ -27,8 +27,10 @@
<PackageProjectUrl>http://lucenenet.apache.org/</PackageProjectUrl>
<PackageIconUrl>https://github.com/apache/lucenenet/blob/master/branding/logo/lucene-net-icon-128x128.png?raw=true</PackageIconUrl>
<PackageLicenseFile>LICENSE.txt</PackageLicenseFile>
+ <PackageNoticeFile>NOTICE.txt</PackageNoticeFile>
</PropertyGroup>
<ItemGroup>
<None Include="$(SolutionDir)LICENSE.txt" Pack="true" PackagePath="$(PackageLicenseFile)"/>
+ <None Include="$(SolutionDir)NOTICE.txt" Pack="true" PackagePath="$(PackageNoticeFile)"/>
</ItemGroup>
</Project>
\ No newline at end of file