You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by George Aroush <ge...@aroush.net> on 2006/06/14 05:51:22 UTC
Lucene.Net 1.9 RC1 Build 4 beta is ready
Hi Folks,
I just released build 4 of Lucene.Net 1.9 -- which I am labeling as "beta".
With this release, the number of failed test is now 44 out of 307 -- some of
those are failing because they are web test which I don't have setup.
Here is the release note:
- Release: Lucene.Net.1.9 RC1 build 004 Beta
- Fix: Lucene.Net.Search.FieldCacheImpl.GetFloats() -- must remove 'f" or
"F" for System.Single.Parse() to work
- Fix: Lucene.Net.Index.GetFieldnames() -- was storing the object instead
the value in the object
- Fix: Test.Lucene.Net.Index.CollectionContains() -- need to compare
strings, not objects
- Fix: Test.Lucene.Net.Serch.TestKnownSetOfDocuments() -- don't fail if an
item doesn't exist
Again, I have to ask, is anyone running the NUnit test and taking on those
failed tests? Is anyone got the cycles to help? Is anyone still interested
and committed to Lucene.Net?
My plan is to finish off the 1.9 by end of June. Then in July, take on
1.9.1 and finally, 2.0. If folks can pitch in, we can do this much sooner.
Regards,
-- George
Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by jxf jxf <dj...@gmail.com>.
In Index\SegmentTermVector.cs
public virtual int IndexOf(System.String termText)
{
....
int res = System.Array.BinarySearch(terms, termText);
....
}
Array.BinarySearch( method by default use the System.IComparable to compare
the two objects, here
string.Compare( will be called, but actually it is the method
string.CompareOrdinal( which should be called I
guess.
Because the term text string[] is sorted in the ASCII order which means
lower case characters are bigger than all Capital ones.
example:
string[] sTermText =
new string[]{"Clear","atom","basic","cat","dog"}; int res =
Array.BinarySearch( sTermText, "Clear" );
or {"C#","atom","basic","cat","dog","edge","fly"} int res =
Array.BinarySearch( sTermText, "C#" );
// res < 0, it is obviously wrong in this case.
it can fix this???
class MyComparer : System.Collections.IComparer
{
public int Compare(object x, object y)
{
return string.CompareOrdinal(x as string, y as string);
}
}
int res = System.Array.BinarySearch(terms, termText, new MyComparer());
2006/6/15, Ben Tregenna <lu...@rekenys.com>:
>
> George Aroush wrote:
>
> >
> >Again, I have to ask, is anyone running the NUnit test and taking on
> those
> >failed tests?
> >
>
> I know how to fix the test, "TestISOLatin1AccentFilter". The problem is
> that the source file isn't saved in unicode format and so the text
> passed into the analyzer is the 'ascii-fication' of the actual ISOLatin
> string. I fixed this test locally by choosing to "Save
> TestISOLatin1AccentFilter.cs As..." and then selecting "Save with
> encoding" from the 'Save button drop-down menu' and choosing 'unicode
> with signature' from the subsequent list. This sticks the correct
> ByteOrderMark at the begining of the file. I then copy/pasted in the
> original test strings as found in lucene-java. The svn patch is attached
> below, I think the first bit of the patch "-/* +/*" is the
> ByteOrderMark bit which is the crucial piece.
>
> I've seen this issue before when trying to test CJK stuff using inline
> strings and the setting for saving as unicode is pretty well hidden in VS.
>
> Ben
>
>
>
>
> ===================================================================
> --- TestISOLatin1AccentFilter.cs (revision 414223)
> +++ TestISOLatin1AccentFilter.cs (working copy)
> @@ -1,4 +1,4 @@
> -/*
> +/*
> * Copyright 2005 The Apache Software Foundation
> *
> * Licensed under the Apache License, Version 2.0 (the "License");
> @@ -25,7 +25,7 @@
> [Test]
> public virtual void TestU()
> {
> - TokenStream stream = new WhitespaceTokenizer(new
> System.IO.StringReader("Des mot clÃ(c)s À LA CHAÃŽNE À Ã? Â Ã Ä Ã… Æ
> Ç È É Ê Ë ÃŒ Ã? ÃŽ Ã? Ã? Ã' Ã' Ã" Ã" Õ Ö Ø Å' Þ Ù Ú Û Ãœ Ã?
> Ÿ à á â ã ä Ã¥ æ ç è Ã(c) ê ë ì à Ã(r) ï ð ñ ò ó ô õ ö
> ø Å" ß þ ù ú û ü ý ÿ"));
> + TokenStream stream = new WhitespaceTokenizer(new
> System.IO.StringReader("Des mot clÃ(c)s À LA CHAÃŽNE À à Â Ã Ä Ã… Æ
> Ç È É Ê Ë ÃŒ à Î à à Ã' Ã' Ã" Ã" Õ Ö Ø Å' Þ Ù Ú Û Ãœ Ã
> Ÿ à á â ã ä Ã¥ æ ç è Ã(c) ê ë ì à Ã(r) ï ð ñ ò ó ô õ ö
> ø Å" ß þ ù ú û ü ý ÿ"));
> ISOLatin1AccentFilter filter = new ISOLatin1AccentFilter(stream);
> Assert.AreEqual("Des", filter.Next().TermText());
> Assert.AreEqual("mot", filter.Next().TermText());
>
>
RE: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by George Aroush <ge...@aroush.net>.
Hi Ben,
Yes they are needed otherwise the test will fail. Few night ago, I fixed
those and I should check them in tonight. The fix is to use MemoryStream
like so:
System.IO.MemoryStream sw = new System.IO.MemoryStream();
System.IO.StreamWriter pw = new System.IO.StreamWriter(sw);
Again, thanks for taking your time and looking at code. There are other
code outside the Test code -- in the Lucene.Net core code -- just search for
"Aroush". Let me know if you have anything to say about them.
Regards,
-- George
-----Original Message-----
From: Ben Tregenna [mailto:lucene@rekenys.com]
Sent: Thursday, June 15, 2006 6:06 PM
To: lucene-net-dev@incubator.apache.org
Cc: George Aroush
Subject: Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
As far as I can tell the StreamWriter in the TestSearch.cs file is not
really needed - it looks from the '(aroush)' comments that the intention was
just to wrap the StringWriter in a StreamWriter, but I don't think this
needs doing. The only methods called on pw in DoTestSearch are TextWriter
methods that they both implement so using the StreamWriter directly appears
fine - or at least consistent which is all the test actually tests Patch
follows...
Ben
Index: TestSearch.cs
===================================================================
--- TestSearch.cs (revision 414637)
+++ TestSearch.cs (working copy)
@@ -56,17 +56,12 @@
public virtual void TestSearch_Renamed_Method()
{
System.IO.StringWriter sw = new System.IO.StringWriter();
- System.IO.StreamWriter pw = null; // new
System.IO.StreamWriter(sw); // {{Aroush}} how do you pass 'sw' to
StreamWriter()?
- DoTestSearch(pw, false);
- pw.Close();
+ DoTestSearch(sw, false);
sw.Close();
System.String multiFileOutput =
sw.GetStringBuilder().ToString();
- //System.out.println(multiFileOutput);
sw = new System.IO.StringWriter();
- pw = null; // new System.IO.StreamWriter(sw); //
{{Aroush}} how do you pass 'sw' to StreamWriter()?
- DoTestSearch(pw, true);
- pw.Close();
+ DoTestSearch(sw, true);
sw.Close();
System.String singleFileOutput =
sw.GetStringBuilder().ToString();
@@ -74,7 +69,7 @@
}
- private void DoTestSearch(System.IO.StreamWriter out_Renamed,
bool useCompoundFile)
+ private void DoTestSearch(System.IO.StringWriter out_Renamed,
bool useCompoundFile)
{
Directory directory = new RAMDirectory();
Analyzer analyzer = new SimpleAnalyzer();
Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by Ben Tregenna <lu...@rekenys.com>.
As far as I can tell the StreamWriter in the TestSearch.cs file is not
really needed - it looks from the '(aroush)' comments that the intention
was just to wrap the StringWriter in a StreamWriter, but I don't think
this needs doing. The only methods called on pw in DoTestSearch are
TextWriter methods that they both implement so using the StreamWriter
directly appears fine - or at least consistent which is all the test
actually tests Patch follows...
Ben
Index: TestSearch.cs
===================================================================
--- TestSearch.cs (revision 414637)
+++ TestSearch.cs (working copy)
@@ -56,17 +56,12 @@
public virtual void TestSearch_Renamed_Method()
{
System.IO.StringWriter sw = new System.IO.StringWriter();
- System.IO.StreamWriter pw = null; // new
System.IO.StreamWriter(sw); // {{Aroush}} how do you pass 'sw' to
StreamWriter()?
- DoTestSearch(pw, false);
- pw.Close();
+ DoTestSearch(sw, false);
sw.Close();
System.String multiFileOutput =
sw.GetStringBuilder().ToString();
- //System.out.println(multiFileOutput);
sw = new System.IO.StringWriter();
- pw = null; // new System.IO.StreamWriter(sw); //
{{Aroush}} how do you pass 'sw' to StreamWriter()?
- DoTestSearch(pw, true);
- pw.Close();
+ DoTestSearch(sw, true);
sw.Close();
System.String singleFileOutput =
sw.GetStringBuilder().ToString();
@@ -74,7 +69,7 @@
}
- private void DoTestSearch(System.IO.StreamWriter out_Renamed,
bool useCompoundFile)
+ private void DoTestSearch(System.IO.StringWriter out_Renamed,
bool useCompoundFile)
{
Directory directory = new RAMDirectory();
Analyzer analyzer = new SimpleAnalyzer();
Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by Ben Tregenna <lu...@rekenys.com>.
George Aroush wrote:
>Thanks Ben, that did it!
>
>If you have cycles and can take on other open issues, please do.
>
>
>
>
OK, another one -
The TestNumberTools are failing as the expectation from the java tests
are that you convert longs to base36 strings and the NumberTools was
converting them to base16. I reckon that was done because
System.Convert.ToInt6() only takes bases up to 16, but fortunately
SupportClass.Number has methods for doing base 36 so using that and
changing the base/radix in NumberTools back to 36 fixes the failures in
TestNumberTools. See below for patch....
Ben
Index: NumberTools.cs
===================================================================
--- NumberTools.cs (revision 414223)
+++ NumberTools.cs (working copy)
@@ -37,7 +37,7 @@
public class NumberTools
{
- private const int RADIX = 16; // 36; {{Arousdh-1.9}} Java's
is 36, but .NET's is 16; will this be an issue?
+ private const int RADIX = 36; // 36; {{Arousdh-1.9}} Java's
is 36, but .NET's is 16; will this be an issue?
private const char NEGATIVE_PREFIX = '-';
@@ -75,7 +75,7 @@
{
buf.Append(POSITIVE_PREFIX);
}
- System.String num = System.Convert.ToString(l, RADIX);
+ System.String num = SupportClass.Number.ToString(l, RADIX);
int padLen = STR_SIZE - num.Length - buf.Length;
while (padLen-- > 0)
@@ -115,7 +115,7 @@
}
char prefix = str[0];
- long l = System.Convert.ToInt64(str.Substring(1), RADIX);
+ long l = SupportClass.Number.Parse(str.Substring(1), RADIX);
if (prefix == POSITIVE_PREFIX)
{
RE: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by George Aroush <ge...@aroush.net>.
Thanks Ben, that did it!
If you have cycles and can take on other open issues, please do.
Regards,
-- George Aroush
-----Original Message-----
From: Ben Tregenna [mailto:lucene@rekenys.com]
Sent: Wednesday, June 14, 2006 2:11 PM
To: lucene-net-dev@incubator.apache.org; George Aroush
Subject: Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
George Aroush wrote:
>
>Again, I have to ask, is anyone running the NUnit test and taking on
>those failed tests?
>
I know how to fix the test, "TestISOLatin1AccentFilter". The problem is
that the source file isn't saved in unicode format and so the text
passed into the analyzer is the 'ascii-fication' of the actual ISOLatin
string. I fixed this test locally by choosing to "Save
TestISOLatin1AccentFilter.cs As..." and then selecting "Save with
encoding" from the 'Save button drop-down menu' and choosing 'unicode
with signature' from the subsequent list. This sticks the correct
ByteOrderMark at the begining of the file. I then copy/pasted in the
original test strings as found in lucene-java. The svn patch is attached
below, I think the first bit of the patch "-/* +/*" is the
ByteOrderMark bit which is the crucial piece.
I've seen this issue before when trying to test CJK stuff using inline
strings and the setting for saving as unicode is pretty well hidden in VS.
Ben
===================================================================
--- TestISOLatin1AccentFilter.cs (revision 414223)
+++ TestISOLatin1AccentFilter.cs (working copy)
@@ -1,4 +1,4 @@
-/*
+/*
* Copyright 2005 The Apache Software Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
@@ -25,7 +25,7 @@
[Test]
public virtual void TestU()
{
- TokenStream stream = new WhitespaceTokenizer(new
System.IO.StringReader("Des mot clés À LA CHAÎNE À � Â Ã Ä Å Æ
Ç È É Ê Ë Ì � Î � � Ñ Ò Ó Ô Õ Ö Ø Œ Þ Ù Ú Û Ü �
Ÿ à á â ã ä å æ ç è é ê ë ì à î ï ð ñ ò ó ô õ ö
ø œ ß þ ù ú û ü ý ÿ"));
+ TokenStream stream = new WhitespaceTokenizer(new
System.IO.StringReader("Des mot clés À LA CHAÎNE À à Â Ã Ä Å Æ
Ç È É Ê Ë ÃŒ à Î à à Ñ Ã’ Ó Ô Õ Ö Ø Å’ Þ Ù Ú Û Ãœ Ã
Ÿ à á â ã ä å æ ç è é ê ë ì à î ï ð ñ ò ó ô õ ö
ø œ ß þ ù ú û ü ý ÿ"));
ISOLatin1AccentFilter filter = new ISOLatin1AccentFilter(stream);
Assert.AreEqual("Des", filter.Next().TermText());
Assert.AreEqual("mot", filter.Next().TermText());
Re: Lucene.Net 1.9 RC1 Build 4 beta is ready
Posted by Ben Tregenna <lu...@rekenys.com>.
George Aroush wrote:
>
>Again, I have to ask, is anyone running the NUnit test and taking on those
>failed tests?
>
I know how to fix the test, "TestISOLatin1AccentFilter". The problem is
that the source file isn't saved in unicode format and so the text
passed into the analyzer is the 'ascii-fication' of the actual ISOLatin
string. I fixed this test locally by choosing to "Save
TestISOLatin1AccentFilter.cs As..." and then selecting "Save with
encoding" from the 'Save button drop-down menu' and choosing 'unicode
with signature' from the subsequent list. This sticks the correct
ByteOrderMark at the begining of the file. I then copy/pasted in the
original test strings as found in lucene-java. The svn patch is attached
below, I think the first bit of the patch "-/* +/*" is the
ByteOrderMark bit which is the crucial piece.
I've seen this issue before when trying to test CJK stuff using inline
strings and the setting for saving as unicode is pretty well hidden in VS.
Ben
===================================================================
--- TestISOLatin1AccentFilter.cs (revision 414223)
+++ TestISOLatin1AccentFilter.cs (working copy)
@@ -1,4 +1,4 @@
-/*
+/*
* Copyright 2005 The Apache Software Foundation
*
* Licensed under the Apache License, Version 2.0 (the "License");
@@ -25,7 +25,7 @@
[Test]
public virtual void TestU()
{
- TokenStream stream = new WhitespaceTokenizer(new
System.IO.StringReader("Des mot clés À LA CHAÎNE À �? Â Ã Ä Å Æ
Ç È É Ê Ë Ì �? Î �? �? Ñ Ò Ó Ô Õ Ö Ø Œ Þ Ù Ú Û Ü �?
Ÿ � á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö
ø œ ß þ ù ú û ü ý ÿ"));
+ TokenStream stream = new WhitespaceTokenizer(new
System.IO.StringReader("Des mot clés À LA CHAÎNE À Á Â Ã Ä Å Æ
Ç È É Ê Ë Ì Í Î Ï Ð Ñ Ò Ó Ô Õ Ö Ø Œ Þ Ù Ú Û Ü Ý
Ÿ � á â ã ä å æ ç è é ê ë ì í î ï ð ñ ò ó ô õ ö
ø œ ß þ ù ú û ü ý ÿ"));
ISOLatin1AccentFilter filter = new ISOLatin1AccentFilter(stream);
Assert.AreEqual("Des", filter.Next().TermText());
Assert.AreEqual("mot", filter.Next().TermText());