You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucenenet.apache.org by Doug Sale <do...@gmail.com> on 2009/07/31 18:05:44 UTC
Lucene.Net 2.4.0
Regarding the 2.4.0 port:
On Tuesday, I checked in the port that I completed earlier this year.
We've been integrating it into a number of systems at MySpace (thanks
to Bala Rangaraj and Michael Garski for catching and fixing a handful
of bugs).
The initial check-in lacks a few classes/features that exist in the
Lucene Java 2.4.0 release.
TODO
====
1) Implement Lucene.Net.Store.NIOFSDirectory.cs
(org.apache.lucene.store.NIOFSDirectory.java)
This is a Java subclass of FSDirectory that allows multiple threads to
read from a single file without blocking each other. Write behavior
is inherited from FSDirectory. If anyone would like to take a stab at
this, please be my guest.
2) File Descriptor syncing in FSDirectory.cs
FSDirectory.Sync(string) is supposed to sync the underlying file
descriptor, flushing all downstream buffers. I'm not sure how to
accomplish this is C#, and this code needs to be reviewed/fixed/nuked.
3) Lucene.Net.Search.TimeLimitedCollector.cs
I have implemented this class and I am currently working on the unit test.
4) Weak References
I plan to work on these later today.
Lucene.Net.Util.CloseableThreadLocal.cs
- Lucene Java uses weak references
- Lucene.Net is using 'regular' references
Lucene.Net.Search.CachingSpanFilter.cs
Lucene.Net.Search.CachingWrapperFilter.cs
Lucene.Net.Search.FieldCacheImpl.cs
- Lucene Java utilizes java.util.WeakHashMap
- Lucene.Net is not using a data structure with weak references
5) Lucene.Net.Store.CheckSumIndex[Input|Output].cs
These, I completed and checked in on Wednesday. They use, to the best
of my knowledge, the same CRC32 algorithm as the java.util.zip classes
(they have to, for index compatibility between Lucene.Net and Lucene
Java.) I provided a new test, TestSupportClass.cs, to verify the
results against a checksum generated in Java.
Thanks,
Doug
RE: Lucene.Net 2.4.0
Posted by Digy <di...@gmail.com>.
Hi Doug,
There was a WeakHashTable (in SupportClass) in Lucene.Net 2.3.2. It is one
of the classes there are lost with 2.4.0
DIGY
-----Original Message-----
From: Doug Sale [mailto:dougsale@gmail.com]
Sent: Friday, July 31, 2009 7:06 PM
To: lucene-net-dev@incubator.apache.org
Subject: Lucene.Net 2.4.0
Regarding the 2.4.0 port:
On Tuesday, I checked in the port that I completed earlier this year.
We've been integrating it into a number of systems at MySpace (thanks
to Bala Rangaraj and Michael Garski for catching and fixing a handful
of bugs).
The initial check-in lacks a few classes/features that exist in the
Lucene Java 2.4.0 release.
TODO
====
1) Implement Lucene.Net.Store.NIOFSDirectory.cs
(org.apache.lucene.store.NIOFSDirectory.java)
This is a Java subclass of FSDirectory that allows multiple threads to
read from a single file without blocking each other. Write behavior
is inherited from FSDirectory. If anyone would like to take a stab at
this, please be my guest.
2) File Descriptor syncing in FSDirectory.cs
FSDirectory.Sync(string) is supposed to sync the underlying file
descriptor, flushing all downstream buffers. I'm not sure how to
accomplish this is C#, and this code needs to be reviewed/fixed/nuked.
3) Lucene.Net.Search.TimeLimitedCollector.cs
I have implemented this class and I am currently working on the unit test.
4) Weak References
I plan to work on these later today.
Lucene.Net.Util.CloseableThreadLocal.cs
- Lucene Java uses weak references
- Lucene.Net is using 'regular' references
Lucene.Net.Search.CachingSpanFilter.cs
Lucene.Net.Search.CachingWrapperFilter.cs
Lucene.Net.Search.FieldCacheImpl.cs
- Lucene Java utilizes java.util.WeakHashMap
- Lucene.Net is not using a data structure with weak references
5) Lucene.Net.Store.CheckSumIndex[Input|Output].cs
These, I completed and checked in on Wednesday. They use, to the best
of my knowledge, the same CRC32 algorithm as the java.util.zip classes
(they have to, for index compatibility between Lucene.Net and Lucene
Java.) I provided a new test, TestSupportClass.cs, to verify the
results against a checksum generated in Java.
Thanks,
Doug
RE: Lucene.Net 2.4.0
Posted by Digy <di...@gmail.com>.
I think that method can be used for FSDirectory.sync
[System.Runtime.InteropServices.DllImport("kernel32")]
public static extern int
FlushFileBuffers(Microsoft.Win32.SafeHandles.SafeFileHandle SafeFileHandle);
public static void Sync(FileStream fs)
{
if (FlushFileBuffers(fs.SafeFileHandle) == 0)
{
throw new SyncFailedException();
}
}
public class SyncFailedException : Exception
{
}
DIGY
-----Original Message-----
From: Doug Sale [mailto:dougsale@gmail.com]
Sent: Friday, July 31, 2009 7:06 PM
To: lucene-net-dev@incubator.apache.org
Subject: Lucene.Net 2.4.0
Regarding the 2.4.0 port:
On Tuesday, I checked in the port that I completed earlier this year.
We've been integrating it into a number of systems at MySpace (thanks
to Bala Rangaraj and Michael Garski for catching and fixing a handful
of bugs).
The initial check-in lacks a few classes/features that exist in the
Lucene Java 2.4.0 release.
TODO
====
1) Implement Lucene.Net.Store.NIOFSDirectory.cs
(org.apache.lucene.store.NIOFSDirectory.java)
This is a Java subclass of FSDirectory that allows multiple threads to
read from a single file without blocking each other. Write behavior
is inherited from FSDirectory. If anyone would like to take a stab at
this, please be my guest.
2) File Descriptor syncing in FSDirectory.cs
FSDirectory.Sync(string) is supposed to sync the underlying file
descriptor, flushing all downstream buffers. I'm not sure how to
accomplish this is C#, and this code needs to be reviewed/fixed/nuked.
3) Lucene.Net.Search.TimeLimitedCollector.cs
I have implemented this class and I am currently working on the unit test.
4) Weak References
I plan to work on these later today.
Lucene.Net.Util.CloseableThreadLocal.cs
- Lucene Java uses weak references
- Lucene.Net is using 'regular' references
Lucene.Net.Search.CachingSpanFilter.cs
Lucene.Net.Search.CachingWrapperFilter.cs
Lucene.Net.Search.FieldCacheImpl.cs
- Lucene Java utilizes java.util.WeakHashMap
- Lucene.Net is not using a data structure with weak references
5) Lucene.Net.Store.CheckSumIndex[Input|Output].cs
These, I completed and checked in on Wednesday. They use, to the best
of my knowledge, the same CRC32 algorithm as the java.util.zip classes
(they have to, for index compatibility between Lucene.Net and Lucene
Java.) I provided a new test, TestSupportClass.cs, to verify the
results against a checksum generated in Java.
Thanks,
Doug
RE: Lucene.Net 2.4.0
Posted by Digy <di...@gmail.com>.
Hi Doug,
Here is a list of lost bug fixes/improvements with v2.4.0
LUCENENET-183
LUCENENET-182
LUCENENET-175
LUCENENET-174
LUCENENET-170
LUCENENET-169
LUCENENET-168
LUCENENET-163
LUCENENET-160
LUCENENET-159
LUCENENET-106
DIGY
-----Original Message-----
From: Doug Sale [mailto:dougsale@gmail.com]
Sent: Friday, July 31, 2009 7:06 PM
To: lucene-net-dev@incubator.apache.org
Subject: Lucene.Net 2.4.0
Regarding the 2.4.0 port:
On Tuesday, I checked in the port that I completed earlier this year.
We've been integrating it into a number of systems at MySpace (thanks
to Bala Rangaraj and Michael Garski for catching and fixing a handful
of bugs).
The initial check-in lacks a few classes/features that exist in the
Lucene Java 2.4.0 release.
TODO
====
1) Implement Lucene.Net.Store.NIOFSDirectory.cs
(org.apache.lucene.store.NIOFSDirectory.java)
This is a Java subclass of FSDirectory that allows multiple threads to
read from a single file without blocking each other. Write behavior
is inherited from FSDirectory. If anyone would like to take a stab at
this, please be my guest.
2) File Descriptor syncing in FSDirectory.cs
FSDirectory.Sync(string) is supposed to sync the underlying file
descriptor, flushing all downstream buffers. I'm not sure how to
accomplish this is C#, and this code needs to be reviewed/fixed/nuked.
3) Lucene.Net.Search.TimeLimitedCollector.cs
I have implemented this class and I am currently working on the unit test.
4) Weak References
I plan to work on these later today.
Lucene.Net.Util.CloseableThreadLocal.cs
- Lucene Java uses weak references
- Lucene.Net is using 'regular' references
Lucene.Net.Search.CachingSpanFilter.cs
Lucene.Net.Search.CachingWrapperFilter.cs
Lucene.Net.Search.FieldCacheImpl.cs
- Lucene Java utilizes java.util.WeakHashMap
- Lucene.Net is not using a data structure with weak references
5) Lucene.Net.Store.CheckSumIndex[Input|Output].cs
These, I completed and checked in on Wednesday. They use, to the best
of my knowledge, the same CRC32 algorithm as the java.util.zip classes
(they have to, for index compatibility between Lucene.Net and Lucene
Java.) I provided a new test, TestSupportClass.cs, to verify the
results against a checksum generated in Java.
Thanks,
Doug
RE: Lucene.Net 2.4.0
Posted by Digy <di...@gmail.com>.
Hi Doug,
WeakHashTable(implemented in SupportClass) was used in FieldCacheImpl,
CachingSpanFilter, CachingWrapperFilter in v2.3.2 because HashTable had
caused memory leak
(http://issues.apache.org/jira/browse/LUCENENET-106) . But as I said before,
some patches made to Lucene.Net 2.3.2(2.3.1?) are lost with your new 2.4.0
DIGY
-----Original Message-----
From: Doug Sale [mailto:dougsale@gmail.com]
Sent: Friday, July 31, 2009 7:06 PM
To: lucene-net-dev@incubator.apache.org
Subject: Lucene.Net 2.4.0
Regarding the 2.4.0 port:
On Tuesday, I checked in the port that I completed earlier this year.
We've been integrating it into a number of systems at MySpace (thanks
to Bala Rangaraj and Michael Garski for catching and fixing a handful
of bugs).
The initial check-in lacks a few classes/features that exist in the
Lucene Java 2.4.0 release.
TODO
====
1) Implement Lucene.Net.Store.NIOFSDirectory.cs
(org.apache.lucene.store.NIOFSDirectory.java)
This is a Java subclass of FSDirectory that allows multiple threads to
read from a single file without blocking each other. Write behavior
is inherited from FSDirectory. If anyone would like to take a stab at
this, please be my guest.
2) File Descriptor syncing in FSDirectory.cs
FSDirectory.Sync(string) is supposed to sync the underlying file
descriptor, flushing all downstream buffers. I'm not sure how to
accomplish this is C#, and this code needs to be reviewed/fixed/nuked.
3) Lucene.Net.Search.TimeLimitedCollector.cs
I have implemented this class and I am currently working on the unit test.
4) Weak References
I plan to work on these later today.
Lucene.Net.Util.CloseableThreadLocal.cs
- Lucene Java uses weak references
- Lucene.Net is using 'regular' references
Lucene.Net.Search.CachingSpanFilter.cs
Lucene.Net.Search.CachingWrapperFilter.cs
Lucene.Net.Search.FieldCacheImpl.cs
- Lucene Java utilizes java.util.WeakHashMap
- Lucene.Net is not using a data structure with weak references
5) Lucene.Net.Store.CheckSumIndex[Input|Output].cs
These, I completed and checked in on Wednesday. They use, to the best
of my knowledge, the same CRC32 algorithm as the java.util.zip classes
(they have to, for index compatibility between Lucene.Net and Lucene
Java.) I provided a new test, TestSupportClass.cs, to verify the
results against a checksum generated in Java.
Thanks,
Doug