You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@lucenenet.apache.org by 蒋建亮 <ji...@gmail.com> on 2021/01/25 02:33:15 UTC
How to serialize and deserialize content in RAMDirectory
Hi Lucene.Net experts,
Thanks for developing Lucene.Net. We'd like to leverage to Lucene.Net's
RAMDirectory to build/servce index. However, our environment has some
restrictions,
we have to separate index building/serving into two processes(and two
processes are in different machines)
As a result, we need to figure out a way to serialize/deserialize content
inside RAMDirectory.
Current approach is that we subclass RAMDirectory and RAMFile to support
accessing internal members. After 'serialization' and 'deserialization', we
can get another instance of Directory, which has
the same internal states as original one. The weird thing is that we
encountered below exceptions when loading with new Directory instance
*System.IO.EndOfStreamException: read past EOF:
RAMInputStream(name=segments_1)*
So is there any gaps we are not aware of?
Attach source code FYI.
--
Jianliang Jiang
RE: How to serialize and deserialize content in RAMDirectory
Posted by Shad Storhaug <sh...@shadstorhaug.com>.
Jianliang,
If all you need is to share the same index across 2 machines, you could use the Lucene.Net.Replicator to accomplish it. Replicator is designed to keep indexes in sync using snapshot views of all of the index files at a particular instant in time. See the following comment for a basic example and the documentation.
https://github.com/apache/lucenenet/issues/382#issuecomment-720782173
Alternatively, you could put your index behind a web API and share the API between your two machines using the API and a single instance of IndexWriter for both writing documents and searching the index with near real-time search (NRT).
Cheers,
Shad
-----Original Message-----
From: 蒋建亮 <ji...@gmail.com>
Sent: Monday, January 25, 2021 2:16 PM
To: user@lucenenet.apache.org
Subject: Re: How to serialize and deserialize content in RAMDirectory
I forgot to maintain Length Property when serialization and deserialization.
Complete example can be found at
https://github.com/jiangjianliang/Examples/tree/master/ExtendLucene
蒋建亮 <ji...@gmail.com> 于2021年1月25日周一 上午10:33写道:
> Hi Lucene.Net experts,
>
> Thanks for developing Lucene.Net. We'd like to leverage to
> Lucene.Net's RAMDirectory to build/servce index. However, our
> environment has some restrictions, we have to separate index
> building/serving into two processes(and two processes are in different
> machines)
>
> As a result, we need to figure out a way to serialize/deserialize
> content inside RAMDirectory.
>
> Current approach is that we subclass RAMDirectory and RAMFile to
> support accessing internal members. After 'serialization' and
> 'deserialization', we can get another instance of Directory, which has
> the same internal states as original one. The weird thing is that we
> encountered below exceptions when loading with new Directory instance
> *System.IO.EndOfStreamException: read past EOF:
> RAMInputStream(name=segments_1)*
>
> So is there any gaps we are not aware of?
>
> Attach source code FYI.
>
> --
> Jianliang Jiang
>
>
--
Jianliang Jiang
Institute of Computer Software
Nanjing University
China
Re: How to serialize and deserialize content in RAMDirectory
Posted by 蒋建亮 <ji...@gmail.com>.
I forgot to maintain Length Property when serialization and deserialization.
Complete example can be found at
https://github.com/jiangjianliang/Examples/tree/master/ExtendLucene
蒋建亮 <ji...@gmail.com> 于2021年1月25日周一 上午10:33写道:
> Hi Lucene.Net experts,
>
> Thanks for developing Lucene.Net. We'd like to leverage to Lucene.Net's
> RAMDirectory to build/servce index. However, our environment has some
> restrictions,
> we have to separate index building/serving into two processes(and two
> processes are in different machines)
>
> As a result, we need to figure out a way to serialize/deserialize content
> inside RAMDirectory.
>
> Current approach is that we subclass RAMDirectory and RAMFile to support
> accessing internal members. After 'serialization' and 'deserialization', we
> can get another instance of Directory, which has
> the same internal states as original one. The weird thing is that we
> encountered below exceptions when loading with new Directory instance
> *System.IO.EndOfStreamException: read past EOF:
> RAMInputStream(name=segments_1)*
>
> So is there any gaps we are not aware of?
>
> Attach source code FYI.
>
> --
> Jianliang Jiang
>
>
--
Jianliang Jiang
Institute of Computer Software
Nanjing University
China