You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com> on 2009/04/13 06:21:10 UTC
Can I run Lucene in google app engine?
Is it possible to run Lucene in google app engine? has anyone tried it?
--
--Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Konstantyn Smirnov <in...@yahoo.com>.
Thanks Mike, I found it.
It's a really elegant way, to serialize the object. No special serialize()
methods, just dump in to stram - that's it :)
--
View this message in context: http://lucene.472066.n3.nabble.com/Can-I-run-Lucene-in-google-app-engine-tp560098p2063140.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Michael McCandless <lu...@mikemccandless.com>.
It's the testSerializable method in
org/apache/lucene/test/store/TestRAMDirectory.java
Mike
On Wed, Dec 8, 2010 at 7:06 PM, Konstantyn Smirnov <in...@yahoo.com> wrote:
>
> Hi Mike,
>
> can you please elaborate?
> Where can I find the test?
>
> TIA
>
>
> Michael McCandless-2 wrote:
>>
>> Yes, I believe so (we have a unit test asserting this).
>>
>> But, there's no guarantee of cross-version compatibility of the serialized
>> form.
>>
>> Mike
>>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Can-I-run-Lucene-in-google-app-engine-tp560098p2054691.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Konstantyn Smirnov <in...@yahoo.com>.
Hi Mike,
can you please elaborate?
Where can I find the test?
TIA
Michael McCandless-2 wrote:
>
> Yes, I believe so (we have a unit test asserting this).
>
> But, there's no guarantee of cross-version compatibility of the serialized
> form.
>
> Mike
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Can-I-run-Lucene-in-google-app-engine-tp560098p2054691.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
yeah we never know which jvm has the latest data
you can have a separate app for writes
say solr_master.appspot.com
instead of POSTing data to the master it may be able to pull data (say
using DataImportHandler) and index it and when it has to commit it can
serialize the data into a Blob and save it. This ensures that all the
writes happen on 1 single host
The slaves can have a cron job which polls for the latest data and
fetch the index and update itself
The catch here is that we must have a small index were everything can
be done in 30 secs (that is the longest a task can run).
--Noble
On Mon, Apr 13, 2009 at 3:55 PM, Erdinc Yilmazel
<er...@gmail.com> wrote:
> Do you think writing a Directory based on google datastore is going to
> be useful? Indexing may be done using the cron jobs. I am not sure how
> the RAMDirectory will fit because the requests can be scattered to
> very different jvm instances in app engine environment.
>
> Erdinc
>
> On Mon, Apr 13, 2009 at 11:11 AM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>> Yes, I believe so (we have a unit test asserting this).
>>
>> But, there's no guarantee of cross-version compatibility of the serialized form.
>>
>> Mike
>>
>> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>>> that is great.
>>>
>>> Is it possible to serialize and deserialize a RAMDirectory?
>>>
>>> --Noble
>>>
>>>
>>>
>>>
>>> On Mon, Apr 13, 2009 at 3:29 PM, Michael McCandless
>>> <lu...@mikemccandless.com> wrote:
>>>> Lucene's ConcurrentMergeScheduler spawns threads, but you can switch
>>>> that to SerialMergeScheduler which does not use threads.
>>>>
>>>> Some optional classes (FilterManager, ParallelMultiSearcher,
>>>> TimeLimitedCollector) spawn threads, but presumably you could avoid
>>>> them.
>>>>
>>>> Otherwise Lucene doesn't create its own threads.
>>>>
>>>> Mike
>>>>
>>>> On Mon, Apr 13, 2009 at 5:20 AM, Erdinc Yilmazel
>>>> <er...@gmail.com> wrote:
>>>>> I looked at the lucene source code out of curiosity to see if it
>>>>> spawns threads in indexing/searching and saw that it does. Are you
>>>>> sure it is going to work even if the index is read only?
>>>>>
>>>>> Erdinc
>>>>>
>>>>> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>>>>>> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>>>>>>> Surely it's possible, but it has too much limitations to prevent a scalable
>>>>>>> Luceen usage. However, it depends on your requirement.
>>>>>>>
>>>>>>> 1) You can not write index on disk, but you can read files. So theoretically
>>>>>>> if the index is read-only and small, you can package it with the war file.
>>>>>> I can probably write the whole index as Blob and read it and serve
>>>>>> the index from RAM. This index can be loaded once in the start time
>>>>>> and i can setup a cron task to see if it got updated
>>>>>>>
>>>>>>> 2) If you need to update the index, you will have to store the index with
>>>>>>> Google's data store, just like store an index into databases. Sure it'll
>>>>>>> work. But performance would suffer because of transferring the whole index
>>>>>>> into memory, then really start searching. On the other hand, this could be a
>>>>>>> good solution for small index with per-user data.
>>>>>> as you said this is a solution if the index size is small
>>>>>>
>>>>>>>
>>>>>>> 3) For large changing indexes, you need to find other solutions to maintain
>>>>>>> lucene index.
>>>>>>>
>>>>>>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>>>>>>> changing the way to code.
>>>>>>
>>>>>> The point is ,does Lucens use threads internally which the sandbox disallows
>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Chris Lu
>>>>>>> -------------------------
>>>>>>> Instant Scalable Full-Text Search On Any Database/Application
>>>>>>> site: http://www.dbsight.net
>>>>>>> demo: http://search.dbsight.com
>>>>>>> Lucene Database Search in 3 minutes:
>>>>>>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>>>>>>> DBSight customer, a shopping comparison site, (anonymous per request) got
>>>>>>> 2.6 Million Euro funding!
>>>>>>>
>>>>>>>
>>>>>>> Noble Paul ??????? ?????? wrote:
>>>>>>>>
>>>>>>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> --Noble Paul
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Noble Paul
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
--Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Erdinc Yilmazel <er...@gmail.com>.
Do you think writing a Directory based on google datastore is going to
be useful? Indexing may be done using the cron jobs. I am not sure how
the RAMDirectory will fit because the requests can be scattered to
very different jvm instances in app engine environment.
Erdinc
On Mon, Apr 13, 2009 at 11:11 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Yes, I believe so (we have a unit test asserting this).
>
> But, there's no guarantee of cross-version compatibility of the serialized form.
>
> Mike
>
> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>> that is great.
>>
>> Is it possible to serialize and deserialize a RAMDirectory?
>>
>> --Noble
>>
>>
>>
>>
>> On Mon, Apr 13, 2009 at 3:29 PM, Michael McCandless
>> <lu...@mikemccandless.com> wrote:
>>> Lucene's ConcurrentMergeScheduler spawns threads, but you can switch
>>> that to SerialMergeScheduler which does not use threads.
>>>
>>> Some optional classes (FilterManager, ParallelMultiSearcher,
>>> TimeLimitedCollector) spawn threads, but presumably you could avoid
>>> them.
>>>
>>> Otherwise Lucene doesn't create its own threads.
>>>
>>> Mike
>>>
>>> On Mon, Apr 13, 2009 at 5:20 AM, Erdinc Yilmazel
>>> <er...@gmail.com> wrote:
>>>> I looked at the lucene source code out of curiosity to see if it
>>>> spawns threads in indexing/searching and saw that it does. Are you
>>>> sure it is going to work even if the index is read only?
>>>>
>>>> Erdinc
>>>>
>>>> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>>>>> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>>>>>> Surely it's possible, but it has too much limitations to prevent a scalable
>>>>>> Luceen usage. However, it depends on your requirement.
>>>>>>
>>>>>> 1) You can not write index on disk, but you can read files. So theoretically
>>>>>> if the index is read-only and small, you can package it with the war file.
>>>>> I can probably write the whole index as Blob and read it and serve
>>>>> the index from RAM. This index can be loaded once in the start time
>>>>> and i can setup a cron task to see if it got updated
>>>>>>
>>>>>> 2) If you need to update the index, you will have to store the index with
>>>>>> Google's data store, just like store an index into databases. Sure it'll
>>>>>> work. But performance would suffer because of transferring the whole index
>>>>>> into memory, then really start searching. On the other hand, this could be a
>>>>>> good solution for small index with per-user data.
>>>>> as you said this is a solution if the index size is small
>>>>>
>>>>>>
>>>>>> 3) For large changing indexes, you need to find other solutions to maintain
>>>>>> lucene index.
>>>>>>
>>>>>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>>>>>> changing the way to code.
>>>>>
>>>>> The point is ,does Lucens use threads internally which the sandbox disallows
>>>>>
>>>>>>
>>>>>> --
>>>>>> Chris Lu
>>>>>> -------------------------
>>>>>> Instant Scalable Full-Text Search On Any Database/Application
>>>>>> site: http://www.dbsight.net
>>>>>> demo: http://search.dbsight.com
>>>>>> Lucene Database Search in 3 minutes:
>>>>>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>>>>>> DBSight customer, a shopping comparison site, (anonymous per request) got
>>>>>> 2.6 Million Euro funding!
>>>>>>
>>>>>>
>>>>>> Noble Paul ??????? ?????? wrote:
>>>>>>>
>>>>>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --Noble Paul
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>>
>> --
>> --Noble Paul
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Yes, I believe so (we have a unit test asserting this).
But, there's no guarantee of cross-version compatibility of the serialized form.
Mike
2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
> that is great.
>
> Is it possible to serialize and deserialize a RAMDirectory?
>
> --Noble
>
>
>
>
> On Mon, Apr 13, 2009 at 3:29 PM, Michael McCandless
> <lu...@mikemccandless.com> wrote:
>> Lucene's ConcurrentMergeScheduler spawns threads, but you can switch
>> that to SerialMergeScheduler which does not use threads.
>>
>> Some optional classes (FilterManager, ParallelMultiSearcher,
>> TimeLimitedCollector) spawn threads, but presumably you could avoid
>> them.
>>
>> Otherwise Lucene doesn't create its own threads.
>>
>> Mike
>>
>> On Mon, Apr 13, 2009 at 5:20 AM, Erdinc Yilmazel
>> <er...@gmail.com> wrote:
>>> I looked at the lucene source code out of curiosity to see if it
>>> spawns threads in indexing/searching and saw that it does. Are you
>>> sure it is going to work even if the index is read only?
>>>
>>> Erdinc
>>>
>>> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>>>> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>>>>> Surely it's possible, but it has too much limitations to prevent a scalable
>>>>> Luceen usage. However, it depends on your requirement.
>>>>>
>>>>> 1) You can not write index on disk, but you can read files. So theoretically
>>>>> if the index is read-only and small, you can package it with the war file.
>>>> I can probably write the whole index as Blob and read it and serve
>>>> the index from RAM. This index can be loaded once in the start time
>>>> and i can setup a cron task to see if it got updated
>>>>>
>>>>> 2) If you need to update the index, you will have to store the index with
>>>>> Google's data store, just like store an index into databases. Sure it'll
>>>>> work. But performance would suffer because of transferring the whole index
>>>>> into memory, then really start searching. On the other hand, this could be a
>>>>> good solution for small index with per-user data.
>>>> as you said this is a solution if the index size is small
>>>>
>>>>>
>>>>> 3) For large changing indexes, you need to find other solutions to maintain
>>>>> lucene index.
>>>>>
>>>>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>>>>> changing the way to code.
>>>>
>>>> The point is ,does Lucens use threads internally which the sandbox disallows
>>>>
>>>>>
>>>>> --
>>>>> Chris Lu
>>>>> -------------------------
>>>>> Instant Scalable Full-Text Search On Any Database/Application
>>>>> site: http://www.dbsight.net
>>>>> demo: http://search.dbsight.com
>>>>> Lucene Database Search in 3 minutes:
>>>>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>>>>> DBSight customer, a shopping comparison site, (anonymous per request) got
>>>>> 2.6 Million Euro funding!
>>>>>
>>>>>
>>>>> Noble Paul ??????? ?????? wrote:
>>>>>>
>>>>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> --Noble Paul
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
>
> --
> --Noble Paul
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
that is great.
Is it possible to serialize and deserialize a RAMDirectory?
--Noble
On Mon, Apr 13, 2009 at 3:29 PM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> Lucene's ConcurrentMergeScheduler spawns threads, but you can switch
> that to SerialMergeScheduler which does not use threads.
>
> Some optional classes (FilterManager, ParallelMultiSearcher,
> TimeLimitedCollector) spawn threads, but presumably you could avoid
> them.
>
> Otherwise Lucene doesn't create its own threads.
>
> Mike
>
> On Mon, Apr 13, 2009 at 5:20 AM, Erdinc Yilmazel
> <er...@gmail.com> wrote:
>> I looked at the lucene source code out of curiosity to see if it
>> spawns threads in indexing/searching and saw that it does. Are you
>> sure it is going to work even if the index is read only?
>>
>> Erdinc
>>
>> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>>> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>>>> Surely it's possible, but it has too much limitations to prevent a scalable
>>>> Luceen usage. However, it depends on your requirement.
>>>>
>>>> 1) You can not write index on disk, but you can read files. So theoretically
>>>> if the index is read-only and small, you can package it with the war file.
>>> I can probably write the whole index as Blob and read it and serve
>>> the index from RAM. This index can be loaded once in the start time
>>> and i can setup a cron task to see if it got updated
>>>>
>>>> 2) If you need to update the index, you will have to store the index with
>>>> Google's data store, just like store an index into databases. Sure it'll
>>>> work. But performance would suffer because of transferring the whole index
>>>> into memory, then really start searching. On the other hand, this could be a
>>>> good solution for small index with per-user data.
>>> as you said this is a solution if the index size is small
>>>
>>>>
>>>> 3) For large changing indexes, you need to find other solutions to maintain
>>>> lucene index.
>>>>
>>>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>>>> changing the way to code.
>>>
>>> The point is ,does Lucens use threads internally which the sandbox disallows
>>>
>>>>
>>>> --
>>>> Chris Lu
>>>> -------------------------
>>>> Instant Scalable Full-Text Search On Any Database/Application
>>>> site: http://www.dbsight.net
>>>> demo: http://search.dbsight.com
>>>> Lucene Database Search in 3 minutes:
>>>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>>>> DBSight customer, a shopping comparison site, (anonymous per request) got
>>>> 2.6 Million Euro funding!
>>>>
>>>>
>>>> Noble Paul ??????? ?????? wrote:
>>>>>
>>>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Noble Paul
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
--Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Michael McCandless <lu...@mikemccandless.com>.
Lucene's ConcurrentMergeScheduler spawns threads, but you can switch
that to SerialMergeScheduler which does not use threads.
Some optional classes (FilterManager, ParallelMultiSearcher,
TimeLimitedCollector) spawn threads, but presumably you could avoid
them.
Otherwise Lucene doesn't create its own threads.
Mike
On Mon, Apr 13, 2009 at 5:20 AM, Erdinc Yilmazel
<er...@gmail.com> wrote:
> I looked at the lucene source code out of curiosity to see if it
> spawns threads in indexing/searching and saw that it does. Are you
> sure it is going to work even if the index is read only?
>
> Erdinc
>
> 2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
>> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>>> Surely it's possible, but it has too much limitations to prevent a scalable
>>> Luceen usage. However, it depends on your requirement.
>>>
>>> 1) You can not write index on disk, but you can read files. So theoretically
>>> if the index is read-only and small, you can package it with the war file.
>> I can probably write the whole index as Blob and read it and serve
>> the index from RAM. This index can be loaded once in the start time
>> and i can setup a cron task to see if it got updated
>>>
>>> 2) If you need to update the index, you will have to store the index with
>>> Google's data store, just like store an index into databases. Sure it'll
>>> work. But performance would suffer because of transferring the whole index
>>> into memory, then really start searching. On the other hand, this could be a
>>> good solution for small index with per-user data.
>> as you said this is a solution if the index size is small
>>
>>>
>>> 3) For large changing indexes, you need to find other solutions to maintain
>>> lucene index.
>>>
>>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>>> changing the way to code.
>>
>> The point is ,does Lucens use threads internally which the sandbox disallows
>>
>>>
>>> --
>>> Chris Lu
>>> -------------------------
>>> Instant Scalable Full-Text Search On Any Database/Application
>>> site: http://www.dbsight.net
>>> demo: http://search.dbsight.com
>>> Lucene Database Search in 3 minutes:
>>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>>> DBSight customer, a shopping comparison site, (anonymous per request) got
>>> 2.6 Million Euro funding!
>>>
>>>
>>> Noble Paul ??????? ?????? wrote:
>>>>
>>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>
>>>
>>
>>
>>
>> --
>> --Noble Paul
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Erdinc Yilmazel <er...@gmail.com>.
I looked at the lucene source code out of curiosity to see if it
spawns threads in indexing/searching and saw that it does. Are you
sure it is going to work even if the index is read only?
Erdinc
2009/4/13 Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>:
> On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
>> Surely it's possible, but it has too much limitations to prevent a scalable
>> Luceen usage. However, it depends on your requirement.
>>
>> 1) You can not write index on disk, but you can read files. So theoretically
>> if the index is read-only and small, you can package it with the war file.
> I can probably write the whole index as Blob and read it and serve
> the index from RAM. This index can be loaded once in the start time
> and i can setup a cron task to see if it got updated
>>
>> 2) If you need to update the index, you will have to store the index with
>> Google's data store, just like store an index into databases. Sure it'll
>> work. But performance would suffer because of transferring the whole index
>> into memory, then really start searching. On the other hand, this could be a
>> good solution for small index with per-user data.
> as you said this is a solution if the index size is small
>
>>
>> 3) For large changing indexes, you need to find other solutions to maintain
>> lucene index.
>>
>> My personal opinion is, finding a $20/month VPS hosting is far easier than
>> changing the way to code.
>
> The point is ,does Lucens use threads internally which the sandbox disallows
>
>>
>> --
>> Chris Lu
>> -------------------------
>> Instant Scalable Full-Text Search On Any Database/Application
>> site: http://www.dbsight.net
>> demo: http://search.dbsight.com
>> Lucene Database Search in 3 minutes:
>> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
>> DBSight customer, a shopping comparison site, (anonymous per request) got
>> 2.6 Million Euro funding!
>>
>>
>> Noble Paul ??????? ?????? wrote:
>>>
>>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
>
>
> --
> --Noble Paul
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Noble Paul നോബിള് नोब्ळ् <no...@gmail.com>.
On Mon, Apr 13, 2009 at 1:10 PM, Chris Lu <ch...@gmail.com> wrote:
> Surely it's possible, but it has too much limitations to prevent a scalable
> Luceen usage. However, it depends on your requirement.
>
> 1) You can not write index on disk, but you can read files. So theoretically
> if the index is read-only and small, you can package it with the war file.
I can probably write the whole index as Blob and read it and serve
the index from RAM. This index can be loaded once in the start time
and i can setup a cron task to see if it got updated
>
> 2) If you need to update the index, you will have to store the index with
> Google's data store, just like store an index into databases. Sure it'll
> work. But performance would suffer because of transferring the whole index
> into memory, then really start searching. On the other hand, this could be a
> good solution for small index with per-user data.
as you said this is a solution if the index size is small
>
> 3) For large changing indexes, you need to find other solutions to maintain
> lucene index.
>
> My personal opinion is, finding a $20/month VPS hosting is far easier than
> changing the way to code.
The point is ,does Lucens use threads internally which the sandbox disallows
>
> --
> Chris Lu
> -------------------------
> Instant Scalable Full-Text Search On Any Database/Application
> site: http://www.dbsight.net
> demo: http://search.dbsight.com
> Lucene Database Search in 3 minutes:
> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
> DBSight customer, a shopping comparison site, (anonymous per request) got
> 2.6 Million Euro funding!
>
>
> Noble Paul ??????? ?????? wrote:
>>
>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
--Noble Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Glen Newton <gl...@gmail.com>.
Another solution is to have your application on the AppEngine, but the
index is on another machine. Then the application 'proxies' the
requests to the machine that has the index, which is using Solr
[http://lucene.apache.org/solr/] or some other way to expose to the
index to the web.
Yes, this means you still need hosting for the index + Solr.
-glen
http://zzzoot.blogspot.com/
2009/4/13 Chris Lu <ch...@gmail.com>:
> Surely it's possible, but it has too much limitations to prevent a scalable
> Luceen usage. However, it depends on your requirement.
>
> 1) You can not write index on disk, but you can read files. So theoretically
> if the index is read-only and small, you can package it with the war file.
>
> 2) If you need to update the index, you will have to store the index with
> Google's data store, just like store an index into databases. Sure it'll
> work. But performance would suffer because of transferring the whole index
> into memory, then really start searching. On the other hand, this could be a
> good solution for small index with per-user data.
>
> 3) For large changing indexes, you need to find other solutions to maintain
> lucene index.
>
> My personal opinion is, finding a $20/month VPS hosting is far easier than
> changing the way to code.
>
> --
> Chris Lu
> -------------------------
> Instant Scalable Full-Text Search On Any Database/Application
> site: http://www.dbsight.net
> demo: http://search.dbsight.com
> Lucene Database Search in 3 minutes:
> http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
> DBSight customer, a shopping comparison site, (anonymous per request) got
> 2.6 Million Euro funding!
>
>
> Noble Paul ??????? ?????? wrote:
>>
>> Is it possible to run Lucene in google app engine? has anyone tried it?
>>
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
-
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Can I run Lucene in google app engine?
Posted by Chris Lu <ch...@gmail.com>.
Surely it's possible, but it has too much limitations to prevent a
scalable Luceen usage. However, it depends on your requirement.
1) You can not write index on disk, but you can read files. So
theoretically if the index is read-only and small, you can package it
with the war file.
2) If you need to update the index, you will have to store the index
with Google's data store, just like store an index into databases. Sure
it'll work. But performance would suffer because of transferring the
whole index into memory, then really start searching. On the other hand,
this could be a good solution for small index with per-user data.
3) For large changing indexes, you need to find other solutions to
maintain lucene index.
My personal opinion is, finding a $20/month VPS hosting is far easier
than changing the way to code.
--
Chris Lu
-------------------------
Instant Scalable Full-Text Search On Any Database/Application
site: http://www.dbsight.net
demo: http://search.dbsight.com
Lucene Database Search in 3 minutes: http://wiki.dbsight.com/index.php?title=Create_Lucene_Database_Search_in_3_minutes
DBSight customer, a shopping comparison site, (anonymous per request) got 2.6 Million Euro funding!
Noble Paul ??????? ?????? wrote:
> Is it possible to run Lucene in google app engine? has anyone tried it?
>
>
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org