You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Cheng <zh...@gmail.com> on 2012/01/11 02:32:43 UTC
shared instance of IndexWriter doesn't improve proformance
Hi,
I use a same instance of writer for multiple threads. It turns out that the
time to finish jobs is more than to create a new writer instance in each
thread. What would be the possible reasons?
Thanks
Re: Unsubscribe failure
Posted by Erick Erickson <er...@gmail.com>.
If your e-mail client sends things in anything but plain text, you might try
switching the format to plain text. I've had the spam filter
reject formatted e-mail before...
May not be relevant, but it's worth a try.
Best
Erick
On Wed, Jan 11, 2012 at 12:44 PM, Bennett, Tony
<Be...@con-way.com> wrote:
> I tried to unsubscribe from this list, without success.
>
> I sent an email to 'java-user-unsubscribe@lucene.apache.org',
> I received the "please confirm" response, requesting that I send
> an email to: java-user-uc.1326295748.hcdpoljefehgobokinbd-Bennett.Tony=con-way.com@lucene.apache.org
>
> I did so, and our company's Email System reported it was rejected...!!!
> Here's the contents of our company's "return mail" message:
>
> This is the mail system at host idcxpr0408.con-way.com.
>
> I'm sorry to have to inform you that your message could not
> be delivered to one or more recipients. It's attached below.
>
> For further assistance, please send mail to postmaster.
>
> If you do so, please include this problem report. You can
> delete your own text from the attached returned message.
>
> The mail system
>
> <ja...@lucene.apache.org>:
> host mx1.eu.apache.org[192.87.106.230] said: 552 spam score (5.5) exceeded
> threshold
> (DCC_CHECK,HTML_MESSAGE,MIME_HTML_MOSTLY,RCVD_IN_DNSWL_LOW,SPF_PASS (in
> reply to end of DATA command)
>
> Any suggestions for what I do next would be greatly appreciated.
>
> Thanks,
> -tony
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Unsubscribe failure
Posted by Chris Hostetter <ho...@fucit.org>.
If anyone is having problems unsubscribing from, or subscribing to, any
apache mailing list, please contact the list modererators by adding
"-owner" to the name of the list, ie...
java-user-owner@lucene.apache.org
...and avoid emailing the entire list with messages that 99.99% of the
subscribers won't be able to help you with.
Tony: If Erick's suggesting didn't help you unsubscribe, and you are still
seeing these emails, please reply to java-user-owner@lucene and we'll see
what we can do (but a human moderator will need to verify your email with
you first)
: I tried to unsubscribe from this list, without success.
:
: I sent an email to 'java-user-unsubscribe@lucene.apache.org',
: I received the "please confirm" response, requesting that I send
: an email to: java-user-uc.1326295748.hcdpoljefehgobokinbd-Bennett.Tony=con-way.com@lucene.apache.org
:
: I did so, and our company's Email System reported it was rejected...!!!
: Here's the contents of our company's "return mail" message:
:
: This is the mail system at host idcxpr0408.con-way.com.
:
: I'm sorry to have to inform you that your message could not
: be delivered to one or more recipients. It's attached below.
:
: For further assistance, please send mail to postmaster.
:
: If you do so, please include this problem report. You can
: delete your own text from the attached returned message.
:
: The mail system
:
: <ja...@lucene.apache.org>:
: host mx1.eu.apache.org[192.87.106.230] said: 552 spam score (5.5) exceeded
: threshold
: (DCC_CHECK,HTML_MESSAGE,MIME_HTML_MOSTLY,RCVD_IN_DNSWL_LOW,SPF_PASS (in
: reply to end of DATA command)
:
: Any suggestions for what I do next would be greatly appreciated.
:
: Thanks,
: -tony
:
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
: For additional commands, e-mail: java-user-help@lucene.apache.org
:
:
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Unsubscribe failure
Posted by "Bennett, Tony" <Be...@con-way.com>.
I tried to unsubscribe from this list, without success.
I sent an email to 'java-user-unsubscribe@lucene.apache.org',
I received the "please confirm" response, requesting that I send
an email to: java-user-uc.1326295748.hcdpoljefehgobokinbd-Bennett.Tony=con-way.com@lucene.apache.org
I did so, and our company's Email System reported it was rejected...!!!
Here's the contents of our company's "return mail" message:
This is the mail system at host idcxpr0408.con-way.com.
I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.
For further assistance, please send mail to postmaster.
If you do so, please include this problem report. You can
delete your own text from the attached returned message.
The mail system
<ja...@lucene.apache.org>:
host mx1.eu.apache.org[192.87.106.230] said: 552 spam score (5.5) exceeded
threshold
(DCC_CHECK,HTML_MESSAGE,MIME_HTML_MOSTLY,RCVD_IN_DNSWL_LOW,SPF_PASS (in
reply to end of DATA command)
Any suggestions for what I do next would be greatly appreciated.
Thanks,
-tony
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: shared instance of IndexWriter doesn't improve proformance
Posted by Michael McCandless <lu...@mikemccandless.com>.
I think it's hard to compare the results here?
In test 1 (single IW shared across threads) you end up with one index.
In test 2 (private IW per thread) you end up with N indexes, which to
be "fair" need to be merged down into one index (eg with .addIndexes)?
Or seen another way, test 1 should have a much larger merge factor to
be comparable to test 2.
You should also call IW.close(false) (aborts all running merges)
whenever comparing two indexing runs... otherwise a long running merge
can completely skew the results.
However, there is known contention in IndexWriter's flushing (when it
writes the buffered documents to a new segment), which is still
present in 3.x but is fixed on trunk (to be 4.0)... see this post:
http://blog.mikemccandless.com/2011/05/265-indexing-speedup-with-lucenes.html
Mike McCandless
http://blog.mikemccandless.com
On Tue, Jan 10, 2012 at 8:32 PM, Cheng <zh...@gmail.com> wrote:
> Hi,
>
> I use a same instance of writer for multiple threads. It turns out that the
> time to finish jobs is more than to create a new writer instance in each
> thread. What would be the possible reasons?
>
> Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: shared instance of IndexWriter doesn't improve proformance
Posted by Ian Lea <ia...@gmail.com>.
Contention. There is always a limit somewhere, I/O, CPU, memory, locks, ...
Use your OS tools or java profiling/logging/debugging to find out what
is going on - or just go with what works for you.
If you're doing something like loading data read from a database, it
is my experience that the bottleneck often lies there rather than in
lucene.
--
Ian.
On Wed, Jan 11, 2012 at 1:32 AM, Cheng <zh...@gmail.com> wrote:
> Hi,
>
> I use a same instance of writer for multiple threads. It turns out that the
> time to finish jobs is more than to create a new writer instance in each
> thread. What would be the possible reasons?
>
> Thanks
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org