You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by "vidit.asthana" <vi...@gmail.com> on 2015/03/31 08:25:35 UTC
SolrJ commit with openSearcher=false
How can I issue a hard commit through SolrJ such that openSearcher=false?
Also how can I issue same request through http? Will this work -
curl
"http://localhost:8983/solr/collection1/update?commit=true&openSearcher=false"
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrJ-commit-with-openSearcher-false-tp4196499.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrJ commit with openSearcher=false
Posted by Erick Erickson <er...@gmail.com>.
Hmmm, you really shouldn't have to do this. What have you tried to
figure out why the strange node isn't getting cleaned up? Is there
anything in the Solr logs that might help?
Is it a Windows machine? Some of the delete semantics for Windows can
leave things around. What happens if you restart the server and
continue indexing, do some of the tlogs disappear (the hypothesis here
is that somehow the files are being held open).
Best,
Erick
On Tue, Mar 31, 2015 at 8:39 AM, Shawn Heisey <ap...@elyograg.org> wrote:
> On 3/31/2015 2:56 AM, vidit.asthana wrote:
>> Thanks for reply Shawn. I will try it out.
>>
>> The reason that I am forced to do a hard commit through code is to handle a
>> problem I am facing with transaction logs.
>>
>> I am forced to delete tlogs manually at regular interval and hence I want to
>> issue a hard commit before deleting them to ensure that no data loss happens
>> in case of node failure.
>>
>> I have explained the issue in detail in another thread -
>> http://lucene.472066.n3.nabble.com/Transaction-logs-not-getting-deleted-td4184635.html
>>
>> If you can provide me some help in finding the fix for the issue, then it
>> would be a huge help for me.
>
> The first thing I would try is to set up autoCommit with a maxTime of
> 300000 (five minutes) and openSearcher set to false, as shown in the
> comments of the the example solrconfig.xml, although that example may
> have a value of 15000 (15 seconds). If that doesn't bring your
> transaction logs under control, then you definitely are facing an
> unusual situation or a bug where old and outdated transaction logs are
> not being automatically deleted. If it does appear that you've got a
> bug, one of the first steps I would take is upgrading from 4.10.0 to
> 4.10.4 - it should be a drop-in replacement of your .war file and any
> contrib jars, and I would delete the extracted version of the war before
> restarting.
>
> Is your data directory on a network filesystem, like NFS or SMB? That
> can sometimes cause weird problems with Solr.
>
> Are you seeing any ERROR or WARN entries in your solr log?
>
> Thanks,
> Shawn
>
Re: SolrJ commit with openSearcher=false
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/31/2015 2:56 AM, vidit.asthana wrote:
> Thanks for reply Shawn. I will try it out.
>
> The reason that I am forced to do a hard commit through code is to handle a
> problem I am facing with transaction logs.
>
> I am forced to delete tlogs manually at regular interval and hence I want to
> issue a hard commit before deleting them to ensure that no data loss happens
> in case of node failure.
>
> I have explained the issue in detail in another thread -
> http://lucene.472066.n3.nabble.com/Transaction-logs-not-getting-deleted-td4184635.html
>
> If you can provide me some help in finding the fix for the issue, then it
> would be a huge help for me.
The first thing I would try is to set up autoCommit with a maxTime of
300000 (five minutes) and openSearcher set to false, as shown in the
comments of the the example solrconfig.xml, although that example may
have a value of 15000 (15 seconds). If that doesn't bring your
transaction logs under control, then you definitely are facing an
unusual situation or a bug where old and outdated transaction logs are
not being automatically deleted. If it does appear that you've got a
bug, one of the first steps I would take is upgrading from 4.10.0 to
4.10.4 - it should be a drop-in replacement of your .war file and any
contrib jars, and I would delete the extracted version of the war before
restarting.
Is your data directory on a network filesystem, like NFS or SMB? That
can sometimes cause weird problems with Solr.
Are you seeing any ERROR or WARN entries in your solr log?
Thanks,
Shawn
Re: SolrJ commit with openSearcher=false
Posted by "vidit.asthana" <vi...@gmail.com>.
Thanks for reply Shawn. I will try it out.
The reason that I am forced to do a hard commit through code is to handle a
problem I am facing with transaction logs.
I am forced to delete tlogs manually at regular interval and hence I want to
issue a hard commit before deleting them to ensure that no data loss happens
in case of node failure.
I have explained the issue in detail in another thread -
http://lucene.472066.n3.nabble.com/Transaction-logs-not-getting-deleted-td4184635.html
If you can provide me some help in finding the fix for the issue, then it
would be a huge help for me.
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrJ-commit-with-openSearcher-false-tp4196499p4196527.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrJ commit with openSearcher=false
Posted by Shawn Heisey <ap...@elyograg.org>.
On 3/31/2015 12:25 AM, vidit.asthana wrote:
> How can I issue a hard commit through SolrJ such that openSearcher=false?
>
> Also how can I issue same request through http? Will this work -
>
> curl
> "http://localhost:8983/solr/collection1/update?commit=true&openSearcher=false"
This SolrJ code should do a hard commit with openSearcher set to false,
assuming "server" is an instance of SolrServer (SolrClient in 5.0):
UpdateRequest req = new UpdateRequest();
req.setAction(UpdateRequest.ACTION.COMMIT, true, true);
req.setParam("openSearcher", "false");
UpdateResponse ur = req.process(server);
I *think* the curl command you have indicated would do the same thing,
but I haven't tried it.
Personally, I would just use autoCommit to handle the hard commits with
openSearcher=false, and do soft commits in my application to control
document visibility. You can make all commits automated by adding
autoSoftCommit to the config or using commitWithin on the udpate
requests, but I like to be completely in control of when new documents
become visible.
Thanks,
Shawn