You are viewing a plain text version of this content. The canonical link for it is here.
Posted to users@solr.apache.org by Karl Stoney <Ka...@autotrader.co.uk.INVALID> on 2021/03/17 18:00:41 UTC
Disable commits during a REINDEXCOLLECTION
Hi all,
We're wanting to use REINDEXCOLLECTION, but our config has a relatively aggressive autoCommit interval configured by default (intentionally).
Ideally I'd like to be able to disable hard commits for the duration of the reindex, but can't see a way to do that without pushing a whole new config and reloading that collection.
Does anyone know of a ninja way at runtime to disable autoCommits on a collection (solr cloud)?
Thanks
Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Disable commits during a REINDEXCOLLECTION
Posted by Joel Bernstein <jo...@gmail.com>.
Commits need to happen or you'll get a memory leak. There are parts of the
transaction log which are held in memory for real-time get. Commits release
this memory as indexing occurs. I'm not aware of way to disable this
behavior.
Joel Bernstein
http://joelsolr.blogspot.com/
On Wed, Mar 17, 2021 at 6:47 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
> Bit of a hacky workaround.... but:
>
>
> * create a collection from the target config name before reindex
> * set property on that (which writes overlay to zookeeper)
> * delete the collection
> * run reindex
> * unset-property on the created collection
>
> ________________________________
> From: Karl Stoney <Ka...@autotrader.co.uk>
> Sent: 17 March 2021 22:20
> To: users@solr.apache.org <us...@solr.apache.org>
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> That's the source collection, I'm referring to the destination.
>
> @David - seeing as REINDEX creates the target collection, I had to start
> the process and then attempt to set the config on the destination.
> However setting the destination config kills the REINDEX:
>
> ```
> 22:18:40.389
> [DaemonStream-karl-test-20015-thread-1-processing-n:solr-0.search-solr.svc.cluster.local:80_solr
> x:at-uk-002_shard1_replica_n1 c:at-uk-002 s:shard1 r:core_node2] ERROR
> org.apache.solr.client.solrj.io.stream.DaemonStream - Err
> or in DaemonStream: karl-test
> java.io.IOException: org.apache.solr.common.SolrException: Could not find
> a healthy node to handle the request.
> at
> org.apache.solr.client.solrj.io.stream.TopicStream.persistCheckpoints(TopicStream.java:472)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> at
> org.apache.solr.client.solrj.io.stream.TopicStream.close(TopicStream.java:342)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> at
> org.apache.solr.client.solrj.io.stream.PushBackStream.close(PushBackStream.java:75)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> at
> org.apache.solr.client.solrj.io.stream.UpdateStream.close(UpdateStream.java:147)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> at
> org.apache.solr.client.solrj.io.stream.CommitStream.close(CommitStream.java:155)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> at
> org.apache.solr.client.solrj.io.stream.DaemonStream$StreamRunner.stream(DaemonStream.java:380)
> ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp
> - 2021-02-16 15:52:04]
> ```
>
> If i remove the set-config; it's fine.
> ________________________________
> From: Vivaldi <uy...@vivaldi.net.INVALID>
> Sent: 17 March 2021 21:59
> To: users@solr.apache.org <us...@solr.apache.org>
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> I thought the REINDEXCOLLECTION puts the source collection in read-only
> mode, am I wrong? Doesn’t that alsı disable commits?
>
> Sent from my iPhone
>
> > On 18 Mar 2021, at 00:28, Karl Stoney <Ka...@autotrader.co.uk.invalid>
> wrote:
> >
> > Can you believe I had never come across that before! Thanks!
> >
> > Every day’s a school day
> >
> > Get Outlook for iOS<
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=hAgyg3QJVOFoukpXl6TlulQHhtOYASlxhiUvnY5O1Ac%3D&reserved=0
> >
> > ________________________________
> > From: David Smiley <ds...@apache.org>
> > Sent: Wednesday, March 17, 2021 8:48:23 PM
> > To: users@solr.apache.org <us...@solr.apache.org>
> > Subject: Re: Disable commits during a REINDEXCOLLECTION
> >
> > Hi Karl,
> >
> > Look into the "config apI". Let us know how it goes!
> > Ideally, this feature would do this automatically.
> >
> > ~ David Smiley
> > Apache Lucene/Solr Search Developer
> >
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=7fZ17ijzSk3rs34K01tJ4SjP4kV4XU6Md8yzpcnu1jw%3D&reserved=0
> >
> >
> > On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
> > <Ka...@autotrader.co.uk.invalid> wrote:
> >
> >> Hi all,
> >> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> >> aggressive autoCommit interval configured by default (intentionally).
> >>
> >> Ideally I'd like to be able to disable hard commits for the duration of
> >> the reindex, but can't see a way to do that without pushing a whole new
> >> config and reloading that collection.
> >>
> >> Does anyone know of a ninja way at runtime to disable autoCommits on a
> >> collection (solr cloud)?
> >>
> >> Thanks
> >> Unless expressly stated otherwise in this email, this e-mail is sent on
> >> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> >> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628).
> Auto
> >> Trader Limited is part of the Auto Trader Group Plc group. This email
> and
> >> any files transmitted with it are confidential and may be legally
> >> privileged, and intended solely for the use of the individual or entity
> to
> >> whom they are addressed. If you have received this email in error please
> >> notify the sender. This email message has been swept for the presence of
> >> computer viruses.
> >>
> > Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>
> Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>
Re: Disable commits during a REINDEXCOLLECTION
Posted by Karl Stoney <Ka...@autotrader.co.uk.INVALID>.
Bit of a hacky workaround.... but:
* create a collection from the target config name before reindex
* set property on that (which writes overlay to zookeeper)
* delete the collection
* run reindex
* unset-property on the created collection
________________________________
From: Karl Stoney <Ka...@autotrader.co.uk>
Sent: 17 March 2021 22:20
To: users@solr.apache.org <us...@solr.apache.org>
Subject: Re: Disable commits during a REINDEXCOLLECTION
That's the source collection, I'm referring to the destination.
@David - seeing as REINDEX creates the target collection, I had to start the process and then attempt to set the config on the destination.
However setting the destination config kills the REINDEX:
```
22:18:40.389 [DaemonStream-karl-test-20015-thread-1-processing-n:solr-0.search-solr.svc.cluster.local:80_solr x:at-uk-002_shard1_replica_n1 c:at-uk-002 s:shard1 r:core_node2] ERROR org.apache.solr.client.solrj.io.stream.DaemonStream - Err
or in DaemonStream: karl-test
java.io.IOException: org.apache.solr.common.SolrException: Could not find a healthy node to handle the request.
at org.apache.solr.client.solrj.io.stream.TopicStream.persistCheckpoints(TopicStream.java:472) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.TopicStream.close(TopicStream.java:342) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.PushBackStream.close(PushBackStream.java:75) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.UpdateStream.close(UpdateStream.java:147) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.CommitStream.close(CommitStream.java:155) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.DaemonStream$StreamRunner.stream(DaemonStream.java:380) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
```
If i remove the set-config; it's fine.
________________________________
From: Vivaldi <uy...@vivaldi.net.INVALID>
Sent: 17 March 2021 21:59
To: users@solr.apache.org <us...@solr.apache.org>
Subject: Re: Disable commits during a REINDEXCOLLECTION
I thought the REINDEXCOLLECTION puts the source collection in read-only mode, am I wrong? Doesn’t that alsı disable commits?
Sent from my iPhone
> On 18 Mar 2021, at 00:28, Karl Stoney <Ka...@autotrader.co.uk.invalid> wrote:
>
> Can you believe I had never come across that before! Thanks!
>
> Every day’s a school day
>
> Get Outlook for iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=hAgyg3QJVOFoukpXl6TlulQHhtOYASlxhiUvnY5O1Ac%3D&reserved=0>
> ________________________________
> From: David Smiley <ds...@apache.org>
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org <us...@solr.apache.org>
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> Hi Karl,
>
> Look into the "config apI". Let us know how it goes!
> Ideally, this feature would do this automatically.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=7fZ17ijzSk3rs34K01tJ4SjP4kV4XU6Md8yzpcnu1jw%3D&reserved=0
>
>
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
> <Ka...@autotrader.co.uk.invalid> wrote:
>
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>>
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>>
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>>
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this email in error please
>> notify the sender. This email message has been swept for the presence of
>> computer viruses.
>>
> Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Disable commits during a REINDEXCOLLECTION
Posted by Karl Stoney <Ka...@autotrader.co.uk.INVALID>.
That's the source collection, I'm referring to the destination.
@David - seeing as REINDEX creates the target collection, I had to start the process and then attempt to set the config on the destination.
However setting the destination config kills the REINDEX:
```
22:18:40.389 [DaemonStream-karl-test-20015-thread-1-processing-n:solr-0.search-solr.svc.cluster.local:80_solr x:at-uk-002_shard1_replica_n1 c:at-uk-002 s:shard1 r:core_node2] ERROR org.apache.solr.client.solrj.io.stream.DaemonStream - Err
or in DaemonStream: karl-test
java.io.IOException: org.apache.solr.common.SolrException: Could not find a healthy node to handle the request.
at org.apache.solr.client.solrj.io.stream.TopicStream.persistCheckpoints(TopicStream.java:472) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.TopicStream.close(TopicStream.java:342) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.PushBackStream.close(PushBackStream.java:75) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.UpdateStream.close(UpdateStream.java:147) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.CommitStream.close(CommitStream.java:155) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
at org.apache.solr.client.solrj.io.stream.DaemonStream$StreamRunner.stream(DaemonStream.java:380) ~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp - 2021-02-16 15:52:04]
```
If i remove the set-config; it's fine.
________________________________
From: Vivaldi <uy...@vivaldi.net.INVALID>
Sent: 17 March 2021 21:59
To: users@solr.apache.org <us...@solr.apache.org>
Subject: Re: Disable commits during a REINDEXCOLLECTION
I thought the REINDEXCOLLECTION puts the source collection in read-only mode, am I wrong? Doesn’t that alsı disable commits?
Sent from my iPhone
> On 18 Mar 2021, at 00:28, Karl Stoney <Ka...@autotrader.co.uk.invalid> wrote:
>
> Can you believe I had never come across that before! Thanks!
>
> Every day’s a school day
>
> Get Outlook for iOS<https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fo0ukef&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=hAgyg3QJVOFoukpXl6TlulQHhtOYASlxhiUvnY5O1Ac%3D&reserved=0>
> ________________________________
> From: David Smiley <ds...@apache.org>
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org <us...@solr.apache.org>
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> Hi Karl,
>
> Look into the "config apI". Let us know how it goes!
> Ideally, this feature would do this automatically.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7Ce328a4cc735f49dc70a708d8e9900e49%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516152316874603%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=7fZ17ijzSk3rs34K01tJ4SjP4kV4XU6Md8yzpcnu1jw%3D&reserved=0
>
>
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
> <Ka...@autotrader.co.uk.invalid> wrote:
>
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>>
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>>
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>>
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this email in error please
>> notify the sender. This email message has been swept for the presence of
>> computer viruses.
>>
> Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Disable commits during a REINDEXCOLLECTION
Posted by Vivaldi <uy...@vivaldi.net.INVALID>.
I thought the REINDEXCOLLECTION puts the source collection in read-only mode, am I wrong? Doesn’t that alsı disable commits?
Sent from my iPhone
> On 18 Mar 2021, at 00:28, Karl Stoney <Ka...@autotrader.co.uk.invalid> wrote:
>
> Can you believe I had never come across that before! Thanks!
>
> Every day’s a school day
>
> Get Outlook for iOS<https://aka.ms/o0ukef>
> ________________________________
> From: David Smiley <ds...@apache.org>
> Sent: Wednesday, March 17, 2021 8:48:23 PM
> To: users@solr.apache.org <us...@solr.apache.org>
> Subject: Re: Disable commits during a REINDEXCOLLECTION
>
> Hi Karl,
>
> Look into the "config apI". Let us know how it goes!
> Ideally, this feature would do this automatically.
>
> ~ David Smiley
> Apache Lucene/Solr Search Developer
> https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7C3bd0a3557c6841f4a91008d8e9860bbe%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516109216986721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JcNE3AWytx2Ny5lyrH82EvS4hzI9eiCBImlZi0Az86M%3D&reserved=0
>
>
> On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
> <Ka...@autotrader.co.uk.invalid> wrote:
>
>> Hi all,
>> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
>> aggressive autoCommit interval configured by default (intentionally).
>>
>> Ideally I'd like to be able to disable hard commits for the duration of
>> the reindex, but can't see a way to do that without pushing a whole new
>> config and reloading that collection.
>>
>> Does anyone know of a ninja way at runtime to disable autoCommits on a
>> collection (solr cloud)?
>>
>> Thanks
>> Unless expressly stated otherwise in this email, this e-mail is sent on
>> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
>> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
>> Trader Limited is part of the Auto Trader Group Plc group. This email and
>> any files transmitted with it are confidential and may be legally
>> privileged, and intended solely for the use of the individual or entity to
>> whom they are addressed. If you have received this email in error please
>> notify the sender. This email message has been swept for the presence of
>> computer viruses.
>>
> Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Disable commits during a REINDEXCOLLECTION
Posted by Karl Stoney <Ka...@autotrader.co.uk.INVALID>.
Can you believe I had never come across that before! Thanks!
Every day’s a school day
Get Outlook for iOS<https://aka.ms/o0ukef>
________________________________
From: David Smiley <ds...@apache.org>
Sent: Wednesday, March 17, 2021 8:48:23 PM
To: users@solr.apache.org <us...@solr.apache.org>
Subject: Re: Disable commits during a REINDEXCOLLECTION
Hi Karl,
Look into the "config apI". Let us know how it goes!
Ideally, this feature would do this automatically.
~ David Smiley
Apache Lucene/Solr Search Developer
https://eur03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Fdavidwsmiley&data=04%7C01%7CKarl.Stoney%40autotrader.co.uk%7C3bd0a3557c6841f4a91008d8e9860bbe%7C926f3743f3d24b8a816818cfcbe776fe%7C0%7C0%7C637516109216986721%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=JcNE3AWytx2Ny5lyrH82EvS4hzI9eiCBImlZi0Az86M%3D&reserved=0
On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
> Hi all,
> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> aggressive autoCommit interval configured by default (intentionally).
>
> Ideally I'd like to be able to disable hard commits for the duration of
> the reindex, but can't see a way to do that without pushing a whole new
> config and reloading that collection.
>
> Does anyone know of a ninja way at runtime to disable autoCommits on a
> collection (solr cloud)?
>
> Thanks
> Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>
Unless expressly stated otherwise in this email, this e-mail is sent on behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited is part of the Auto Trader Group Plc group. This email and any files transmitted with it are confidential and may be legally privileged, and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This email message has been swept for the presence of computer viruses.
Re: Disable commits during a REINDEXCOLLECTION
Posted by David Smiley <ds...@apache.org>.
Hi Karl,
Look into the "config apI". Let us know how it goes!
Ideally, this feature would do this automatically.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Wed, Mar 17, 2021 at 2:02 PM Karl Stoney
<Ka...@autotrader.co.uk.invalid> wrote:
> Hi all,
> We're wanting to use REINDEXCOLLECTION, but our config has a relatively
> aggressive autoCommit interval configured by default (intentionally).
>
> Ideally I'd like to be able to disable hard commits for the duration of
> the reindex, but can't see a way to do that without pushing a whole new
> config and reloading that collection.
>
> Does anyone know of a ninja way at runtime to disable autoCommits on a
> collection (solr cloud)?
>
> Thanks
> Unless expressly stated otherwise in this email, this e-mail is sent on
> behalf of Auto Trader Limited Registered Office: 1 Tony Wilson Place,
> Manchester, Lancashire, M15 4FN (Registered in England No. 03909628). Auto
> Trader Limited is part of the Auto Trader Group Plc group. This email and
> any files transmitted with it are confidential and may be legally
> privileged, and intended solely for the use of the individual or entity to
> whom they are addressed. If you have received this email in error please
> notify the sender. This email message has been swept for the presence of
> computer viruses.
>