You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Saumitra Srivastav <sa...@gmail.com> on 2014/05/15 14:27:19 UTC
Unload collection in SolrCloud
Is there a way to unload the complete collection in SolrCloud env? I can
achieve the same by unloading all shards of collection using core admin
API, but is there a better/cleaner approach?
-Saumitra
--
View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by didier deshommes <df...@gmail.com>.
I added a JIRA issue here: https://issues.apache.org/jira/browse/SOLR-6399
On Thu, May 22, 2014 at 4:16 PM, Erick Erickson <er...@gmail.com>
wrote:
> "Age out" in this context is just implementing a LRU cache for open
> cores. When the cache limit is exceeded, the oldest core is closed
> automatically.
>
> Best,
> Erick
>
> On Thu, May 22, 2014 at 10:27 AM, Saumitra Srivastav
> <sa...@gmail.com> wrote:
> > Eric,
> >
> > Can you elaborate more on what you mean by "age out"?
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137707.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Unload collection in SolrCloud
Posted by Erick Erickson <er...@gmail.com>.
"Age out" in this context is just implementing a LRU cache for open
cores. When the cache limit is exceeded, the oldest core is closed
automatically.
Best,
Erick
On Thu, May 22, 2014 at 10:27 AM, Saumitra Srivastav
<sa...@gmail.com> wrote:
> Eric,
>
> Can you elaborate more on what you mean by "age out"?
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137707.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by Saumitra Srivastav <sa...@gmail.com>.
Eric,
Can you elaborate more on what you mean by "age out"?
--
View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137707.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by Saumitra S <sa...@gmail.com>.
Eric,
Can you elaborate more on what you mean by "age out"?
On Thu, May 22, 2014 at 9:00 PM, Erick Erickson <er...@gmail.com>wrote:
> If we manage to extend the "lazy core" loading from stand-alone to
> "lazy collection" loading in SolrCloud would that satisfy the
> use-case? It still doesn't allow manual unloading of the collection,
> but the large collection would "age out" if it was truly not used all
> that much. That said, I don't know if there's active work in this
> direction right now.
>
> Best,
> Erick
>
> On Thu, May 22, 2014 at 5:35 AM, Saumitra Srivastav
> <sa...@gmail.com> wrote:
> > Yes, that's what I am doing.
> >
> > IMO in addition to search, Solr satisfies the needs of lot of analytics
> > applications as well, and on-demand loading is a common use case in
> > analytics(to keep TCO low), so it would be nice to keep this supported.
> >
> >
> > Regards,
> > Saumitra
> >
> >
> >
> > On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] <
> > ml-node+s472066n4137630h17@n3.nabble.com> wrote:
> >
> >> Ah, I see. So if I understand it correctly, you are sharing the cluster
> >> with other collections which are more frequently used and you want to
> keep
> >> resources available for them so you keep your collection dormant most of
> >> the time until requested.
> >>
> >> No, we don't have such an API. It'd be cool to have a lazy loaded
> >> collection though. Thank you for describing the use-case because the way
> >> that we're moving towards (ZK as truth etc.), the core admin APIs will
> >> gradually be phased out and satisfying your use-case would become
> >> impossible. Let me think more on this.
> >>
> >>
> >> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
> >> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>>
> >> wrote:
> >>
> >> > I don't want to delete the collection/shards. I just want to unload
> all
> >> > shards/replica of the collection temporarily.
> >> >
> >> > Let me explain my use case.
> >> >
> >> > I have a collection alias say *collectionA* which consists of n
> >> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine
> >> > cluster.
> >> > *collectionA* is quite big in size and used very rarely, so we keep
> all
> >> > shards/replica of *collectionA* unloaded most of the time. Only when
> >> user
> >> > request to use it, we load it in memory. To load/unload shards/replica
> >> of
> >> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
> >> > shards/replicas in aliased collection and then use CORE ADMIN api to
> >> > load/unload them.
> >> >
> >> > As you can see there is lot of manual work involved, so I want to know
> >> if
> >> > there is an API to load/unload ALL shards/replicas of a collection?
> >> >
> >> >
> >> > Regards,
> >> > Saumitra
> >> >
> >> >
> >> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
> >> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1
> >>
> >> wrote:
> >> >
> >> > > You can use the delete Collection API.
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
> >> > >
> >> > >
> >> > >
> >> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
> >> > > [hidden email] <
> http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
> >>
> >> > > wrote:
> >> > >
> >> > > > Guys, any suggestions for this??
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > View this message in context:
> >> > > >
> >> > >
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> >> > > > Sent from the Solr - User mailing list archive at Nabble.com.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Regards,
> >> > > Shalin Shekhar Mangar.
> >> > >
> >> > >
> >> > > ------------------------------
> >> > > If you reply to this email, your message will be added to the
> >> discussion
> >> > > below:
> >> > >
> >> > >
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
> >> > > To start a new topic under Solr - User, email
> >> > > [hidden email] <
> http://user/SendEmail.jtp?type=node&node=4137630&i=2>
> >> > > To unsubscribe from Unload collection in SolrCloud, click here<
> >> >
> >> >
> >> > > .
> >> > > NAML<
> >> >
> >>
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >>
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > View this message in context:
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
> >>
> >> > Sent from the Solr - User mailing list archive at Nabble.com.
> >> >
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >>
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html
> >> To start a new topic under Solr - User, email
> >> ml-node+s472066n472068h95@n3.nabble.com
> >> To unsubscribe from Unload collection in SolrCloud, click here<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=
> >
> >> .
> >> NAML<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >>
> >
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Unload collection in SolrCloud
Posted by didier deshommes <df...@gmail.com>.
On Thu, May 22, 2014 at 10:30 AM, Erick Erickson <er...@gmail.com>wrote:
> If we manage to extend the "lazy core" loading from stand-alone to
> "lazy collection" loading in SolrCloud would that satisfy the
> use-case? It still doesn't allow manual unloading of the collection,
> but the large collection would "age out" if it was truly not used all
> that much. That said, I don't know if there's active work in this
> direction right now.
>
>
This is a nice option to have but having the ability to manually load and
unload a collection is still needed. For example, if you're doing analytics
work and storing a day's data in a collection, you still want to be able to
access day 32 and day 64 even if you keep 30 days of data loaded in memory.
I also think having a manual option would allow people more flexibility in
how they manage the number of collections they keep loaded.
> Best,
> Erick
>
> On Thu, May 22, 2014 at 5:35 AM, Saumitra Srivastav
> <sa...@gmail.com> wrote:
> > Yes, that's what I am doing.
> >
> > IMO in addition to search, Solr satisfies the needs of lot of analytics
> > applications as well, and on-demand loading is a common use case in
> > analytics(to keep TCO low), so it would be nice to keep this supported.
> >
> >
> > Regards,
> > Saumitra
> >
> >
> >
> > On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] <
> > ml-node+s472066n4137630h17@n3.nabble.com> wrote:
> >
> >> Ah, I see. So if I understand it correctly, you are sharing the cluster
> >> with other collections which are more frequently used and you want to
> keep
> >> resources available for them so you keep your collection dormant most of
> >> the time until requested.
> >>
> >> No, we don't have such an API. It'd be cool to have a lazy loaded
> >> collection though. Thank you for describing the use-case because the way
> >> that we're moving towards (ZK as truth etc.), the core admin APIs will
> >> gradually be phased out and satisfying your use-case would become
> >> impossible. Let me think more on this.
> >>
> >>
> >> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
> >> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>>
> >> wrote:
> >>
> >> > I don't want to delete the collection/shards. I just want to unload
> all
> >> > shards/replica of the collection temporarily.
> >> >
> >> > Let me explain my use case.
> >> >
> >> > I have a collection alias say *collectionA* which consists of n
> >> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine
> >> > cluster.
> >> > *collectionA* is quite big in size and used very rarely, so we keep
> all
> >> > shards/replica of *collectionA* unloaded most of the time. Only when
> >> user
> >> > request to use it, we load it in memory. To load/unload shards/replica
> >> of
> >> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
> >> > shards/replicas in aliased collection and then use CORE ADMIN api to
> >> > load/unload them.
> >> >
> >> > As you can see there is lot of manual work involved, so I want to know
> >> if
> >> > there is an API to load/unload ALL shards/replicas of a collection?
> >> >
> >> >
> >> > Regards,
> >> > Saumitra
> >> >
> >> >
> >> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
> >> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1
> >>
> >> wrote:
> >> >
> >> > > You can use the delete Collection API.
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
> >> > >
> >> > >
> >> > >
> >> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
> >> > > [hidden email] <
> http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
> >>
> >> > > wrote:
> >> > >
> >> > > > Guys, any suggestions for this??
> >> > > >
> >> > > >
> >> > > >
> >> > > > --
> >> > > > View this message in context:
> >> > > >
> >> > >
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> >> > > > Sent from the Solr - User mailing list archive at Nabble.com.
> >> > > >
> >> > >
> >> > >
> >> > >
> >> > > --
> >> > > Regards,
> >> > > Shalin Shekhar Mangar.
> >> > >
> >> > >
> >> > > ------------------------------
> >> > > If you reply to this email, your message will be added to the
> >> discussion
> >> > > below:
> >> > >
> >> > >
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
> >> > > To start a new topic under Solr - User, email
> >> > > [hidden email] <
> http://user/SendEmail.jtp?type=node&node=4137630&i=2>
> >> > > To unsubscribe from Unload collection in SolrCloud, click here<
> >> >
> >> >
> >> > > .
> >> > > NAML<
> >> >
> >>
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >>
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > View this message in context:
> >> >
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
> >>
> >> > Sent from the Solr - User mailing list archive at Nabble.com.
> >> >
> >>
> >>
> >>
> >> --
> >> Regards,
> >> Shalin Shekhar Mangar.
> >>
> >>
> >> ------------------------------
> >> If you reply to this email, your message will be added to the
> discussion
> >> below:
> >>
> >>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html
> >> To start a new topic under Solr - User, email
> >> ml-node+s472066n472068h95@n3.nabble.com
> >> To unsubscribe from Unload collection in SolrCloud, click here<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=
> >
> >> .
> >> NAML<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >>
> >
> >
> >
> >
> > --
> > View this message in context:
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
>
Re: Unload collection in SolrCloud
Posted by Erick Erickson <er...@gmail.com>.
If we manage to extend the "lazy core" loading from stand-alone to
"lazy collection" loading in SolrCloud would that satisfy the
use-case? It still doesn't allow manual unloading of the collection,
but the large collection would "age out" if it was truly not used all
that much. That said, I don't know if there's active work in this
direction right now.
Best,
Erick
On Thu, May 22, 2014 at 5:35 AM, Saumitra Srivastav
<sa...@gmail.com> wrote:
> Yes, that's what I am doing.
>
> IMO in addition to search, Solr satisfies the needs of lot of analytics
> applications as well, and on-demand loading is a common use case in
> analytics(to keep TCO low), so it would be nice to keep this supported.
>
>
> Regards,
> Saumitra
>
>
>
> On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] <
> ml-node+s472066n4137630h17@n3.nabble.com> wrote:
>
>> Ah, I see. So if I understand it correctly, you are sharing the cluster
>> with other collections which are more frequently used and you want to keep
>> resources available for them so you keep your collection dormant most of
>> the time until requested.
>>
>> No, we don't have such an API. It'd be cool to have a lazy loaded
>> collection though. Thank you for describing the use-case because the way
>> that we're moving towards (ZK as truth etc.), the core admin APIs will
>> gradually be phased out and satisfying your use-case would become
>> impossible. Let me think more on this.
>>
>>
>> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
>> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>>
>> wrote:
>>
>> > I don't want to delete the collection/shards. I just want to unload all
>> > shards/replica of the collection temporarily.
>> >
>> > Let me explain my use case.
>> >
>> > I have a collection alias say *collectionA* which consists of n
>> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine
>> > cluster.
>> > *collectionA* is quite big in size and used very rarely, so we keep all
>> > shards/replica of *collectionA* unloaded most of the time. Only when
>> user
>> > request to use it, we load it in memory. To load/unload shards/replica
>> of
>> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
>> > shards/replicas in aliased collection and then use CORE ADMIN api to
>> > load/unload them.
>> >
>> > As you can see there is lot of manual work involved, so I want to know
>> if
>> > there is an API to load/unload ALL shards/replicas of a collection?
>> >
>> >
>> > Regards,
>> > Saumitra
>> >
>> >
>> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
>> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1>>
>> wrote:
>> >
>> > > You can use the delete Collection API.
>> > >
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
>> > >
>> > >
>> > >
>> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
>> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
>>
>> > > wrote:
>> > >
>> > > > Guys, any suggestions for this??
>> > > >
>> > > >
>> > > >
>> > > > --
>> > > > View this message in context:
>> > > >
>> > >
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
>> > > > Sent from the Solr - User mailing list archive at Nabble.com.
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Regards,
>> > > Shalin Shekhar Mangar.
>> > >
>> > >
>> > > ------------------------------
>> > > If you reply to this email, your message will be added to the
>> discussion
>> > > below:
>> > >
>> > >
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
>> > > To start a new topic under Solr - User, email
>> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=2>
>> > > To unsubscribe from Unload collection in SolrCloud, click here<
>> >
>> >
>> > > .
>> > > NAML<
>> >
>> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
>>
>> > >
>> > >
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> >
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
>>
>> > Sent from the Solr - User mailing list archive at Nabble.com.
>> >
>>
>>
>>
>> --
>> Regards,
>> Shalin Shekhar Mangar.
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the discussion
>> below:
>>
>> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html
>> To start a new topic under Solr - User, email
>> ml-node+s472066n472068h95@n3.nabble.com
>> To unsubscribe from Unload collection in SolrCloud, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=>
>> .
>> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by Saumitra Srivastav <sa...@gmail.com>.
Yes, that's what I am doing.
IMO in addition to search, Solr satisfies the needs of lot of analytics
applications as well, and on-demand loading is a common use case in
analytics(to keep TCO low), so it would be nice to keep this supported.
Regards,
Saumitra
On Thu, May 22, 2014 at 5:37 PM, Shalin Shekhar Mangar [via Lucene] <
ml-node+s472066n4137630h17@n3.nabble.com> wrote:
> Ah, I see. So if I understand it correctly, you are sharing the cluster
> with other collections which are more frequently used and you want to keep
> resources available for them so you keep your collection dormant most of
> the time until requested.
>
> No, we don't have such an API. It'd be cool to have a lazy loaded
> collection though. Thank you for describing the use-case because the way
> that we're moving towards (ZK as truth etc.), the core admin APIs will
> gradually be phased out and satisfying your use-case would become
> impossible. Let me think more on this.
>
>
> On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=0>>
> wrote:
>
> > I don't want to delete the collection/shards. I just want to unload all
> > shards/replica of the collection temporarily.
> >
> > Let me explain my use case.
> >
> > I have a collection alias say *collectionA* which consists of n
> > collections(n<=5) each with 8 shards and 2 replica over a 16 machine
> > cluster.
> > *collectionA* is quite big in size and used very rarely, so we keep all
> > shards/replica of *collectionA* unloaded most of the time. Only when
> user
> > request to use it, we load it in memory. To load/unload shards/replica
> of
> > aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
> > shards/replicas in aliased collection and then use CORE ADMIN api to
> > load/unload them.
> >
> > As you can see there is lot of manual work involved, so I want to know
> if
> > there is an API to load/unload ALL shards/replicas of a collection?
> >
> >
> > Regards,
> > Saumitra
> >
> >
> > On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=1>>
> wrote:
> >
> > > You can use the delete Collection API.
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
> > >
> > >
> > >
> > > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
>
> > > wrote:
> > >
> > > > Guys, any suggestions for this??
> > > >
> > > >
> > > >
> > > > --
> > > > View this message in context:
> > > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> > > > Sent from the Solr - User mailing list archive at Nabble.com.
> > > >
> > >
> > >
> > >
> > > --
> > > Regards,
> > > Shalin Shekhar Mangar.
> > >
> > >
> > > ------------------------------
> > > If you reply to this email, your message will be added to the
> discussion
> > > below:
> > >
> > >
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
> > > To start a new topic under Solr - User, email
> > > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137630&i=2>
> > > To unsubscribe from Unload collection in SolrCloud, click here<
> >
> >
> > > .
> > > NAML<
> >
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
>
> > >
> > >
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
>
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137630.html
> To start a new topic under Solr - User, email
> ml-node+s472066n472068h95@n3.nabble.com
> To unsubscribe from Unload collection in SolrCloud, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137639.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
Ah, I see. So if I understand it correctly, you are sharing the cluster
with other collections which are more frequently used and you want to keep
resources available for them so you keep your collection dormant most of
the time until requested.
No, we don't have such an API. It'd be cool to have a lazy loaded
collection though. Thank you for describing the use-case because the way
that we're moving towards (ZK as truth etc.), the core admin APIs will
gradually be phased out and satisfying your use-case would become
impossible. Let me think more on this.
On Thu, May 22, 2014 at 4:57 PM, Saumitra Srivastav <
saumitra.srivastav7@gmail.com> wrote:
> I don't want to delete the collection/shards. I just want to unload all
> shards/replica of the collection temporarily.
>
> Let me explain my use case.
>
> I have a collection alias say *collectionA* which consists of n
> collections(n<=5) each with 8 shards and 2 replica over a 16 machine
> cluster.
> *collectionA* is quite big in size and used very rarely, so we keep all
> shards/replica of *collectionA* unloaded most of the time. Only when user
> request to use it, we load it in memory. To load/unload shards/replica of
> aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
> shards/replicas in aliased collection and then use CORE ADMIN api to
> load/unload them.
>
> As you can see there is lot of manual work involved, so I want to know if
> there is an API to load/unload ALL shards/replicas of a collection?
>
>
> Regards,
> Saumitra
>
>
> On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
> ml-node+s472066n4137608h35@n3.nabble.com> wrote:
>
> > You can use the delete Collection API.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
> >
> >
> >
> > On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
> > [hidden email] <http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
> > wrote:
> >
> > > Guys, any suggestions for this??
> > >
> > >
> > >
> > > --
> > > View this message in context:
> > >
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> > > Sent from the Solr - User mailing list archive at Nabble.com.
> > >
> >
> >
> >
> > --
> > Regards,
> > Shalin Shekhar Mangar.
> >
> >
> > ------------------------------
> > If you reply to this email, your message will be added to the discussion
> > below:
> >
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
> > To start a new topic under Solr - User, email
> > ml-node+s472066n472068h95@n3.nabble.com
> > To unsubscribe from Unload collection in SolrCloud, click here<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=
> >
> > .
> > NAML<
> http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml
> >
> >
>
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards,
Shalin Shekhar Mangar.
Re: Unload collection in SolrCloud
Posted by Saumitra Srivastav <sa...@gmail.com>.
I don't want to delete the collection/shards. I just want to unload all
shards/replica of the collection temporarily.
Let me explain my use case.
I have a collection alias say *collectionA* which consists of n
collections(n<=5) each with 8 shards and 2 replica over a 16 machine
cluster.
*collectionA* is quite big in size and used very rarely, so we keep all
shards/replica of *collectionA* unloaded most of the time. Only when user
request to use it, we load it in memory. To load/unload shards/replica of
aliased *collectionA*, we use CLUSTERSTATUS api to get list of all
shards/replicas in aliased collection and then use CORE ADMIN api to
load/unload them.
As you can see there is lot of manual work involved, so I want to know if
there is an API to load/unload ALL shards/replicas of a collection?
Regards,
Saumitra
On Thu, May 22, 2014 at 4:36 PM, Shalin Shekhar Mangar [via Lucene] <
ml-node+s472066n4137608h35@n3.nabble.com> wrote:
> You can use the delete Collection API.
>
>
> https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
>
>
>
> On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
> [hidden email] <http://user/SendEmail.jtp?type=node&node=4137608&i=0>>
> wrote:
>
> > Guys, any suggestions for this??
> >
> >
> >
> > --
> > View this message in context:
> >
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> > Sent from the Solr - User mailing list archive at Nabble.com.
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>
>
> ------------------------------
> If you reply to this email, your message will be added to the discussion
> below:
>
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137608.html
> To start a new topic under Solr - User, email
> ml-node+s472066n472068h95@n3.nabble.com
> To unsubscribe from Unload collection in SolrCloud, click here<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=4135706&code=c2F1bWl0cmEuc3JpdmFzdGF2N0BnbWFpbC5jb218NDEzNTcwNnw4NDI1NzU5NTM=>
> .
> NAML<http://lucene.472066.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>
--
View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137612.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Unload collection in SolrCloud
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
You can use the delete Collection API.
https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-api6
On Thu, May 22, 2014 at 3:56 PM, Saumitra Srivastav <
saumitra.srivastav7@gmail.com> wrote:
> Guys, any suggestions for this??
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
--
Regards,
Shalin Shekhar Mangar.
Re: Unload collection in SolrCloud
Posted by Saumitra Srivastav <sa...@gmail.com>.
Guys, any suggestions for this??
--
View this message in context: http://lucene.472066.n3.nabble.com/Unload-collection-in-SolrCloud-tp4135706p4137602.html
Sent from the Solr - User mailing list archive at Nabble.com.