Maintaining variable values between transition states

[Shashank Pedamallu <sp...@vmware.com>]

Hi All,

I have some configuration variables that I need to hold in Solr as it switches between transient states on a transient core. What is the best way to do this? These variables can change value during a running environment. So, I need to have read and write access to the persistent store. Some options I was thinking was store them in core.properties file. But then, I did not find a way to write to coreProperties. Similarly for handler parameters in solrconfig.xml. Please let me know if there are any recommended ways to handle this case.

Thanks,
Shashank

Re: Maintaining variable values between transition states

[Shawn Heisey <ap...@elyograg.org>]

On 3/30/2017 12:34 PM, Shashank Pedamallu wrote:
> I have some configuration variables that I need to hold in Solr as it
> switches between transient states on a transient core. What is the
> best way to do this? These variables can change value during a running
> environment. So, I need to have read and write access to the
> persistent store. Some options I was thinking was store them in
> core.properties file. But then, I did not find a way to write to
> coreProperties. Similarly for handler parameters in solrconfig.xml.
> Please let me know if there are any recommended ways to handle this case.

The way I would do this is by storing information in the Solr index. 
You could put a special document in each core, but I think I would set
up a dedicated core with a special schema to contain such information
for all the other cores.  The schema would have a field like "id" for a
uniqueKey that would have the name of the core in it, and the other
fields would each hold one piece of information (property) associated
with that core.  You would update that document before finishing the
work on that core, then you can later retrieve it with a query.  Atomic
Updates would work well with that core.

As far as your specific question about built in states like
enabling/disabling replication, and what the state will be when the core
is unloaded and then re-loaded, I do not know.

Thanks,
Shawn

Re: Maintaining variable values between transition states

[Shashank Pedamallu <sp...@vmware.com>]

Hi Erick,

Thanks for your response. So, by the way you say it, I understand that there is no way to persist variables between transient states.

I just looked at ReplicationHandler class and it has the api to enable or disable replication on a core which is stored as an AtomicBoolean (defaulted to True). In such case, what happens in following case
1. This core happens to have the property transient=true
2. I trigger a disable replication on this core using the api: http://master_host:port/solr/core_name/replication?command=disablereplication
3. Solr decides to unload this core because it had to load another
4. This core loads again after some time. What will be the status of enableReplication variable now? I’m wondering if it will it be true because it could not retain the value from previous api call?

Once again, thank you very much for your prompt responses and replies. Really appreciate them :)

Thanks,
Shashank






On 3/30/17, 12:29 PM, "Erick Erickson" <er...@gmail.com> wrote:

>Short form: There's no easy to do that ATM. The whole synchronization
>process when working with transient cores (i.e. synchronizing on some
>of the internal structures is pretty hairy and would require you to
>fork a version of Solr to change.
>
>Much of this is being worked out in SOLR-8906 where you can have your
>own transient cache plugin. There are some edge cases that need to be
>taken care of in SOLR-10008 and SOLR-10009, particularly the
>CoreDescriptor (which is the in-memory form of core.properties) has
>some issues with multiple copies being kept around so directly writing
>core.properties has some gotchas currently that should be fixed in the
>above.
>
>Best,
>Erick
>
>
>On Thu, Mar 30, 2017 at 11:34 AM, Shashank Pedamallu
><sp...@vmware.com> wrote:
>> Hi All,
>>
>> I have some configuration variables that I need to hold in Solr as it switches between transient states on a transient core. What is the best way to do this? These variables can change value during a running environment. So, I need to have read and write access to the persistent store. Some options I was thinking was store them in core.properties file. But then, I did not find a way to write to coreProperties. Similarly for handler parameters in solrconfig.xml. Please let me know if there are any recommended ways to handle this case.
>>
>> Thanks,
>> Shashank

Re: Maintaining variable values between transition states

[Erick Erickson <er...@gmail.com>]

Short form: There's no easy to do that ATM. The whole synchronization
process when working with transient cores (i.e. synchronizing on some
of the internal structures is pretty hairy and would require you to
fork a version of Solr to change.

Much of this is being worked out in SOLR-8906 where you can have your
own transient cache plugin. There are some edge cases that need to be
taken care of in SOLR-10008 and SOLR-10009, particularly the
CoreDescriptor (which is the in-memory form of core.properties) has
some issues with multiple copies being kept around so directly writing
core.properties has some gotchas currently that should be fixed in the
above.

Best,
Erick


On Thu, Mar 30, 2017 at 11:34 AM, Shashank Pedamallu
<sp...@vmware.com> wrote:
> Hi All,
>
> I have some configuration variables that I need to hold in Solr as it switches between transient states on a transient core. What is the best way to do this? These variables can change value during a running environment. So, I need to have read and write access to the persistent store. Some options I was thinking was store them in core.properties file. But then, I did not find a way to write to coreProperties. Similarly for handler parameters in solrconfig.xml. Please let me know if there are any recommended ways to handle this case.
>
> Thanks,
> Shashank