You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Ivan Rakov <iv...@gmail.com> on 2017/10/01 11:41:29 UTC
Re: Persistence per memory policy configuration
Denis,
1) You're right. I forgot to include the main flag in
DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
enabled globally if at least one memory region has this flag set.
Regarding default data region, I've added
*isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration.
Check the design draft again.
2) Of course, we have to maintain API compatibility. Deprecating old
classes and introducing new is just what I meant.
3) We can't do that - MemoryMetrics are calculated per memory policy and
PersistenceMetrics are calculated globally. It's a major change to
implement it another way.
By the way, let's assure the namings for new metrics classes.
DataRegionMetrics instead of MemoryMetrics looks fine.
About PersistenceMetrics - name "*DataStorageMetrics*" is not fair
enough as it will contain only metrics of persistent storage. Probably
*DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
keeping *PersistenceMetrics* are better.
What do you think?
Best Regards,
Ivan Rakov
On 29.09.2017 21:12, Denis Magda wrote:
> Ivan,
>
> Several questions/concerns:
>
> 1. Looking at the new API I can’t grasp how to enable the persistence. First, how can I enable it globally if there is only one default data region defined. Second, how do I enable it per data region. Can’t find any related switches in the draft.
>
> 2. We cannot renamed anything like you’re suggesting to do for MemoryMetrics and their beans. We have to deprecate old and introduce new.
>
> 3. I think we should merge Memory and Persistence Metrics into DataStorageMetrics for clarity.
>
> —
> Denis
>
>
>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Guys,
>>
>> I've attached new configuration design draft to the ticket description: https://issues.apache.org/jira/browse/IGNITE-6030
>> Please, take a look. Right now is the best time to change name of any property.
>>
>> And question about metrics: are we going to rename MemoryMetrics and PersistenceMetrics respectively (along with their MBeans)?
>> It's not a problem to implement it at all. The only thing that concerns me is that we have to keep deprecated old classes in the codebase. Perhaps, MemoryMetrics/PersistenceMetrics are intuitive enough.
>>
>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>> StorageRegion sounds like bad English to me.
>>>
>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>
>>> D.
>>>
>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
>>> wrote:
>>>
>>>> Guys,
>>>>
>>>> But what is exact desicion? :-) I saw two final options:
>>>>
>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>
>>>> Which one we choose?
>>>>
>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>> wrote:
>>>>
>>>>>> I guess it is safe to assume that at this point we came to a consensus?
>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>
>>>>> --Yakov
>>>>>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Vladimir,
*DataRegionMetrics* is for former memory metrics.
*DataStorageMetrics* will contain metrics about persistence.
If DataStorageMetrics is ok, let's go this way.
Best Regards,
Ivan Rakov
On 01.10.2017 15:19, Vladimir Ozerov wrote:
> We merged memory and persistence on config level. So we should merge
> metrics in the same way. DataRegionMetrics is absolutely normal name, even
> if it contains only persistence-related stuff at the moment.
>
> вс, 1 окт. 2017 г. в 14:41, Ivan Rakov <iv...@gmail.com>:
>
>> Denis,
>>
>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>> Regarding default data region, I've added
>> *isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration.
>> Check the design draft again.
>>
>> 2) Of course, we have to maintain API compatibility. Deprecating old
>> classes and introducing new is just what I meant.
>>
>> 3) We can't do that - MemoryMetrics are calculated per memory policy and
>> PersistenceMetrics are calculated globally. It's a major change to
>> implement it another way.
>>
>> By the way, let's assure the namings for new metrics classes.
>> DataRegionMetrics instead of MemoryMetrics looks fine.
>> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair
>> enough as it will contain only metrics of persistent storage. Probably
>> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
>> keeping *PersistenceMetrics* are better.
>>
>> What do you think?
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 29.09.2017 21:12, Denis Magda wrote:
>>> Ivan,
>>>
>>> Several questions/concerns:
>>>
>>> 1. Looking at the new API I can’t grasp how to enable the persistence.
>> First, how can I enable it globally if there is only one default data
>> region defined. Second, how do I enable it per data region. Can’t find any
>> related switches in the draft.
>>> 2. We cannot renamed anything like you’re suggesting to do for
>> MemoryMetrics and their beans. We have to deprecate old and introduce new.
>>> 3. I think we should merge Memory and Persistence Metrics into
>> DataStorageMetrics for clarity.
>>> —
>>> Denis
>>>
>>>
>>>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Guys,
>>>>
>>>> I've attached new configuration design draft to the ticket description:
>> https://issues.apache.org/jira/browse/IGNITE-6030
>>>> Please, take a look. Right now is the best time to change name of any
>> property.
>>>> And question about metrics: are we going to rename MemoryMetrics and
>> PersistenceMetrics respectively (along with their MBeans)?
>>>> It's not a problem to implement it at all. The only thing that concerns
>> me is that we have to keep deprecated old classes in the codebase. Perhaps,
>> MemoryMetrics/PersistenceMetrics are intuitive enough.
>>>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>>>> StorageRegion sounds like bad English to me.
>>>>>
>>>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>>>
>>>>> D.
>>>>>
>>>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vozerov@gridgain.com
>>>>> wrote:
>>>>>
>>>>>> Guys,
>>>>>>
>>>>>> But what is exact desicion? :-) I saw two final options:
>>>>>>
>>>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>>>
>>>>>> Which one we choose?
>>>>>>
>>>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>>> I guess it is safe to assume that at this point we came to a
>> consensus?
>>>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>>>
>>>>>>> --Yakov
>>>>>>>
>>
Re: Persistence per memory policy configuration
Posted by Alexey Goncharuk <al...@gmail.com>.
The changes are implemented in ignite-6748 branch. TC run is pending.
2017-10-25 11:26 GMT+03:00 Alexey Goncharuk <al...@gmail.com>:
> Igniters,
>
> I've found one more shortcoming which we missed during the review. The
> checkpoint page buffer is attributed to a data region, so originally it was
> a mistake to place the checkpoint page buffer size to the global
> configuration. This property should go to the data region configuration.
>
> I will prepare a PR shortly so we can review the change.
>
> 2017-10-18 4:52 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>
>> Ivan,
>>
>> Please don’t forget to update all the persistence and memory pools related
>> examples to the new configuration format. Let’s make sure non of our
>> example prints out the class deprecated warning.
>>
>> Denis
>>
>> On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>
>> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
>> > for our users to understand. We should also document this configuration
>> on
>> > readme.
>> >
>> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>> wrote:
>> >
>> > > Dmitriy,
>> > >
>> > > Please check description of https://issues.apache.org/jira
>> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
>> > >
>> > > Best Regards,
>> > > Ivan Rakov
>> > >
>> > >
>> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>> > >
>> > >> I am now confused. Can I please ask for the final configuration
>> again?
>> > >> What
>> > >> will it look like?
>> > >>
>> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> > >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
>> > >>
>> > >> Agree with Ivan. If we implemented backward compatibility, this
>> would be
>> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
>> is.
>> > >>>
>> > >>> As for the swap path, I see nothing wrong with having it for
>> in-memory
>> > >>> caches. This is a simple overflow mechanism that works fine if you
>> do
>> > not
>> > >>> need persistence guarantees.
>> > >>>
>> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>>:
>> > >>>
>> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
>> > >>>> *walArchivePath*.
>> > >>>>
>> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
>> path
>> > >>>>
>> > >>> for
>> > >>>
>> > >>>> WAL is implicitly changed, especially when we have separate option
>> for
>> > >>>>
>> > >>> it.
>> > >>>
>> > >>>> WAL and storage files are already located under same $IGNITE_HOME
>> > root.
>> > >>>> From user perspective, there's no need to change root for all
>> > >>>> persistence-related directories as long as $IGNITE_HOME points to
>> the
>> > >>>> correct disk.
>> > >>>> From developer perspective, this change breaks backwards
>> > compatibility.
>> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
>> > >>>> old-style and new-style paths) is complex and hard to maintain in
>> the
>> > >>>> codebase.
>> > >>>>
>> > >>>> Best Regards,
>> > >>>> Ivan Rakov
>> > >>>>
>> > >>>> My vote is for *storagePath* and keeping behavior as is.
>> > >>>>
>> > >>>>
>> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>> > >>>>
>> > >>>> Igniters, another thing to consider:
>> > >>>>>
>> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> > >>>>> since this is actually not a single file, but a directory path.
>> > >>>>>
>> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
>> > <javascript:;>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>> Seems I've got what you’re talking about.
>> > >>>>>
>> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
>> saw
>> > >>>>>>
>> > >>>>> that
>> > >>>
>> > >>>> only data/indexes were placed to it while wal stayed somewhere in
>> my
>> > >>>>>>
>> > >>>>> work
>> > >>>
>> > >>>> dir. It works counterintuitive and causes non productive
>> discussions
>> > >>>>>>
>> > >>>>> like
>> > >>>
>> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> > >>>>>>
>> > >>>>> name
>> > >>>
>> > >>>> fits this behavior.
>> > >>>>>>
>> > >>>>>> My suggestion will be the following:
>> > >>>>>> - *persistencePath* refers to the path of all storage files
>> > >>>>>> (data/indexes,
>> > >>>>>> wal, archive). If the path is changed *all the files* will be
>> under
>> > >>>>>> the
>> > >>>>>> new
>> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>> > >>>>>> *explicitly*.
>> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
>> > >>>>>> setPersistencePath)
>> > >>>>>> - *setWalArchivePath* overrides the default location of the
>> archive
>> > >>>>>> (which
>> > >>>>>> is again has to be setPersistencePath).
>> > >>>>>>
>> > >>>>>> If we follow this approach the configuration and behavior becomes
>> > >>>>>>
>> > >>>>> vivid.
>> > >>>
>> > >>>> Thoughts?
>> > >>>>>>
>> > >>>>>> —
>> > >>>>>> Denis
>> > >>>>>>
>> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Denis,
>> > >>>>>>>
>> > >>>>>>> Data/index storage and WAL are located under the same root by
>> > >>>>>>> default.
>> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>> > >>>>>>> properties
>> > >>>>>>>
>> > >>>>>>> can contain both absolute and relative paths. If paths are
>> > absolute,
>> > >>>>>> storage and WAL can reside on different devices, like this:
>> > >>>>>>
>> > >>>>>> storagePath: /storage1/NMVe_drive/storage
>> > >>>>>>>
>> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>> > >>>>>>>>
>> > >>>>>>>> We even recommend this in tuning guide:
>> > https://apacheignite.readme
>> > >>>>>>> .
>> > >>>>>>>
>> > >>>>>>> io/docs/durable-memory-tuning
>> > >>>>>>
>> > >>>>>> That's why I think *persistencePath* is misleading.
>> > >>>>>>>
>> > >>>>>>> Best Regards,
>> > >>>>>>> Ivan Rakov
>> > >>>>>>>
>> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> > >>>>>>>
>> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
>> dmagda@gridgain.com
>> > <javascript:;>>
>> > >>>>>>>>
>> > >>>>>>>> wrote:
>> > >>>>>>> From what I see after running an example they are under the
>> same
>> > >>>>>>> root
>> > >>>>>>>
>> > >>>>>>>> folder and in different subdirectories. The root folder should
>> be
>> > >>>>>>>>>
>> > >>>>>>>>> defined
>> > >>>>>>>>
>> > >>>>>>> by setPersistencePath as I guess.
>> > >>>>>>>
>> > >>>>>>>> If that is the case, then you are right. Then we should not
>> have
>> > >>>>>>>>>
>> > >>>>>>>> storagePath or WalPath, and store them both under
>> > "persistencePath"
>> > >>>>>>>>
>> > >>>>>>>> root.
>> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
>> > this.
>> > >>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >
>> >
>>
>
>
Re: Persistence per memory policy configuration
Posted by Alexey Goncharuk <al...@gmail.com>.
Igniters,
I've found one more shortcoming which we missed during the review. The
checkpoint page buffer is attributed to a data region, so originally it was
a mistake to place the checkpoint page buffer size to the global
configuration. This property should go to the data region configuration.
I will prepare a PR shortly so we can review the change.
2017-10-18 4:52 GMT+03:00 Denis Magda <dm...@gridgain.com>:
> Ivan,
>
> Please don’t forget to update all the persistence and memory pools related
> examples to the new configuration format. Let’s make sure non of our
> example prints out the class deprecated warning.
>
> Denis
>
> On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> > for our users to understand. We should also document this configuration
> on
> > readme.
> >
> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>> wrote:
> >
> > > Dmitriy,
> > >
> > > Please check description of https://issues.apache.org/jira
> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> > >
> > >> I am now confused. Can I please ask for the final configuration again?
> > >> What
> > >> will it look like?
> > >>
> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> > >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> > >>
> > >> Agree with Ivan. If we implemented backward compatibility, this would
> be
> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
> is.
> > >>>
> > >>> As for the swap path, I see nothing wrong with having it for
> in-memory
> > >>> caches. This is a simple overflow mechanism that works fine if you do
> > not
> > >>> need persistence guarantees.
> > >>>
> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>>:
> > >>>
> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> > >>>> *walArchivePath*.
> > >>>>
> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
> path
> > >>>>
> > >>> for
> > >>>
> > >>>> WAL is implicitly changed, especially when we have separate option
> for
> > >>>>
> > >>> it.
> > >>>
> > >>>> WAL and storage files are already located under same $IGNITE_HOME
> > root.
> > >>>> From user perspective, there's no need to change root for all
> > >>>> persistence-related directories as long as $IGNITE_HOME points to
> the
> > >>>> correct disk.
> > >>>> From developer perspective, this change breaks backwards
> > compatibility.
> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
> > >>>> old-style and new-style paths) is complex and hard to maintain in
> the
> > >>>> codebase.
> > >>>>
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> My vote is for *storagePath* and keeping behavior as is.
> > >>>>
> > >>>>
> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > >>>>
> > >>>> Igniters, another thing to consider:
> > >>>>>
> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> > >>>>> since this is actually not a single file, but a directory path.
> > >>>>>
> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
> > <javascript:;>>
> > >>>>> wrote:
> > >>>>>
> > >>>>> Seems I've got what you’re talking about.
> > >>>>>
> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
> saw
> > >>>>>>
> > >>>>> that
> > >>>
> > >>>> only data/indexes were placed to it while wal stayed somewhere in my
> > >>>>>>
> > >>>>> work
> > >>>
> > >>>> dir. It works counterintuitive and causes non productive discussions
> > >>>>>>
> > >>>>> like
> > >>>
> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> > >>>>>>
> > >>>>> name
> > >>>
> > >>>> fits this behavior.
> > >>>>>>
> > >>>>>> My suggestion will be the following:
> > >>>>>> - *persistencePath* refers to the path of all storage files
> > >>>>>> (data/indexes,
> > >>>>>> wal, archive). If the path is changed *all the files* will be
> under
> > >>>>>> the
> > >>>>>> new
> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> > >>>>>> *explicitly*.
> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
> > >>>>>> setPersistencePath)
> > >>>>>> - *setWalArchivePath* overrides the default location of the
> archive
> > >>>>>> (which
> > >>>>>> is again has to be setPersistencePath).
> > >>>>>>
> > >>>>>> If we follow this approach the configuration and behavior becomes
> > >>>>>>
> > >>>>> vivid.
> > >>>
> > >>>> Thoughts?
> > >>>>>>
> > >>>>>> —
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Denis,
> > >>>>>>>
> > >>>>>>> Data/index storage and WAL are located under the same root by
> > >>>>>>> default.
> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> > >>>>>>> properties
> > >>>>>>>
> > >>>>>>> can contain both absolute and relative paths. If paths are
> > absolute,
> > >>>>>> storage and WAL can reside on different devices, like this:
> > >>>>>>
> > >>>>>> storagePath: /storage1/NMVe_drive/storage
> > >>>>>>>
> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> > >>>>>>>>
> > >>>>>>>> We even recommend this in tuning guide:
> > https://apacheignite.readme
> > >>>>>>> .
> > >>>>>>>
> > >>>>>>> io/docs/durable-memory-tuning
> > >>>>>>
> > >>>>>> That's why I think *persistencePath* is misleading.
> > >>>>>>>
> > >>>>>>> Best Regards,
> > >>>>>>> Ivan Rakov
> > >>>>>>>
> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>>>>>
> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
> dmagda@gridgain.com
> > <javascript:;>>
> > >>>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>> From what I see after running an example they are under the
> same
> > >>>>>>> root
> > >>>>>>>
> > >>>>>>>> folder and in different subdirectories. The root folder should
> be
> > >>>>>>>>>
> > >>>>>>>>> defined
> > >>>>>>>>
> > >>>>>>> by setPersistencePath as I guess.
> > >>>>>>>
> > >>>>>>>> If that is the case, then you are right. Then we should not have
> > >>>>>>>>>
> > >>>>>>>> storagePath or WalPath, and store them both under
> > "persistencePath"
> > >>>>>>>>
> > >>>>>>>> root.
> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> > this.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >
> >
>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@gridgain.com>.
Ivan,
Please don’t forget to update all the persistence and memory pools related
examples to the new configuration format. Let’s make sure non of our
example prints out the class deprecated warning.
Denis
On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> for our users to understand. We should also document this configuration on
> readme.
>
> On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>> wrote:
>
> > Dmitriy,
> >
> > Please check description of https://issues.apache.org/jira
> > /browse/IGNITE-6030, I've updated it with actual list of properties.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> >
> >> I am now confused. Can I please ask for the final configuration again?
> >> What
> >> will it look like?
> >>
> >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> >>
> >> Agree with Ivan. If we implemented backward compatibility, this would be
> >>> completely counterintuitive behavior, so +1 to keep the behavior as is.
> >>>
> >>> As for the swap path, I see nothing wrong with having it for in-memory
> >>> caches. This is a simple overflow mechanism that works fine if you do
> not
> >>> need persistence guarantees.
> >>>
> >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>:
> >>>
> >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> >>>> *walArchivePath*.
> >>>>
> >>>> Regarding persistencePath/storagePath, I don't like the idea when path
> >>>>
> >>> for
> >>>
> >>>> WAL is implicitly changed, especially when we have separate option for
> >>>>
> >>> it.
> >>>
> >>>> WAL and storage files are already located under same $IGNITE_HOME
> root.
> >>>> From user perspective, there's no need to change root for all
> >>>> persistence-related directories as long as $IGNITE_HOME points to the
> >>>> correct disk.
> >>>> From developer perspective, this change breaks backwards
> compatibility.
> >>>> Maintaining backwards compatibility in fail-safe way (checking both
> >>>> old-style and new-style paths) is complex and hard to maintain in the
> >>>> codebase.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> My vote is for *storagePath* and keeping behavior as is.
> >>>>
> >>>>
> >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >>>>
> >>>> Igniters, another thing to consider:
> >>>>>
> >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >>>>> since this is actually not a single file, but a directory path.
> >>>>>
> >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
> <javascript:;>>
> >>>>> wrote:
> >>>>>
> >>>>> Seems I've got what you’re talking about.
> >>>>>
> >>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
> >>>>>>
> >>>>> that
> >>>
> >>>> only data/indexes were placed to it while wal stayed somewhere in my
> >>>>>>
> >>>>> work
> >>>
> >>>> dir. It works counterintuitive and causes non productive discussions
> >>>>>>
> >>>>> like
> >>>
> >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> >>>>>>
> >>>>> name
> >>>
> >>>> fits this behavior.
> >>>>>>
> >>>>>> My suggestion will be the following:
> >>>>>> - *persistencePath* refers to the path of all storage files
> >>>>>> (data/indexes,
> >>>>>> wal, archive). If the path is changed *all the files* will be under
> >>>>>> the
> >>>>>> new
> >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>>>>> *explicitly*.
> >>>>>> - *setWalPath* overrides the default location of WAL (which is
> >>>>>> setPersistencePath)
> >>>>>> - *setWalArchivePath* overrides the default location of the archive
> >>>>>> (which
> >>>>>> is again has to be setPersistencePath).
> >>>>>>
> >>>>>> If we follow this approach the configuration and behavior becomes
> >>>>>>
> >>>>> vivid.
> >>>
> >>>> Thoughts?
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Denis,
> >>>>>>>
> >>>>>>> Data/index storage and WAL are located under the same root by
> >>>>>>> default.
> >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> >>>>>>> properties
> >>>>>>>
> >>>>>>> can contain both absolute and relative paths. If paths are
> absolute,
> >>>>>> storage and WAL can reside on different devices, like this:
> >>>>>>
> >>>>>> storagePath: /storage1/NMVe_drive/storage
> >>>>>>>
> >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>>>>
> >>>>>>>> We even recommend this in tuning guide:
> https://apacheignite.readme
> >>>>>>> .
> >>>>>>>
> >>>>>>> io/docs/durable-memory-tuning
> >>>>>>
> >>>>>> That's why I think *persistencePath* is misleading.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>>>>
> >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dmagda@gridgain.com
> <javascript:;>>
> >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>> From what I see after running an example they are under the same
> >>>>>>> root
> >>>>>>>
> >>>>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>>>>
> >>>>>>>>> defined
> >>>>>>>>
> >>>>>>> by setPersistencePath as I guess.
> >>>>>>>
> >>>>>>>> If that is the case, then you are right. Then we should not have
> >>>>>>>>>
> >>>>>>>> storagePath or WalPath, and store them both under
> "persistencePath"
> >>>>>>>>
> >>>>>>>> root.
> >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >
>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Thanks Ivan! Let's make sure that every property gets sufficient javadoc
for our users to understand. We should also document this configuration on
readme.
On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <iv...@gmail.com> wrote:
> Dmitriy,
>
> Please check description of https://issues.apache.org/jira
> /browse/IGNITE-6030, I've updated it with actual list of properties.
>
> Best Regards,
> Ivan Rakov
>
>
> On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>
>> I am now confused. Can I please ask for the final configuration again?
>> What
>> will it look like?
>>
>> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> alexey.goncharuk@gmail.com> wrote:
>>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>>
>>> As for the swap path, I see nothing wrong with having it for in-memory
>>> caches. This is a simple overflow mechanism that works fine if you do not
>>> need persistence guarantees.
>>>
>>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>>> *walArchivePath*.
>>>>
>>>> Regarding persistencePath/storagePath, I don't like the idea when path
>>>>
>>> for
>>>
>>>> WAL is implicitly changed, especially when we have separate option for
>>>>
>>> it.
>>>
>>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>>> From user perspective, there's no need to change root for all
>>>> persistence-related directories as long as $IGNITE_HOME points to the
>>>> correct disk.
>>>> From developer perspective, this change breaks backwards compatibility.
>>>> Maintaining backwards compatibility in fail-safe way (checking both
>>>> old-style and new-style paths) is complex and hard to maintain in the
>>>> codebase.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> My vote is for *storagePath* and keeping behavior as is.
>>>>
>>>>
>>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>>
>>>> Igniters, another thing to consider:
>>>>>
>>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>>> since this is actually not a single file, but a directory path.
>>>>>
>>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Seems I've got what you’re talking about.
>>>>>
>>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>>>>>>
>>>>> that
>>>
>>>> only data/indexes were placed to it while wal stayed somewhere in my
>>>>>>
>>>>> work
>>>
>>>> dir. It works counterintuitive and causes non productive discussions
>>>>>>
>>>>> like
>>>
>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>>>>>>
>>>>> name
>>>
>>>> fits this behavior.
>>>>>>
>>>>>> My suggestion will be the following:
>>>>>> - *persistencePath* refers to the path of all storage files
>>>>>> (data/indexes,
>>>>>> wal, archive). If the path is changed *all the files* will be under
>>>>>> the
>>>>>> new
>>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>>> *explicitly*.
>>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>>> setPersistencePath)
>>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>>> (which
>>>>>> is again has to be setPersistencePath).
>>>>>>
>>>>>> If we follow this approach the configuration and behavior becomes
>>>>>>
>>>>> vivid.
>>>
>>>> Thoughts?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Denis,
>>>>>>>
>>>>>>> Data/index storage and WAL are located under the same root by
>>>>>>> default.
>>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>>>>>>> properties
>>>>>>>
>>>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>>> storage and WAL can reside on different devices, like this:
>>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>>
>>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>>
>>>>>>>> We even recommend this in tuning guide: https://apacheignite.readme
>>>>>>> .
>>>>>>>
>>>>>>> io/docs/durable-memory-tuning
>>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>>>>
>>>>>>>> wrote:
>>>>>>> From what I see after running an example they are under the same
>>>>>>> root
>>>>>>>
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>>
>>>>>>>>> defined
>>>>>>>>
>>>>>>> by setPersistencePath as I guess.
>>>>>>>
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>>>>
>>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>>
>>>>>>>> root.
>>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>
>>>>>>>>
>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Dmitriy,
Please check description of
https://issues.apache.org/jira/browse/IGNITE-6030, I've updated it with
actual list of properties.
Best Regards,
Ivan Rakov
On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> I am now confused. Can I please ask for the final configuration again? What
> will it look like?
>
> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>
>> As for the swap path, I see nothing wrong with having it for in-memory
>> caches. This is a simple overflow mechanism that works fine if you do not
>> need persistence guarantees.
>>
>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>> *walArchivePath*.
>>>
>>> Regarding persistencePath/storagePath, I don't like the idea when path
>> for
>>> WAL is implicitly changed, especially when we have separate option for
>> it.
>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>> From user perspective, there's no need to change root for all
>>> persistence-related directories as long as $IGNITE_HOME points to the
>>> correct disk.
>>> From developer perspective, this change breaks backwards compatibility.
>>> Maintaining backwards compatibility in fail-safe way (checking both
>>> old-style and new-style paths) is complex and hard to maintain in the
>>> codebase.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> My vote is for *storagePath* and keeping behavior as is.
>>>
>>>
>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>
>>>> Igniters, another thing to consider:
>>>>
>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>> since this is actually not a single file, but a directory path.
>>>>
>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> Seems I've got what you’re talking about.
>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>> that
>>>>> only data/indexes were placed to it while wal stayed somewhere in my
>> work
>>>>> dir. It works counterintuitive and causes non productive discussions
>> like
>>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> name
>>>>> fits this behavior.
>>>>>
>>>>> My suggestion will be the following:
>>>>> - *persistencePath* refers to the path of all storage files
>>>>> (data/indexes,
>>>>> wal, archive). If the path is changed *all the files* will be under the
>>>>> new
>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>> *explicitly*.
>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>> setPersistencePath)
>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>> (which
>>>>> is again has to be setPersistencePath).
>>>>>
>>>>> If we follow this approach the configuration and behavior becomes
>> vivid.
>>>>> Thoughts?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>> Denis,
>>>>>>
>>>>>> Data/index storage and WAL are located under the same root by default.
>>>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>>>
>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>> storage and WAL can reside on different devices, like this:
>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>
>>>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>>>
>>>>> io/docs/durable-memory-tuning
>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>>>
>>>>>> wrote:
>>>>>> From what I see after running an example they are under the same root
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>
>>>>>>> defined
>>>>>> by setPersistencePath as I guess.
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>
>>>>>> root.
>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am now confused. Can I please ask for the final configuration again? What
will it look like?
On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Agree with Ivan. If we implemented backward compatibility, this would be
> completely counterintuitive behavior, so +1 to keep the behavior as is.
>
> As for the swap path, I see nothing wrong with having it for in-memory
> caches. This is a simple overflow mechanism that works fine if you do not
> need persistence guarantees.
>
> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> for
> > WAL is implicitly changed, especially when we have separate option for
> it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> > From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> > From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >
> >> Igniters, another thing to consider:
> >>
> >> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >> since this is actually not a single file, but a directory path.
> >>
> >> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
> >>
> >> Seems I've got what you’re talking about.
> >>>
> >>> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> >>> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >>> dir. It works counterintuitive and causes non productive discussions
> like
> >>> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> >>> fits this behavior.
> >>>
> >>> My suggestion will be the following:
> >>> - *persistencePath* refers to the path of all storage files
> >>> (data/indexes,
> >>> wal, archive). If the path is changed *all the files* will be under the
> >>> new
> >>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>> *explicitly*.
> >>> - *setWalPath* overrides the default location of WAL (which is
> >>> setPersistencePath)
> >>> - *setWalArchivePath* overrides the default location of the archive
> >>> (which
> >>> is again has to be setPersistencePath).
> >>>
> >>> If we follow this approach the configuration and behavior becomes
> vivid.
> >>> Thoughts?
> >>>
> >>> —
> >>> Denis
> >>>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>>
> >>>> Denis,
> >>>>
> >>>> Data/index storage and WAL are located under the same root by default.
> >>>> However, this is not mandatory: *storagePath* and *walPath* properties
> >>>>
> >>> can contain both absolute and relative paths. If paths are absolute,
> >>> storage and WAL can reside on different devices, like this:
> >>>
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>
> >>>> We even recommend this in tuning guide: https://apacheignite.readme.
> >>>>
> >>> io/docs/durable-memory-tuning
> >>>
> >>>> That's why I think *persistencePath* is misleading.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>
> >>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> >>>>>
> >>>> wrote:
> >>>
> >>>> From what I see after running an example they are under the same root
> >>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>
> >>>>> defined
> >>>
> >>>> by setPersistencePath as I guess.
> >>>>>>
> >>>>>> If that is the case, then you are right. Then we should not have
> >>>>> storagePath or WalPath, and store them both under "persistencePath"
> >>>>>
> >>>> root.
> >>>
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>>
> >>>>>
> >>>
> >
>
Re: Persistence per memory policy configuration
Posted by Alexey Goncharuk <al...@gmail.com>.
Agree with Ivan. If we implemented backward compatibility, this would be
completely counterintuitive behavior, so +1 to keep the behavior as is.
As for the swap path, I see nothing wrong with having it for in-memory
caches. This is a simple overflow mechanism that works fine if you do not
need persistence guarantees.
2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path for
> WAL is implicitly changed, especially when we have separate option for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
> From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
> From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>
>> Igniters, another thing to consider:
>>
>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> since this is actually not a single file, but a directory path.
>>
>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>>
>> Seems I've got what you’re talking about.
>>>
>>> I’ve tried to change the root directory (*persistencePath*) and saw that
>>> only data/indexes were placed to it while wal stayed somewhere in my work
>>> dir. It works counterintuitive and causes non productive discussions like
>>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>>> fits this behavior.
>>>
>>> My suggestion will be the following:
>>> - *persistencePath* refers to the path of all storage files
>>> (data/indexes,
>>> wal, archive). If the path is changed *all the files* will be under the
>>> new
>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>> *explicitly*.
>>> - *setWalPath* overrides the default location of WAL (which is
>>> setPersistencePath)
>>> - *setWalArchivePath* overrides the default location of the archive
>>> (which
>>> is again has to be setPersistencePath).
>>>
>>> If we follow this approach the configuration and behavior becomes vivid.
>>> Thoughts?
>>>
>>> —
>>> Denis
>>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Denis,
>>>>
>>>> Data/index storage and WAL are located under the same root by default.
>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>
>>> can contain both absolute and relative paths. If paths are absolute,
>>> storage and WAL can reside on different devices, like this:
>>>
>>>> storagePath: /storage1/NMVe_drive/storage
>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>
>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>
>>> io/docs/durable-memory-tuning
>>>
>>>> That's why I think *persistencePath* is misleading.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>
>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>
>>>> wrote:
>>>
>>>> From what I see after running an example they are under the same root
>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>
>>>>> defined
>>>
>>>> by setPersistencePath as I guess.
>>>>>>
>>>>>> If that is the case, then you are right. Then we should not have
>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>
>>>> root.
>>>
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>
>>>>>
>>>
>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Igniters, we should not have Swap in the property names. Swap is something
that behaves completely differently from Ignite persistence, and I don't
think it should be present within Ignite altogether.
D.
On Mon, Oct 16, 2017 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> +1 to Ivan's suggestion.
>
> пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <iv...@gmail.com>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> > for WAL is implicitly changed, especially when we have separate option
> > for it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> > From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> > From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > > Igniters, another thing to consider:
> > >
> > > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > > since this is actually not a single file, but a directory path.
> > >
> > > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > >> Seems I've got what you’re talking about.
> > >>
> > >> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> > >> only data/indexes were placed to it while wal stayed somewhere in my
> > work
> > >> dir. It works counterintuitive and causes non productive discussions
> > like
> > >> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> > >> fits this behavior.
> > >>
> > >> My suggestion will be the following:
> > >> - *persistencePath* refers to the path of all storage files
> > (data/indexes,
> > >> wal, archive). If the path is changed *all the files* will be under
> the
> > new
> > >> directory unless *setWalPath* and *setWalArchivePath* are set
> > *explicitly*.
> > >> - *setWalPath* overrides the default location of WAL (which is
> > >> setPersistencePath)
> > >> - *setWalArchivePath* overrides the default location of the archive
> > (which
> > >> is again has to be setPersistencePath).
> > >>
> > >> If we follow this approach the configuration and behavior becomes
> vivid.
> > >> Thoughts?
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com>
> wrote:
> > >>>
> > >>> Denis,
> > >>>
> > >>> Data/index storage and WAL are located under the same root by
> default.
> > >>> However, this is not mandatory: *storagePath* and *walPath*
> properties
> > >> can contain both absolute and relative paths. If paths are absolute,
> > >> storage and WAL can reside on different devices, like this:
> > >>>> storagePath: /storage1/NMVe_drive/storage
> > >>>> walPath: /storage2/Big_SSD_drive/wal
> > >>> We even recommend this in tuning guide: https://apacheignite.readme.
> > >> io/docs/durable-memory-tuning
> > >>> That's why I think *persistencePath* is misleading.
> > >>>
> > >>> Best Regards,
> > >>> Ivan Rakov
> > >>>
> > >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> > >> wrote:
> > >>>>> From what I see after running an example they are under the same
> > root
> > >>>>> folder and in different subdirectories. The root folder should be
> > >> defined
> > >>>>> by setPersistencePath as I guess.
> > >>>>>
> > >>>> If that is the case, then you are right. Then we should not have
> > >>>> storagePath or WalPath, and store them both under "persistencePath"
> > >> root.
> > >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> > >>>>
> > >>
> >
> >
>
Re: Persistence per memory policy configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
+1 to Ivan's suggestion.
пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <iv...@gmail.com>:
> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path
> for WAL is implicitly changed, especially when we have separate option
> for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
> From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
> From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > Igniters, another thing to consider:
> >
> > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > since this is actually not a single file, but a directory path.
> >
> > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Seems I've got what you’re talking about.
> >>
> >> I’ve tried to change the root directory (*persistencePath*) and saw that
> >> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >> dir. It works counterintuitive and causes non productive discussions
> like
> >> we are in arguing about *persistencePath* or *storagePath*. Neither name
> >> fits this behavior.
> >>
> >> My suggestion will be the following:
> >> - *persistencePath* refers to the path of all storage files
> (data/indexes,
> >> wal, archive). If the path is changed *all the files* will be under the
> new
> >> directory unless *setWalPath* and *setWalArchivePath* are set
> *explicitly*.
> >> - *setWalPath* overrides the default location of WAL (which is
> >> setPersistencePath)
> >> - *setWalArchivePath* overrides the default location of the archive
> (which
> >> is again has to be setPersistencePath).
> >>
> >> If we follow this approach the configuration and behavior becomes vivid.
> >> Thoughts?
> >>
> >> —
> >> Denis
> >>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>
> >>> Denis,
> >>>
> >>> Data/index storage and WAL are located under the same root by default.
> >>> However, this is not mandatory: *storagePath* and *walPath* properties
> >> can contain both absolute and relative paths. If paths are absolute,
> >> storage and WAL can reside on different devices, like this:
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>> walPath: /storage2/Big_SSD_drive/wal
> >>> We even recommend this in tuning guide: https://apacheignite.readme.
> >> io/docs/durable-memory-tuning
> >>> That's why I think *persistencePath* is misleading.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> >> wrote:
> >>>>> From what I see after running an example they are under the same
> root
> >>>>> folder and in different subdirectories. The root folder should be
> >> defined
> >>>>> by setPersistencePath as I guess.
> >>>>>
> >>>> If that is the case, then you are right. Then we should not have
> >>>> storagePath or WalPath, and store them both under "persistencePath"
> >> root.
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>
> >>
>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
*swapPath* is ok for me. It is also consistent with *walPath* and
*walArchivePath*.
Regarding persistencePath/storagePath, I don't like the idea when path
for WAL is implicitly changed, especially when we have separate option
for it.
WAL and storage files are already located under same $IGNITE_HOME root.
From user perspective, there's no need to change root for all
persistence-related directories as long as $IGNITE_HOME points to the
correct disk.
From developer perspective, this change breaks backwards compatibility.
Maintaining backwards compatibility in fail-safe way (checking both
old-style and new-style paths) is complex and hard to maintain in the
codebase.
Best Regards,
Ivan Rakov
My vote is for *storagePath* and keeping behavior as is.
On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> Igniters, another thing to consider:
>
> DataRegionConfiguration.SwapFilePath should be SwapPath,
> since this is actually not a single file, but a directory path.
>
> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Seems I've got what you’re talking about.
>>
>> I’ve tried to change the root directory (*persistencePath*) and saw that
>> only data/indexes were placed to it while wal stayed somewhere in my work
>> dir. It works counterintuitive and causes non productive discussions like
>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>> fits this behavior.
>>
>> My suggestion will be the following:
>> - *persistencePath* refers to the path of all storage files (data/indexes,
>> wal, archive). If the path is changed *all the files* will be under the new
>> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
>> - *setWalPath* overrides the default location of WAL (which is
>> setPersistencePath)
>> - *setWalArchivePath* overrides the default location of the archive (which
>> is again has to be setPersistencePath).
>>
>> If we follow this approach the configuration and behavior becomes vivid.
>> Thoughts?
>>
>> —
>> Denis
>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis,
>>>
>>> Data/index storage and WAL are located under the same root by default.
>>> However, this is not mandatory: *storagePath* and *walPath* properties
>> can contain both absolute and relative paths. If paths are absolute,
>> storage and WAL can reside on different devices, like this:
>>>> storagePath: /storage1/NMVe_drive/storage
>>>> walPath: /storage2/Big_SSD_drive/wal
>>> We even recommend this in tuning guide: https://apacheignite.readme.
>> io/docs/durable-memory-tuning
>>> That's why I think *persistencePath* is misleading.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>> wrote:
>>>>> From what I see after running an example they are under the same root
>>>>> folder and in different subdirectories. The root folder should be
>> defined
>>>>> by setPersistencePath as I guess.
>>>>>
>>>> If that is the case, then you are right. Then we should not have
>>>> storagePath or WalPath, and store them both under "persistencePath"
>> root.
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>
>>
Re: Persistence per memory policy configuration
Posted by Pavel Tupitsyn <pt...@apache.org>.
Igniters, another thing to consider:
DataRegionConfiguration.SwapFilePath should be SwapPath,
since this is actually not a single file, but a directory path.
On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
> Seems I've got what you’re talking about.
>
> I’ve tried to change the root directory (*persistencePath*) and saw that
> only data/indexes were placed to it while wal stayed somewhere in my work
> dir. It works counterintuitive and causes non productive discussions like
> we are in arguing about *persistencePath* or *storagePath*. Neither name
> fits this behavior.
>
> My suggestion will be the following:
> - *persistencePath* refers to the path of all storage files (data/indexes,
> wal, archive). If the path is changed *all the files* will be under the new
> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
> - *setWalPath* overrides the default location of WAL (which is
> setPersistencePath)
> - *setWalArchivePath* overrides the default location of the archive (which
> is again has to be setPersistencePath).
>
> If we follow this approach the configuration and behavior becomes vivid.
> Thoughts?
>
> —
> Denis
>
> > On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Denis,
> >
> > Data/index storage and WAL are located under the same root by default.
> > However, this is not mandatory: *storagePath* and *walPath* properties
> can contain both absolute and relative paths. If paths are absolute,
> storage and WAL can reside on different devices, like this:
> >
> >> storagePath: /storage1/NMVe_drive/storage
> >> walPath: /storage2/Big_SSD_drive/wal
> > We even recommend this in tuning guide: https://apacheignite.readme.
> io/docs/durable-memory-tuning
> >
> > That's why I think *persistencePath* is misleading.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> wrote:
> >>
> >>> From what I see after running an example they are under the same root
> >>> folder and in different subdirectories. The root folder should be
> defined
> >>> by setPersistencePath as I guess.
> >>>
> >> If that is the case, then you are right. Then we should not have
> >> storagePath or WalPath, and store them both under "persistencePath"
> root.
> >> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>
> >
>
>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
Seems I've got what you’re talking about.
I’ve tried to change the root directory (*persistencePath*) and saw that only data/indexes were placed to it while wal stayed somewhere in my work dir. It works counterintuitive and causes non productive discussions like we are in arguing about *persistencePath* or *storagePath*. Neither name fits this behavior.
My suggestion will be the following:
- *persistencePath* refers to the path of all storage files (data/indexes, wal, archive). If the path is changed *all the files* will be under the new directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
- *setWalPath* overrides the default location of WAL (which is setPersistencePath)
- *setWalArchivePath* overrides the default location of the archive (which is again has to be setPersistencePath).
If we follow this approach the configuration and behavior becomes vivid. Thoughts?
—
Denis
> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> Denis,
>
> Data/index storage and WAL are located under the same root by default.
> However, this is not mandatory: *storagePath* and *walPath* properties can contain both absolute and relative paths. If paths are absolute, storage and WAL can reside on different devices, like this:
>
>> storagePath: /storage1/NMVe_drive/storage
>> walPath: /storage2/Big_SSD_drive/wal
> We even recommend this in tuning guide: https://apacheignite.readme.io/docs/durable-memory-tuning
>
> That's why I think *persistencePath* is misleading.
>
> Best Regards,
> Ivan Rakov
>
> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:
>>
>>> From what I see after running an example they are under the same root
>>> folder and in different subdirectories. The root folder should be defined
>>> by setPersistencePath as I guess.
>>>
>> If that is the case, then you are right. Then we should not have
>> storagePath or WalPath, and store them both under "persistencePath" root.
>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Denis,
Data/index storage and WAL are located under the same root by default.
However, this is not mandatory: *storagePath* and *walPath* properties
can contain both absolute and relative paths. If paths are absolute,
storage and WAL can reside on different devices, like this:
> storagePath: /storage1/NMVe_drive/storage
> walPath: /storage2/Big_SSD_drive/wal
We even recommend this in tuning guide:
https://apacheignite.readme.io/docs/durable-memory-tuning
That's why I think *persistencePath* is misleading.
Best Regards,
Ivan Rakov
On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>> From what I see after running an example they are under the same root
>> folder and in different subdirectories. The root folder should be defined
>> by setPersistencePath as I guess.
>>
> If that is the case, then you are right. Then we should not have
> storagePath or WalPath, and store them both under "persistencePath" root.
> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:
> From what I see after running an example they are under the same root
> folder and in different subdirectories. The root folder should be defined
> by setPersistencePath as I guess.
>
If that is the case, then you are right. Then we should not have
storagePath or WalPath, and store them both under "persistencePath" root.
However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@gridgain.com>.
From what I see after running an example they are under the same root
folder and in different subdirectories. The root folder should be defined
by setPersistencePath as I guess.
Denis
On Thursday, October 12, 2017, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> On Thu, Oct 12, 2017 at 4:05 PM, Denis Magda <dmagda@apache.org
> <javascript:;>> wrote:
>
> > Both terms “persistence” and “storage” mean same to me. If to choose
> > between them I would go for “persistence” because this is how
> > we name the feature - “Ignite Persistence” [1]
>
>
> > Furthermore, by default WAL files, the archive and data/index files are
> > located under the same root which is “setPersistencePath” directory.
> >
>
> Denis, to my knowledge, WAL can span multiple caches, no? How can it end up
> in the same folder with data and indexes, if those are per cache?
>
>
> >
> > [1] https://ignite.apache.org/features/persistence.html <
> > https://ignite.apache.org/features/persistence.html>
> >
> > —
> > Denis
> >
> > > On Oct 12, 2017, at 12:00 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>> wrote:
> > >
> > > Name *setPersistencePath* looks confusing to me because both WAL and
> > index/partition files storage provide persistence. That's why we
> separated
> > API methods as *setWalPath* and *setStoragePath*.
> > > I think, *setStoragePath* is good enough as long as it's supported by
> > explaining javadoc.*
> > > *
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > > On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> > >> Is the storage path the root folder for the persistence or only the
> root
> > >> path for the main storage?
> > >>
> > >> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dmagda@apache.org
> <javascript:;>> wrote:
> > >>
> > >>> Ivan,
> > >>>
> > >>> Instead of “setStoragePath” I would suggest “setPersistencePath”.
> Left
> > >>> some extra notes in the ticket.
> > >>>
> > >>
> > >>> —
> > >>> Denis
> > >>>
> > >>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>
> > wrote:
> > >>>>
> > >>>> Vladimir,
> > >>>>
> > >>>> Thanks for focusing on existing namings. Most of your suggestions
> > really
> > >>> sound better. I've posted my thoughts under your comment.
> > >>>> By the way, we should decide two things:
> > >>>>
> > >>>> 1) Naming for methods for configuring store path. I suggest the
> > >>> following:
> > >>>> *setStoragePath* - for partition and index files
> > >>>> *setWalPath* - for WAL files
> > >>>> *walArchivePath* - for WAL archive files
> > >>>>
> > >>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same
> > with
> > >>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both
> options
> > >>> sounds ok to me, let's see what community thinks.
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
> > >>>>> Ivan,
> > >>>>>
> > >>>>> I left some comments in the ticket [1], please take a look.
> > >>>>>
> > >>>>> [1]
> > >>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
> > >>> focusedCommentId=16200050&page=com.atlassian.jira.
> > >>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> > >>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>>
> > >>> wrote:
> > >>>>>> Igniters,
> > >>>>>>
> > >>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> > >>> enqueued
> > >>>>>> for TC run.
> > >>>>>> PR: https://github.com/apache/ignite/pull/2828
> > >>>>>>
> > >>>>>> Everyone interested in new data storage configuration API, please
> > pay
> > >>>>>> attention and review.
> > >>>>>>
> > >>>>>>
> > >>>>>> Best Regards,
> > >>>>>> Ivan Rakov
> > >>>>>>
> > >>>>>>
> > >>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> > >>>>>>
> > >>>>>>> Sounds good to me.
> > >>>>>>>
> > >>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>
> > >
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Pavel,
> > >>>>>>>> Sounds reasonable.
> > >>>>>>>> I suggest to include both "data" and "configuration" to make it
> > even
> > >>> more
> > >>>>>>>> obvious:
> > >>>>>>>>
> > >>>>>>>> set/getDefaultDataRegionConfiguration
> > >>>>>>>> set/getDataRegionConfigurations
> > >>>>>>>>
> > >>>>>>>> Best Regards,
> > >>>>>>>> Ivan Rakov
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> > >>>>>>>>
> > >>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
> > >>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
> > >>>>>>>>> DataRegionConfiguration[] getDataRegions
> > >>>>>>>>>
> > >>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
> > >>>>>>>>> - or -
> > >>>>>>>>> defaultDataRegion + getDataRegions
> > >>>>>>>>>
> > >>>>>>>>> Thoughts?
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Denis,
> > >>>>>>>>>
> > >>>>>>>>>> Yes, you're right. All cache groups without specific data
> region
> > >>>>>>>>>> configured will be persistent.
> > >>>>>>>>>> And if you want to add another persistent data region, you
> > should
> > >>> set
> > >>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> > >>> explictly.
> > >>>>>>>>>> Best Regards,
> > >>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to
> > wrap
> > >>> this
> > >>>>>>>>>>> up, if I want to enable the persistence globally and don’t
> have
> > >>> any
> > >>>>>>>>>>> regions
> > >>>>>>>>>>> configured explicitly I need to take the default region and
> > >>> switch the
> > >>>>>>>>>>> persistence on for it. Is my understanding correct?
> > >>>>>>>>>>>
> > >>>>>>>>>>> —
> > >>>>>>>>>>> Denis
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>
> > >
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Denis, why do you need to access an instance of the default
> > region
> > >>>>>>>>>>>> bean?
> > >>>>>>>>>>>> If you want to set any parameter, just instantiate new bean
> > with
> > >>> this
> > >>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters
> > will
> > >>> be
> > >>>>>>>>>>>> automatically initialized with their default values.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> <property name="dataStorageConfiguration">
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> > >>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
> > >>>>>>>>>>>>>>> value="#{100
> > >>>>>>>>>>>>>>> *
> > >>>>>>>>>>>>>>> 1024 * 1024}"/>
> > >>>>>>>>>>>>>>> <property name="
> > defaultRegionConfiguration">
> > >>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> > >>>>>>>>>>>>>>> <property name="maxSize"
> > value="#{5 *
> > >>>>>>>>>>>>>>> 1024 *
> > >>>>>>>>>>>>>>> 102 * 1024}"/>
> > >>>>>>>>>>>>>>> </bean>
> > >>>>>>>>>>>>>>> </property>
> > >>>>>>>>>>>>>>> </bean>
> > >>>>>>>>>>>>>>> </property>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> > default.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean
> and
> > >>> change
> > >>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the
> > persistence I
> > >>>>>>>>>>>>> don’t want
> > >>>>>>>>>>>>> to create the default region bean from scratch.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —
> > >>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Agree with Alexey.
> > >>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
> > >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>> can confuse users who don't know that there's such thing
> as
> > >>> default
> > >>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>> region. They can decide they are inherited by all data
> > regions
> > >>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>> size
> > >>>>>>>>>>>>>> and persistence flag are not explicitly set.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Let's get rid of these properties and add
> > >>>>>>>>>>>>>> *defaultRegionConfiguration*
> > >>>>>>>>>>>>>> property with explicit configuration of default data
> region.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence
> > flag
> > >>> of
> > >>>>>>>>>>>>>> default data region will be just two lines longer (for
> bean
> > >>>>>>>>>>>>>> description):
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> <property name="dataStorageConfiguration">
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> > >>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
> > >>>>>>>>>>>>>>> value="#{100
> > >>>>>>>>>>>>>>> *
> > >>>>>>>>>>>>>>> 1024 * 1024}"/>
> > >>>>>>>>>>>>>>> <property name="
> > defaultRegionConfiguration">
> > >>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> > >>>>>>>>>>>>>>> <property name="maxSize"
> > value="#{5 *
> > >>>>>>>>>>>>>>> 1024 *
> > >>>>>>>>>>>>>>> 102 * 1024}"/>
> > >>>>>>>>>>>>>>> </bean>
> > >>>>>>>>>>>>>>> </property>
> > >>>>>>>>>>>>>>> </bean>
> > >>>>>>>>>>>>>>> </property>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> > default.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> > >>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> To resolve this, I suggest to
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> and
> > >>> get
> > >>>>>>>>>>>>>>>> rid
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> > >>> file? I’m
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>> an expert in Spring so how do I get
> > defaultRegionConfiguration
> > >>>>>>>>>>>>>>> bean
> > >>>>>>>>>>>>>>> first
> > >>>>>>>>>>>>>>> to change any parameter?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> > >>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we
> would
> > >>> either
> > >>>>>>>>>>>>>>>> need to
> > >>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> > >>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this
> > field
> > >>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>> is also
> > >>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> > >>> defaults
> > >>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> > >>> suggest
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> > and
> > >>> get
> > >>>>>>>>>>>>>>>> rid
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thoughts?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>
> > >>>> :
> > >>>>>>>>>>>>>>>> Vladimir,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
> > >>>>>>>>>>>>>>>>> However, user may be confused by setting
> > >>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by
> > default
> > >>> in
> > >>>>>>>>>>>>>>>>> custom memory
> > >>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Ivan,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I do not think this is correct approach, because it
> will
> > be
> > >>> hard
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> > >>>>>>>>>>>>>>>>>> "boolean"
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need
> default
> > >>>>>>>>>>>>>>>>>> "persistence
> > >>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> > >>> "persistence
> > >>>>>>>>>>>>>>>>>> enabled"
> > >>>>>>>>>>>>>>>>>> flag for default region only. It should not be
> > propagated
> > >>> to
> > >>>>>>>>>>>>>>>>>> custom
> > >>>>>>>>>>>>>>>>>> regions.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> > >>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com <javascript:;>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Let's check the final design:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> > >>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
> > >>> enabling
> > >>>>>>>>>>>>>>>>>>> persistence
> > >>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>> default data region.
> > >>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have
> > *isPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> property,
> > >>>>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>> will be used for enabling persistence in
> corresponding
> > >>> data
> > >>>>>>>>>>>>>>>>>>> region. If
> > >>>>>>>>>>>>>>>>>>> value is not set, value of
> > *DataStorageConfiguration::isD
> > >>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> will be used by default.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> > >>>>>>>>>>>>>>>>>>> dmagda@apache.org <javascript:;>>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> > >>> ivan.glukos@gmail.com <javascript:;>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag
> in
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> > >>> Persistence
> > >>>>>>>>>>>>>>>>>>>>>> will be
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has
> > this
> > >>>>>>>>>>>>>>>>>>>>>> flag
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> set.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> > >>>>>>>>>>>>>>>>>>>>> *globally*
> > >>>>>>>>>>>>>>>>>>>>> if the
> > >>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If
> > it’s
> > >>>>>>>>>>>>>>>>>>>>> enabled for
> > >>>>>>>>>>>>>>>>>>>>> region
> > >>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for
> region
> > B.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will
> > be
> > >>>>>>>>>>>>>>>>>>>>> disabled
> > >>>>>>>>>>>>>>>>>>>>> globally.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> > >>> default
> > >>>>>>>>>>>>>>>>>>>>> behavior from
> > >>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>
> > >
> >
> >
>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Thu, Oct 12, 2017 at 4:05 PM, Denis Magda <dm...@apache.org> wrote:
> Both terms “persistence” and “storage” mean same to me. If to choose
> between them I would go for “persistence” because this is how
> we name the feature - “Ignite Persistence” [1]
> Furthermore, by default WAL files, the archive and data/index files are
> located under the same root which is “setPersistencePath” directory.
>
Denis, to my knowledge, WAL can span multiple caches, no? How can it end up
in the same folder with data and indexes, if those are per cache?
>
> [1] https://ignite.apache.org/features/persistence.html <
> https://ignite.apache.org/features/persistence.html>
>
> —
> Denis
>
> > On Oct 12, 2017, at 12:00 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Name *setPersistencePath* looks confusing to me because both WAL and
> index/partition files storage provide persistence. That's why we separated
> API methods as *setWalPath* and *setStoragePath*.
> > I think, *setStoragePath* is good enough as long as it's supported by
> explaining javadoc.*
> > *
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> >> Is the storage path the root folder for the persistence or only the root
> >> path for the main storage?
> >>
> >> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
> >>
> >>> Ivan,
> >>>
> >>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
> >>> some extra notes in the ticket.
> >>>
> >>
> >>> —
> >>> Denis
> >>>
> >>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com>
> wrote:
> >>>>
> >>>> Vladimir,
> >>>>
> >>>> Thanks for focusing on existing namings. Most of your suggestions
> really
> >>> sound better. I've posted my thoughts under your comment.
> >>>> By the way, we should decide two things:
> >>>>
> >>>> 1) Naming for methods for configuring store path. I suggest the
> >>> following:
> >>>> *setStoragePath* - for partition and index files
> >>>> *setWalPath* - for WAL files
> >>>> *walArchivePath* - for WAL archive files
> >>>>
> >>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same
> with
> >>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
> >>> sounds ok to me, let's see what community thinks.
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
> >>>>> Ivan,
> >>>>>
> >>>>> I left some comments in the ticket [1], please take a look.
> >>>>>
> >>>>> [1]
> >>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
> >>> focusedCommentId=16200050&page=com.atlassian.jira.
> >>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> >>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
> >>> wrote:
> >>>>>> Igniters,
> >>>>>>
> >>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> >>> enqueued
> >>>>>> for TC run.
> >>>>>> PR: https://github.com/apache/ignite/pull/2828
> >>>>>>
> >>>>>> Everyone interested in new data storage configuration API, please
> pay
> >>>>>> attention and review.
> >>>>>>
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Ivan Rakov
> >>>>>>
> >>>>>>
> >>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> >>>>>>
> >>>>>>> Sounds good to me.
> >>>>>>>
> >>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <ivan.glukos@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Pavel,
> >>>>>>>> Sounds reasonable.
> >>>>>>>> I suggest to include both "data" and "configuration" to make it
> even
> >>> more
> >>>>>>>> obvious:
> >>>>>>>>
> >>>>>>>> set/getDefaultDataRegionConfiguration
> >>>>>>>> set/getDataRegionConfigurations
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Ivan Rakov
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> >>>>>>>>
> >>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
> >>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
> >>>>>>>>> DataRegionConfiguration[] getDataRegions
> >>>>>>>>>
> >>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
> >>>>>>>>> - or -
> >>>>>>>>> defaultDataRegion + getDataRegions
> >>>>>>>>>
> >>>>>>>>> Thoughts?
> >>>>>>>>>
> >>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <
> ivan.glukos@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Denis,
> >>>>>>>>>
> >>>>>>>>>> Yes, you're right. All cache groups without specific data region
> >>>>>>>>>> configured will be persistent.
> >>>>>>>>>> And if you want to add another persistent data region, you
> should
> >>> set
> >>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> >>> explictly.
> >>>>>>>>>> Best Regards,
> >>>>>>>>>> Ivan Rakov
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> >>>>>>>>>>
> >>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to
> wrap
> >>> this
> >>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
> >>> any
> >>>>>>>>>>> regions
> >>>>>>>>>>> configured explicitly I need to take the default region and
> >>> switch the
> >>>>>>>>>>> persistence on for it. Is my understanding correct?
> >>>>>>>>>>>
> >>>>>>>>>>> —
> >>>>>>>>>>> Denis
> >>>>>>>>>>>
> >>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <ivan.glukos@gmail.com
> >
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Denis, why do you need to access an instance of the default
> region
> >>>>>>>>>>>> bean?
> >>>>>>>>>>>> If you want to set any parameter, just instantiate new bean
> with
> >>> this
> >>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters
> will
> >>> be
> >>>>>>>>>>>> automatically initialized with their default values.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> <property name="dataStorageConfiguration">
> >>>>>>>>>>>>
> >>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
> >>>>>>>>>>>>>>> value="#{100
> >>>>>>>>>>>>>>> *
> >>>>>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>>>> <property name="
> defaultRegionConfiguration">
> >>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>>>> <property name="maxSize"
> value="#{5 *
> >>>>>>>>>>>>>>> 1024 *
> >>>>>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>>>> </bean>
> >>>>>>>>>>>>>>> </property>
> >>>>>>>>>>>>>>> </bean>
> >>>>>>>>>>>>>>> </property>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> default.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
> >>> change
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the
> persistence I
> >>>>>>>>>>>>> don’t want
> >>>>>>>>>>>>> to create the default region bean from scratch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <
> ivan.glukos@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Agree with Alexey.
> >>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
> >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
> >>> default
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>> region. They can decide they are inherited by all data
> regions
> >>>>>>>>>>>>>> where
> >>>>>>>>>>>>>> size
> >>>>>>>>>>>>>> and persistence flag are not explicitly set.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Let's get rid of these properties and add
> >>>>>>>>>>>>>> *defaultRegionConfiguration*
> >>>>>>>>>>>>>> property with explicit configuration of default data region.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence
> flag
> >>> of
> >>>>>>>>>>>>>> default data region will be just two lines longer (for bean
> >>>>>>>>>>>>>> description):
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <property name="dataStorageConfiguration">
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
> >>>>>>>>>>>>>>> value="#{100
> >>>>>>>>>>>>>>> *
> >>>>>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>>>> <property name="
> defaultRegionConfiguration">
> >>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>>>> <property name="maxSize"
> value="#{5 *
> >>>>>>>>>>>>>>> 1024 *
> >>>>>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>>>> </bean>
> >>>>>>>>>>>>>>> </property>
> >>>>>>>>>>>>>>> </bean>
> >>>>>>>>>>>>>>> </property>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> default.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> >>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To resolve this, I suggest to
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> >>> get
> >>>>>>>>>>>>>>>> rid
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> >>> file? I’m
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> an expert in Spring so how do I get
> defaultRegionConfiguration
> >>>>>>>>>>>>>>> bean
> >>>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>> to change any parameter?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
> >>> either
> >>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> >>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this
> field
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>> is also
> >>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> >>> defaults
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> >>> suggest
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> and
> >>> get
> >>>>>>>>>>>>>>>> rid
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <
> ivan.glukos@gmail.com
> >>>> :
> >>>>>>>>>>>>>>>> Vladimir,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
> >>>>>>>>>>>>>>>>> However, user may be confused by setting
> >>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by
> default
> >>> in
> >>>>>>>>>>>>>>>>> custom memory
> >>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ivan,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will
> be
> >>> hard
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> >>>>>>>>>>>>>>>>>> "boolean"
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
> >>>>>>>>>>>>>>>>>> "persistence
> >>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> >>> "persistence
> >>>>>>>>>>>>>>>>>> enabled"
> >>>>>>>>>>>>>>>>>> flag for default region only. It should not be
> propagated
> >>> to
> >>>>>>>>>>>>>>>>>> custom
> >>>>>>>>>>>>>>>>>> regions.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> >>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Let's check the final design:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> >>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
> >>> enabling
> >>>>>>>>>>>>>>>>>>> persistence
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> default data region.
> >>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have
> *isPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> property,
> >>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
> >>> data
> >>>>>>>>>>>>>>>>>>> region. If
> >>>>>>>>>>>>>>>>>>> value is not set, value of
> *DataStorageConfiguration::isD
> >>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> will be used by default.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> >>>>>>>>>>>>>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> >>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> >>> Persistence
> >>>>>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has
> this
> >>>>>>>>>>>>>>>>>>>>>> flag
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> set.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> >>>>>>>>>>>>>>>>>>>>> *globally*
> >>>>>>>>>>>>>>>>>>>>> if the
> >>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If
> it’s
> >>>>>>>>>>>>>>>>>>>>> enabled for
> >>>>>>>>>>>>>>>>>>>>> region
> >>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region
> B.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will
> be
> >>>>>>>>>>>>>>>>>>>>> disabled
> >>>>>>>>>>>>>>>>>>>>> globally.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> >>> default
> >>>>>>>>>>>>>>>>>>>>> behavior from
> >>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>
> >
>
>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
Both terms “persistence” and “storage” mean same to me. If to choose between them I would go for “persistence” because this is how
we name the feature - “Ignite Persistence” [1].
Furthermore, by default WAL files, the archive and data/index files are located under the same root which is “setPersistencePath” directory.
[1] https://ignite.apache.org/features/persistence.html <https://ignite.apache.org/features/persistence.html>
—
Denis
> On Oct 12, 2017, at 12:00 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> Name *setPersistencePath* looks confusing to me because both WAL and index/partition files storage provide persistence. That's why we separated API methods as *setWalPath* and *setStoragePath*.
> I think, *setStoragePath* is good enough as long as it's supported by explaining javadoc.*
> *
>
> Best Regards,
> Ivan Rakov
>
> On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
>> Is the storage path the root folder for the persistence or only the root
>> path for the main storage?
>>
>> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
>>
>>> Ivan,
>>>
>>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
>>> some extra notes in the ticket.
>>>
>>
>>> —
>>> Denis
>>>
>>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Vladimir,
>>>>
>>>> Thanks for focusing on existing namings. Most of your suggestions really
>>> sound better. I've posted my thoughts under your comment.
>>>> By the way, we should decide two things:
>>>>
>>>> 1) Naming for methods for configuring store path. I suggest the
>>> following:
>>>> *setStoragePath* - for partition and index files
>>>> *setWalPath* - for WAL files
>>>> *walArchivePath* - for WAL archive files
>>>>
>>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
>>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
>>> sounds ok to me, let's see what community thinks.
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>>>>> Ivan,
>>>>>
>>>>> I left some comments in the ticket [1], please take a look.
>>>>>
>>>>> [1]
>>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
>>> focusedCommentId=16200050&page=com.atlassian.jira.
>>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>>>>> Igniters,
>>>>>>
>>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
>>> enqueued
>>>>>> for TC run.
>>>>>> PR: https://github.com/apache/ignite/pull/2828
>>>>>>
>>>>>> Everyone interested in new data storage configuration API, please pay
>>>>>> attention and review.
>>>>>>
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>>
>>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>>>>
>>>>>>> Sounds good to me.
>>>>>>>
>>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Pavel,
>>>>>>>> Sounds reasonable.
>>>>>>>> I suggest to include both "data" and "configuration" to make it even
>>> more
>>>>>>>> obvious:
>>>>>>>>
>>>>>>>> set/getDefaultDataRegionConfiguration
>>>>>>>> set/getDataRegionConfigurations
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>>
>>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>>>>
>>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>>>>
>>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>>>>> - or -
>>>>>>>>> defaultDataRegion + getDataRegions
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Denis,
>>>>>>>>>
>>>>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>>>>> configured will be persistent.
>>>>>>>>>> And if you want to add another persistent data region, you should
>>> set
>>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
>>> explictly.
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>>>>
>>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
>>> this
>>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
>>> any
>>>>>>>>>>> regions
>>>>>>>>>>> configured explicitly I need to take the default region and
>>> switch the
>>>>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>>>>
>>>>>>>>>>> —
>>>>>>>>>>> Denis
>>>>>>>>>>>
>>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>>>>> bean?
>>>>>>>>>>>> If you want to set any parameter, just instantiate new bean with
>>> this
>>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters will
>>> be
>>>>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>
>>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>>
>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>>> *
>>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
>>> change
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>
>>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>>>>> don’t want
>>>>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>>>>
>>>>>>>>>>>>> —
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
>>> default
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>>>>> where
>>>>>>>>>>>>>> size
>>>>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
>>> of
>>>>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>>>>> description):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>>> *
>>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>>> get
>>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
>>> file? I’m
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>>>>> bean
>>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
>>> either
>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> is also
>>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
>>> defaults
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
>>> suggest
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>>> get
>>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>>>> :
>>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
>>> in
>>>>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will be
>>> hard
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
>>> "persistence
>>>>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>>>>> flag for default region only. It should not be propagated
>>> to
>>>>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
>>> enabling
>>>>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
>>> data
>>>>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
>>> Persistence
>>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
>>> default
>>>>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Name *setPersistencePath* looks confusing to me because both WAL and
index/partition files storage provide persistence. That's why we
separated API methods as *setWalPath* and *setStoragePath*.
I think, *setStoragePath* is good enough as long as it's supported by
explaining javadoc.*
*
Best Regards,
Ivan Rakov
On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> Is the storage path the root folder for the persistence or only the root
> path for the main storage?
>
> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Ivan,
>>
>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
>> some extra notes in the ticket.
>>
>
>> —
>> Denis
>>
>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Vladimir,
>>>
>>> Thanks for focusing on existing namings. Most of your suggestions really
>> sound better. I've posted my thoughts under your comment.
>>> By the way, we should decide two things:
>>>
>>> 1) Naming for methods for configuring store path. I suggest the
>> following:
>>> *setStoragePath* - for partition and index files
>>> *setWalPath* - for WAL files
>>> *walArchivePath* - for WAL archive files
>>>
>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
>> sounds ok to me, let's see what community thinks.
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>>>> Ivan,
>>>>
>>>> I left some comments in the ticket [1], please take a look.
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
>> focusedCommentId=16200050&page=com.atlassian.jira.
>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>>>> Igniters,
>>>>>
>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
>> enqueued
>>>>> for TC run.
>>>>> PR: https://github.com/apache/ignite/pull/2828
>>>>>
>>>>> Everyone interested in new data storage configuration API, please pay
>>>>> attention and review.
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>>>
>>>>>> Sounds good to me.
>>>>>>
>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Pavel,
>>>>>>> Sounds reasonable.
>>>>>>> I suggest to include both "data" and "configuration" to make it even
>> more
>>>>>>> obvious:
>>>>>>>
>>>>>>> set/getDefaultDataRegionConfiguration
>>>>>>> set/getDataRegionConfigurations
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>>>
>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>>>
>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>>>> - or -
>>>>>>>> defaultDataRegion + getDataRegions
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis,
>>>>>>>>
>>>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>>>> configured will be persistent.
>>>>>>>>> And if you want to add another persistent data region, you should
>> set
>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
>> explictly.
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
>> this
>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
>> any
>>>>>>>>>> regions
>>>>>>>>>> configured explicitly I need to take the default region and
>> switch the
>>>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>>>> bean?
>>>>>>>>>>> If you want to set any parameter, just instantiate new bean with
>> this
>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters will
>> be
>>>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>>>
>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>
>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>> *
>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
>> change
>>>>>>>>>>>>> a
>>>>>>>>>>>>>
>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>>>> don’t want
>>>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>>>
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
>> default
>>>>>>>>>>>>> data
>>>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>>>> where
>>>>>>>>>>>>> size
>>>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
>> of
>>>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>>>> description):
>>>>>>>>>>>>>
>>>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>>>
>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>> *
>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>> </bean>
>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>>>
>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>> get
>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
>> file? I’m
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>>>> bean
>>>>>>>>>>>>>> first
>>>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
>> either
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> is also
>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
>> defaults
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
>> suggest
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>> get
>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>>> :
>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
>> in
>>>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will be
>> hard
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
>> "persistence
>>>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>>>> flag for default region only. It should not be propagated
>> to
>>>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
>> enabling
>>>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
>> data
>>>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
>> Persistence
>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
>> default
>>>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Is the storage path the root folder for the persistence or only the root
path for the main storage?
On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
> Ivan,
>
> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
> some extra notes in the ticket.
>
> —
> Denis
>
> > On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Vladimir,
> >
> > Thanks for focusing on existing namings. Most of your suggestions really
> sound better. I've posted my thoughts under your comment.
> >
> > By the way, we should decide two things:
> >
> > 1) Naming for methods for configuring store path. I suggest the
> following:
> >
> > *setStoragePath* - for partition and index files
> > *setWalPath* - for WAL files
> > *walArchivePath* - for WAL archive files
> >
> > 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
> sounds ok to me, let's see what community thinks.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 11.10.2017 14:05, Vladimir Ozerov wrote:
> >> Ivan,
> >>
> >> I left some comments in the ticket [1], please take a look.
> >>
> >> [1]
> >> https://issues.apache.org/jira/browse/IGNITE-6030?
> focusedCommentId=16200050&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> >>
> >> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
> wrote:
> >>
> >>> Igniters,
> >>>
> >>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> enqueued
> >>> for TC run.
> >>> PR: https://github.com/apache/ignite/pull/2828
> >>>
> >>> Everyone interested in new data storage configuration API, please pay
> >>> attention and review.
> >>>
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>>
> >>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> >>>
> >>>> Sounds good to me.
> >>>>
> >>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Pavel,
> >>>>> Sounds reasonable.
> >>>>> I suggest to include both "data" and "configuration" to make it even
> more
> >>>>> obvious:
> >>>>>
> >>>>> set/getDefaultDataRegionConfiguration
> >>>>> set/getDataRegionConfigurations
> >>>>>
> >>>>> Best Regards,
> >>>>> Ivan Rakov
> >>>>>
> >>>>>
> >>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> >>>>>
> >>>>> Sorry that I'm late to the party, but this looks inconsistent:
> >>>>>> DataStorageConfiguration defaultRegionConfiguration
> >>>>>> DataRegionConfiguration[] getDataRegions
> >>>>>>
> >>>>>> defaultRegionConfiguration + getRegionConfigurations
> >>>>>> - or -
> >>>>>> defaultDataRegion + getDataRegions
> >>>>>>
> >>>>>> Thoughts?
> >>>>>>
> >>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Denis,
> >>>>>>
> >>>>>>> Yes, you're right. All cache groups without specific data region
> >>>>>>> configured will be persistent.
> >>>>>>> And if you want to add another persistent data region, you should
> set
> >>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> explictly.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>>
> >>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> >>>>>>>
> >>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
> this
> >>>>>>>
> >>>>>>>> up, if I want to enable the persistence globally and don’t have
> any
> >>>>>>>> regions
> >>>>>>>> configured explicitly I need to take the default region and
> switch the
> >>>>>>>> persistence on for it. Is my understanding correct?
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> Denis
> >>>>>>>>
> >>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Denis, why do you need to access an instance of the default region
> >>>>>>>>> bean?
> >>>>>>>>> If you want to set any parameter, just instantiate new bean with
> this
> >>>>>>>>> parameter set (like in XML snipped below). Other parameters will
> be
> >>>>>>>>> automatically initialized with their default values.
> >>>>>>>>>
> >>>>>>>>> Best Regards,
> >>>>>>>>> Ivan Rakov
> >>>>>>>>>
> >>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> >>>>>>>>>
> >>>>>>>>> <property name="dataStorageConfiguration">
> >>>>>>>>>
> >>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>> <property name="systemCacheInitialSize"
> >>>>>>>>>>>> value="#{100
> >>>>>>>>>>>> *
> >>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>> <property name="defaultRegionConfiguration">
> >>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>> <property name="maxSize" value="#{5 *
> >>>>>>>>>>>> 1024 *
> >>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>> </bean>
> >>>>>>>>>>>> </property>
> >>>>>>>>>>>> </bean>
> >>>>>>>>>>>> </property>
> >>>>>>>>>>>>
> >>>>>>>>>>>> In other data regions persistence will be disabled by default.
> >>>>>>>>>>>>
> >>>>>>>>>>> Ivan, how to get an instance to the default region bean and
> change
> >>>>>>>>>>> a
> >>>>>>>>>>>
> >>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
> >>>>>>>>>> don’t want
> >>>>>>>>>> to create the default region bean from scratch.
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Agree with Alexey.
> >>>>>>>>>>> Properties like *defaultDataRegionSize*,
> >>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>> can confuse users who don't know that there's such thing as
> default
> >>>>>>>>>>> data
> >>>>>>>>>>> region. They can decide they are inherited by all data regions
> >>>>>>>>>>> where
> >>>>>>>>>>> size
> >>>>>>>>>>> and persistence flag are not explicitly set.
> >>>>>>>>>>>
> >>>>>>>>>>> Let's get rid of these properties and add
> >>>>>>>>>>> *defaultRegionConfiguration*
> >>>>>>>>>>> property with explicit configuration of default data region.
> >>>>>>>>>>>
> >>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
> of
> >>>>>>>>>>> default data region will be just two lines longer (for bean
> >>>>>>>>>>> description):
> >>>>>>>>>>>
> >>>>>>>>>>> <property name="dataStorageConfiguration">
> >>>>>>>>>>>
> >>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>> <property name="systemCacheInitialSize"
> >>>>>>>>>>>> value="#{100
> >>>>>>>>>>>> *
> >>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>> <property name="defaultRegionConfiguration">
> >>>>>>>>>>>> <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>> <property name="maxSize" value="#{5 *
> >>>>>>>>>>>> 1024 *
> >>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>> </bean>
> >>>>>>>>>>>> </property>
> >>>>>>>>>>>> </bean>
> >>>>>>>>>>>> </property>
> >>>>>>>>>>>>
> >>>>>>>>>>>> In other data regions persistence will be disabled by default.
> >>>>>>>>>>>>
> >>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> >>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>
> >>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> To resolve this, I suggest to
> >>>>>>>>>>>
> >>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> get
> >>>>>>>>>>>>> rid
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> file? I’m
> >>>>>>>>>>>>>
> >>>>>>>>>>>> not
> >>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
> >>>>>>>>>>>> bean
> >>>>>>>>>>>> first
> >>>>>>>>>>>> to change any parameter?
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> >>>>>>>>>>>>
> >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
> either
> >>>>>>>>>>>>> need to
> >>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> >>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
> >>>>>>>>>>>>> which
> >>>>>>>>>>>>> is also
> >>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> defaults
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> suggest
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> get
> >>>>>>>>>>>>> rid
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> >:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Vladimir,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I like your approach because it's easier to implement.
> >>>>>>>>>>>>>> However, user may be confused by setting
> >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
> in
> >>>>>>>>>>>>>> custom memory
> >>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ivan,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I do not think this is correct approach, because it will be
> hard
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> >>>>>>>>>>>>>>> "boolean"
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
> >>>>>>>>>>>>>>> "persistence
> >>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> "persistence
> >>>>>>>>>>>>>>> enabled"
> >>>>>>>>>>>>>>> flag for default region only. It should not be propagated
> to
> >>>>>>>>>>>>>>> custom
> >>>>>>>>>>>>>>> regions.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> >>>>>>>>>>>>>>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Guys, I think I got the point now.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let's check the final design:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> >>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>> property (default = false), which will be used for
> enabling
> >>>>>>>>>>>>>>>> persistence
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> default data region.
> >>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
> >>>>>>>>>>>>>>>> property,
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
> data
> >>>>>>>>>>>>>>>> region. If
> >>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
> >>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> >>>>>>>>>>>>>>>> will be used by default.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> >>>>>>>>>>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> Persistence
> >>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
> >>>>>>>>>>>>>>>>>>> flag
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> set.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> >>>>>>>>>>>>>>>>>> *globally*
> >>>>>>>>>>>>>>>>>> if the
> >>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
> >>>>>>>>>>>>>>>>>> enabled for
> >>>>>>>>>>>>>>>>>> region
> >>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
> >>>>>>>>>>>>>>>>>> disabled
> >>>>>>>>>>>>>>>>>> globally.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> default
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> behavior from
> >>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >
>
>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
Ivan,
Instead of “setStoragePath” I would suggest “setPersistencePath”. Left some extra notes in the ticket.
—
Denis
> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> Vladimir,
>
> Thanks for focusing on existing namings. Most of your suggestions really sound better. I've posted my thoughts under your comment.
>
> By the way, we should decide two things:
>
> 1) Naming for methods for configuring store path. I suggest the following:
>
> *setStoragePath* - for partition and index files
> *setWalPath* - for WAL files
> *walArchivePath* - for WAL archive files
>
> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with *checkpointingPageBufferSize* and *checkpointingThreads*). Both options sounds ok to me, let's see what community thinks.
>
> Best Regards,
> Ivan Rakov
>
> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>> Ivan,
>>
>> I left some comments in the ticket [1], please take a look.
>>
>> [1]
>> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>
>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>>> Igniters,
>>>
>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>>> for TC run.
>>> PR: https://github.com/apache/ignite/pull/2828
>>>
>>> Everyone interested in new data storage configuration API, please pay
>>> attention and review.
>>>
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>
>>>> Sounds good to me.
>>>>
>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Pavel,
>>>>> Sounds reasonable.
>>>>> I suggest to include both "data" and "configuration" to make it even more
>>>>> obvious:
>>>>>
>>>>> set/getDefaultDataRegionConfiguration
>>>>> set/getDataRegionConfigurations
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>
>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>
>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>> - or -
>>>>>> defaultDataRegion + getDataRegions
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Denis,
>>>>>>
>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>> configured will be persistent.
>>>>>>> And if you want to add another persistent data region, you should set
>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>
>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>>>
>>>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>>>> regions
>>>>>>>> configured explicitly I need to take the default region and switch the
>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>> bean?
>>>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>
>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>>>> a
>>>>>>>>>>>
>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>> don’t want
>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>>>> data
>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>> where
>>>>>>>>>>> size
>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>
>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>
>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>> description):
>>>>>>>>>>>
>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>
>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>
>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>> bean
>>>>>>>>>>>> first
>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>> which
>>>>>>>>>>>>> is also
>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>>>> we
>>>>>>>>>>>>> are
>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>>>> to
>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Typo in my previous reply: *walArchivePath* -> *setWalArchivePath*
Best Regards,
Ivan Rakov
On 11.10.2017 14:30, Ivan Rakov wrote:
> Vladimir,
>
> Thanks for focusing on existing namings. Most of your suggestions
> really sound better. I've posted my thoughts under your comment.
>
> By the way, we should decide two things:
>
> 1) Naming for methods for configuring store path. I suggest the
> following:
>
> *setStoragePath* - for partition and index files
> *setWalPath* - for WAL files
> *walArchivePath* - for WAL archive files
>
> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same
> with *checkpointingPageBufferSize* and *checkpointingThreads*). Both
> options sounds ok to me, let's see what community thinks.
>
> Best Regards,
> Ivan Rakov
>
> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>> Ivan,
>>
>> I left some comments in the ticket [1], please take a look.
>>
>> [1]
>> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>
>>
>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>
>>> Igniters,
>>>
>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>>> for TC run.
>>> PR: https://github.com/apache/ignite/pull/2828
>>>
>>> Everyone interested in new data storage configuration API, please pay
>>> attention and review.
>>>
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>
>>>> Sounds good to me.
>>>>
>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Pavel,
>>>>> Sounds reasonable.
>>>>> I suggest to include both "data" and "configuration" to make it
>>>>> even more
>>>>> obvious:
>>>>>
>>>>> set/getDefaultDataRegionConfiguration
>>>>> set/getDataRegionConfigurations
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>
>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>
>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>> - or -
>>>>>> defaultDataRegion + getDataRegions
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Denis,
>>>>>>
>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>> configured will be persistent.
>>>>>>> And if you want to add another persistent data region, you
>>>>>>> should set
>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
>>>>>>> explictly.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>
>>>>>>> Missed the point with defaults. Makes sense to me now. So to
>>>>>>> wrap this
>>>>>>>
>>>>>>>> up, if I want to enable the persistence globally and don’t have
>>>>>>>> any
>>>>>>>> regions
>>>>>>>> configured explicitly I need to take the default region and
>>>>>>>> switch the
>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>> bean?
>>>>>>>>> If you want to set any parameter, just instantiate new bean
>>>>>>>>> with this
>>>>>>>>> parameter set (like in XML snipped below). Other parameters
>>>>>>>>> will be
>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>
>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
>>>>>>>>>>> change
>>>>>>>>>>> a
>>>>>>>>>>>
>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>> don’t want
>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> can confuse users who don't know that there's such thing as
>>>>>>>>>>> default
>>>>>>>>>>> data
>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>> where
>>>>>>>>>>> size
>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>
>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>
>>>>>>>>>>> Regarding XML configuration, changing size or persistence
>>>>>>>>>>> flag of
>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>> description):
>>>>>>>>>>>
>>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>>
>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>> </bean>
>>>>>>>>>>>> </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>
>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>>>>>>>>>>>> get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
>>>>>>>>>>>>> file? I’m
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>> bean
>>>>>>>>>>>> first
>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
>>>>>>>>>>>>> either
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>> which
>>>>>>>>>>>>> is also
>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
>>>>>>>>>>>>> defaults
>>>>>>>>>>>>> we
>>>>>>>>>>>>> are
>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
>>>>>>>>>>>>> suggest
>>>>>>>>>>>>> to
>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
>>>>>>>>>>>>> and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov
>>>>>>>>>>>>> <iv...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by
>>>>>>>>>>>>>> default in
>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do not think this is correct approach, because it will
>>>>>>>>>>>>>> be hard
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>> flag for default region only. It should not be
>>>>>>>>>>>>>>> propagated to
>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> property (default = false), which will be used for
>>>>>>>>>>>>>>>> enabling
>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov
>>>>>>>>>>>>>>>> <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
>>>>>>>>>>>>>>>>> Persistence
>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
>>>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Vladimir,
Thanks for focusing on existing namings. Most of your suggestions really
sound better. I've posted my thoughts under your comment.
By the way, we should decide two things:
1) Naming for methods for configuring store path. I suggest the following:
*setStoragePath* - for partition and index files
*setWalPath* - for WAL files
*walArchivePath* - for WAL archive files
2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
*checkpointingPageBufferSize* and *checkpointingThreads*). Both options
sounds ok to me, let's see what community thinks.
Best Regards,
Ivan Rakov
On 11.10.2017 14:05, Vladimir Ozerov wrote:
> Ivan,
>
> I left some comments in the ticket [1], please take a look.
>
> [1]
> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>
> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Igniters,
>>
>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>> for TC run.
>> PR: https://github.com/apache/ignite/pull/2828
>>
>> Everyone interested in new data storage configuration API, please pay
>> attention and review.
>>
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>
>>> Sounds good to me.
>>>
>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>>
>>> Pavel,
>>>> Sounds reasonable.
>>>> I suggest to include both "data" and "configuration" to make it even more
>>>> obvious:
>>>>
>>>> set/getDefaultDataRegionConfiguration
>>>> set/getDataRegionConfigurations
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>
>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>> DataRegionConfiguration[] getDataRegions
>>>>>
>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>> - or -
>>>>> defaultDataRegion + getDataRegions
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Denis,
>>>>>
>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>> configured will be persistent.
>>>>>> And if you want to add another persistent data region, you should set
>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>>
>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>
>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>>
>>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>>> regions
>>>>>>> configured explicitly I need to take the default region and switch the
>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>> bean?
>>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>>> automatically initialized with their default values.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>
>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>
>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>> value="#{100
>>>>>>>>>>> *
>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>> 1024 *
>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>> </bean>
>>>>>>>>>>> </property>
>>>>>>>>>>> </bean>
>>>>>>>>>>> </property>
>>>>>>>>>>>
>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>
>>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>>> a
>>>>>>>>>>
>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>> don’t want
>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Agree with Alexey.
>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>>> data
>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>> where
>>>>>>>>>> size
>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>
>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>
>>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>> description):
>>>>>>>>>>
>>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>>
>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>>> value="#{100
>>>>>>>>>>> *
>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>>> 1024 *
>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>> </bean>
>>>>>>>>>>> </property>
>>>>>>>>>>> </bean>
>>>>>>>>>>> </property>
>>>>>>>>>>>
>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>
>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>
>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>
>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>> rid
>>>>>>>>>>>> of
>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>
>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>>
>>>>>>>>>>> not
>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>> bean
>>>>>>>>>>> first
>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>
>>>>>>>>>>> —
>>>>>>>>>>> Denis
>>>>>>>>>>>
>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>
>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>>> need to
>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>> which
>>>>>>>>>>>> is also
>>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>>> we
>>>>>>>>>>>> are
>>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>>> to
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>> rid
>>>>>>>>>>>> of
>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>
>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
Re: Persistence per memory policy configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Ivan,
I left some comments in the ticket [1], please take a look.
[1]
https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:
> Igniters,
>
> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
> for TC run.
> PR: https://github.com/apache/ignite/pull/2828
>
> Everyone interested in new data storage configuration API, please pay
> attention and review.
>
>
> Best Regards,
> Ivan Rakov
>
>
> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>
>> Sounds good to me.
>>
>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>
>> Pavel,
>>>
>>> Sounds reasonable.
>>> I suggest to include both "data" and "configuration" to make it even more
>>> obvious:
>>>
>>> set/getDefaultDataRegionConfiguration
>>> set/getDataRegionConfigurations
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>
>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>
>>>> DataStorageConfiguration defaultRegionConfiguration
>>>> DataRegionConfiguration[] getDataRegions
>>>>
>>>> defaultRegionConfiguration + getRegionConfigurations
>>>> - or -
>>>> defaultDataRegion + getDataRegions
>>>>
>>>> Thoughts?
>>>>
>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Denis,
>>>>
>>>>> Yes, you're right. All cache groups without specific data region
>>>>> configured will be persistent.
>>>>> And if you want to add another persistent data region, you should set
>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>
>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>
>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>> regions
>>>>>> configured explicitly I need to take the default region and switch the
>>>>>> persistence on for it. Is my understanding correct?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>> bean?
>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>> automatically initialized with their default values.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>
>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>
>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>
>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>> value="#{100
>>>>>>>>>> *
>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>> 1024 *
>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>> </bean>
>>>>>>>>>> </property>
>>>>>>>>>> </bean>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>
>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>> a
>>>>>>>>>
>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>> don’t want
>>>>>>>> to create the default region bean from scratch.
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Agree with Alexey.
>>>>>>>>>
>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>> data
>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>> where
>>>>>>>>> size
>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>
>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>
>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>> description):
>>>>>>>>>
>>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>>
>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>> <property name="systemCacheInitialSize"
>>>>>>>>>> value="#{100
>>>>>>>>>> *
>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>> <property name="maxSize" value="#{5 *
>>>>>>>>>> 1024 *
>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>> </bean>
>>>>>>>>>> </property>
>>>>>>>>>> </bean>
>>>>>>>>>> </property>
>>>>>>>>>>
>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>
>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>> rid
>>>>>>>>>>> of
>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>
>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>> bean
>>>>>>>>>> first
>>>>>>>>>> to change any parameter?
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>
>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>> need to
>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>> which
>>>>>>>>>>> is also
>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>> we
>>>>>>>>>>> are
>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>> to
>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>> rid
>>>>>>>>>>> of
>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>
>>>>>>>>>>> Thoughts?
>>>>>>>>>>>
>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>> Vladimir,
>>>>>>>>>>>
>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>
>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>> custom memory
>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>
>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>> to
>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>> for
>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>> custom
>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Igniters,
https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
for TC run.
PR: https://github.com/apache/ignite/pull/2828
Everyone interested in new data storage configuration API, please pay
attention and review.
Best Regards,
Ivan Rakov
On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> Sounds good to me.
>
> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Pavel,
>>
>> Sounds reasonable.
>> I suggest to include both "data" and "configuration" to make it even more
>> obvious:
>>
>> set/getDefaultDataRegionConfiguration
>> set/getDataRegionConfigurations
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>
>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>
>>> DataStorageConfiguration defaultRegionConfiguration
>>> DataRegionConfiguration[] getDataRegions
>>>
>>> defaultRegionConfiguration + getRegionConfigurations
>>> - or -
>>> defaultDataRegion + getDataRegions
>>>
>>> Thoughts?
>>>
>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis,
>>>> Yes, you're right. All cache groups without specific data region
>>>> configured will be persistent.
>>>> And if you want to add another persistent data region, you should set
>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>
>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>> regions
>>>>> configured explicitly I need to take the default region and switch the
>>>>> persistence on for it. Is my understanding correct?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>
>>>>>> Denis, why do you need to access an instance of the default region
>>>>>> bean?
>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>> automatically initialized with their default values.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>
>>>>>> <property name="dataStorageConfiguration">
>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>> <property name="systemCacheInitialSize" value="#{100
>>>>>>>>> *
>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>>>> 102 * 1024}"/>
>>>>>>>>> </bean>
>>>>>>>>> </property>
>>>>>>>>> </bean>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>> don’t want
>>>>>>> to create the default region bean from scratch.
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Agree with Alexey.
>>>>>>>>
>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>> data
>>>>>>>> region. They can decide they are inherited by all data regions where
>>>>>>>> size
>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>
>>>>>>>> Let's get rid of these properties and add
>>>>>>>> *defaultRegionConfiguration*
>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>
>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>> description):
>>>>>>>>
>>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>>
>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>> <property name="systemCacheInitialSize" value="#{100
>>>>>>>>> *
>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>>>> 102 * 1024}"/>
>>>>>>>>> </bean>
>>>>>>>>> </property>
>>>>>>>>> </bean>
>>>>>>>>> </property>
>>>>>>>>>
>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>
>>>>>>>> To resolve this, I suggest to
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>>> of
>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>
>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>> not
>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean
>>>>>>>>> first
>>>>>>>>> to change any parameter?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>
>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>> need to
>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>>>> is also
>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>>>> are
>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>>> of
>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>
>>>>>>>>>> Thoughts?
>>>>>>>>>>
>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Vladimir,
>>>>>>>>>>
>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>
>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>> custom memory
>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>
>>>>>>>>>>> Ivan,
>>>>>>>>>>>
>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>> to
>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>>>> for
>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>> "persistence
>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>> enabled"
>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>> custom
>>>>>>>>>>>> regions.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>
>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>> persistence
>>>>>>>>>>>>> in
>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>> property,
>>>>>>>>>>>>> which
>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>> region. If
>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
Re: Persistence per memory policy configuration
Posted by Pavel Tupitsyn <pt...@apache.org>.
Sounds good to me.
On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com> wrote:
> Pavel,
>
> Sounds reasonable.
> I suggest to include both "data" and "configuration" to make it even more
> obvious:
>
> set/getDefaultDataRegionConfiguration
> set/getDataRegionConfigurations
>
> Best Regards,
> Ivan Rakov
>
>
> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>
>> Sorry that I'm late to the party, but this looks inconsistent:
>>
>> DataStorageConfiguration defaultRegionConfiguration
>> DataRegionConfiguration[] getDataRegions
>>
>> defaultRegionConfiguration + getRegionConfigurations
>> - or -
>> defaultDataRegion + getDataRegions
>>
>> Thoughts?
>>
>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Denis,
>>>
>>> Yes, you're right. All cache groups without specific data region
>>> configured will be persistent.
>>> And if you want to add another persistent data region, you should set
>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>
>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>> up, if I want to enable the persistence globally and don’t have any
>>>> regions
>>>> configured explicitly I need to take the default region and switch the
>>>> persistence on for it. Is my understanding correct?
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>>> Denis, why do you need to access an instance of the default region
>>>>> bean?
>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>> automatically initialized with their default values.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>
>>>>> <property name="dataStorageConfiguration">
>>>>>>
>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>> <property name="systemCacheInitialSize" value="#{100
>>>>>>>> *
>>>>>>>> 1024 * 1024}"/>
>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>>> 102 * 1024}"/>
>>>>>>>> </bean>
>>>>>>>> </property>
>>>>>>>> </bean>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>
>>>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>> don’t want
>>>>>> to create the default region bean from scratch.
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>
>>>>>>> Agree with Alexey.
>>>>>>>
>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>> data
>>>>>>> region. They can decide they are inherited by all data regions where
>>>>>>> size
>>>>>>> and persistence flag are not explicitly set.
>>>>>>>
>>>>>>> Let's get rid of these properties and add
>>>>>>> *defaultRegionConfiguration*
>>>>>>> property with explicit configuration of default data region.
>>>>>>>
>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>> default data region will be just two lines longer (for bean
>>>>>>> description):
>>>>>>>
>>>>>>> <property name="dataStorageConfiguration">
>>>>>>>
>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>> <property name="systemCacheInitialSize" value="#{100
>>>>>>>> *
>>>>>>>> 1024 * 1024}"/>
>>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>>> 102 * 1024}"/>
>>>>>>>> </bean>
>>>>>>>> </property>
>>>>>>>> </bean>
>>>>>>>> </property>
>>>>>>>>
>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>
>>>>>>> To resolve this, I suggest to
>>>>>>>>
>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>> of
>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>
>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>> not
>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean
>>>>>>>> first
>>>>>>>> to change any parameter?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>
>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>> need to
>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>>> is also
>>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>>> are
>>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>> of
>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Vladimir,
>>>>>>>>>
>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>
>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>> custom memory
>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>
>>>>>>>>>> Ivan,
>>>>>>>>>>
>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>> to
>>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>>> for
>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>> "persistence
>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>> enabled"
>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>> custom
>>>>>>>>>>> regions.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>
>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>
>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>> persistence
>>>>>>>>>>>> in
>>>>>>>>>>>> default data region.
>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>> property,
>>>>>>>>>>>> which
>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>> region. If
>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>
>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>> region
>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>
>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Pavel,
Sounds reasonable.
I suggest to include both "data" and "configuration" to make it even
more obvious:
set/getDefaultDataRegionConfiguration
set/getDataRegionConfigurations
Best Regards,
Ivan Rakov
On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> Sorry that I'm late to the party, but this looks inconsistent:
>
> DataStorageConfiguration defaultRegionConfiguration
> DataRegionConfiguration[] getDataRegions
>
> defaultRegionConfiguration + getRegionConfigurations
> - or -
> defaultDataRegion + getDataRegions
>
> Thoughts?
>
> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Denis,
>>
>> Yes, you're right. All cache groups without specific data region
>> configured will be persistent.
>> And if you want to add another persistent data region, you should set
>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 02.10.2017 21:01, Denis Magda wrote:
>>
>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>> up, if I want to enable the persistence globally and don’t have any regions
>>> configured explicitly I need to take the default region and switch the
>>> persistence on for it. Is my understanding correct?
>>>
>>> —
>>> Denis
>>>
>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>> Denis, why do you need to access an instance of the default region bean?
>>>> If you want to set any parameter, just instantiate new bean with this
>>>> parameter set (like in XML snipped below). Other parameters will be
>>>> automatically initialized with their default values.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>
>>>>> <property name="dataStorageConfiguration">
>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>> guration.DataStorageConfiguration">
>>>>>>> <property name="systemCacheInitialSize" value="#{100 *
>>>>>>> 1024 * 1024}"/>
>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>> guration.DataRegionConfiguration">
>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>> 102 * 1024}"/>
>>>>>>> </bean>
>>>>>>> </property>
>>>>>>> </bean>
>>>>>>> </property>
>>>>>>>
>>>>>> In other data regions persistence will be disabled by default.
>>>>>>
>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>> parameter? Obviously, if the goal is to enable the persistence I don’t want
>>>>> to create the default region bean from scratch.
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>> Agree with Alexey.
>>>>>>
>>>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled*
>>>>>> can confuse users who don't know that there's such thing as default data
>>>>>> region. They can decide they are inherited by all data regions where size
>>>>>> and persistence flag are not explicitly set.
>>>>>>
>>>>>> Let's get rid of these properties and add *defaultRegionConfiguration*
>>>>>> property with explicit configuration of default data region.
>>>>>>
>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>> default data region will be just two lines longer (for bean description):
>>>>>>
>>>>>> <property name="dataStorageConfiguration">
>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>> guration.DataStorageConfiguration">
>>>>>>> <property name="systemCacheInitialSize" value="#{100 *
>>>>>>> 1024 * 1024}"/>
>>>>>>> <property name="defaultRegionConfiguration">
>>>>>>> <bean class="org.apache.ignite.confi
>>>>>>> guration.DataRegionConfiguration">
>>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>>> 102 * 1024}"/>
>>>>>>> </bean>
>>>>>>> </property>
>>>>>>> </bean>
>>>>>>> </property>
>>>>>>>
>>>>>> In other data regions persistence will be disabled by default.
>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>
>>>>>>> To resolve this, I suggest to
>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>> of
>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>
>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not
>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean first
>>>>>>> to change any parameter?
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>> need to
>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>> is also
>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>> are
>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>> of
>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>
>>>>>>>> Vladimir,
>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>
>>>>>>>>> However, user may be confused by setting
>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>> custom memory
>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>
>>>>>>>>> Ivan,
>>>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>> for
>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>> "persistence
>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>> enabled"
>>>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>>>> regions.
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>
>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>
>>>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>> persistence
>>>>>>>>>>> in
>>>>>>>>>>> default data region.
>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>> property,
>>>>>>>>>>> which
>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>> region. If
>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>> will be used by default.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>
>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>> set.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>> if the
>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>> region
>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>> behavior from
>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
Re: Persistence per memory policy configuration
Posted by Pavel Tupitsyn <pt...@apache.org>.
Sorry that I'm late to the party, but this looks inconsistent:
DataStorageConfiguration defaultRegionConfiguration
DataRegionConfiguration[] getDataRegions
defaultRegionConfiguration + getRegionConfigurations
- or -
defaultDataRegion + getDataRegions
Thoughts?
On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
> Denis,
>
> Yes, you're right. All cache groups without specific data region
> configured will be persistent.
> And if you want to add another persistent data region, you should set
> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>
> Best Regards,
> Ivan Rakov
>
>
> On 02.10.2017 21:01, Denis Magda wrote:
>
>> Missed the point with defaults. Makes sense to me now. So to wrap this
>> up, if I want to enable the persistence globally and don’t have any regions
>> configured explicitly I need to take the default region and switch the
>> persistence on for it. Is my understanding correct?
>>
>> —
>> Denis
>>
>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis, why do you need to access an instance of the default region bean?
>>> If you want to set any parameter, just instantiate new bean with this
>>> parameter set (like in XML snipped below). Other parameters will be
>>> automatically initialized with their default values.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>
>>>> <property name="dataStorageConfiguration">
>>>>>> <bean class="org.apache.ignite.confi
>>>>>> guration.DataStorageConfiguration">
>>>>>> <property name="systemCacheInitialSize" value="#{100 *
>>>>>> 1024 * 1024}"/>
>>>>>> <property name="defaultRegionConfiguration">
>>>>>> <bean class="org.apache.ignite.confi
>>>>>> guration.DataRegionConfiguration">
>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>> 102 * 1024}"/>
>>>>>> </bean>
>>>>>> </property>
>>>>>> </bean>
>>>>>> </property>
>>>>>>
>>>>> In other data regions persistence will be disabled by default.
>>>>>
>>>> Ivan, how to get an instance to the default region bean and change a
>>>> parameter? Obviously, if the goal is to enable the persistence I don’t want
>>>> to create the default region bean from scratch.
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>
>>>>> Agree with Alexey.
>>>>>
>>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled*
>>>>> can confuse users who don't know that there's such thing as default data
>>>>> region. They can decide they are inherited by all data regions where size
>>>>> and persistence flag are not explicitly set.
>>>>>
>>>>> Let's get rid of these properties and add *defaultRegionConfiguration*
>>>>> property with explicit configuration of default data region.
>>>>>
>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>> default data region will be just two lines longer (for bean description):
>>>>>
>>>>> <property name="dataStorageConfiguration">
>>>>>> <bean class="org.apache.ignite.confi
>>>>>> guration.DataStorageConfiguration">
>>>>>> <property name="systemCacheInitialSize" value="#{100 *
>>>>>> 1024 * 1024}"/>
>>>>>> <property name="defaultRegionConfiguration">
>>>>>> <bean class="org.apache.ignite.confi
>>>>>> guration.DataRegionConfiguration">
>>>>>> <property name="maxSize" value="#{5 * 1024 *
>>>>>> 102 * 1024}"/>
>>>>>> </bean>
>>>>>> </property>
>>>>>> </bean>
>>>>>> </property>
>>>>>>
>>>>> In other data regions persistence will be disabled by default.
>>>>> I've updated draft in https://issues.apache.org/jira
>>>>> /browse/IGNITE-6030 with these changes.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>
>>>>>> To resolve this, I suggest to
>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>> of
>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>
>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not
>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean first
>>>>>> to change any parameter?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>
>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>> need to
>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>> is also
>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>> are
>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>> of
>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>
>>>>>>> Vladimir,
>>>>>>>>
>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>
>>>>>>>> However, user may be confused by setting
>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>> custom memory
>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>
>>>>>>>> Ivan,
>>>>>>>>>
>>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>> for
>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>> "persistence
>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>> enabled"
>>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>>> regions.
>>>>>>>>>
>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>> >
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>
>>>>>>>>>> Let's check the final design:
>>>>>>>>>>
>>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>> persistence
>>>>>>>>>> in
>>>>>>>>>> default data region.
>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>> property,
>>>>>>>>>> which
>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>> region. If
>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>> will be used by default.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>
>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>> will be
>>>>>>>>>>>>>
>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>> set.
>>>>>>>>>>>>
>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>> if the
>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>> enabled for
>>>>>>>>>>>> region
>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>>> globally.
>>>>>>>>>>>>
>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>> behavior from
>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Denis,
Yes, you're right. All cache groups without specific data region
configured will be persistent.
And if you want to add another persistent data region, you should set
*isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
Best Regards,
Ivan Rakov
On 02.10.2017 21:01, Denis Magda wrote:
> Missed the point with defaults. Makes sense to me now. So to wrap this up, if I want to enable the persistence globally and don’t have any regions configured explicitly I need to take the default region and switch the persistence on for it. Is my understanding correct?
>
> —
> Denis
>
>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Denis, why do you need to access an instance of the default region bean? If you want to set any parameter, just instantiate new bean with this parameter set (like in XML snipped below). Other parameters will be automatically initialized with their default values.
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>> <property name="dataStorageConfiguration">
>>>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>> <property name="defaultRegionConfiguration">
>>>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>> </bean>
>>>>> </property>
>>>>> </bean>
>>>>> </property>
>>>> In other data regions persistence will be disabled by default.
>>> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>>>
>>> —
>>> Denis
>>>
>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Agree with Alexey.
>>>>
>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>>>
>>>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>>>
>>>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>>>
>>>>> <property name="dataStorageConfiguration">
>>>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>> <property name="defaultRegionConfiguration">
>>>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>> </bean>
>>>>> </property>
>>>>> </bean>
>>>>> </property>
>>>> In other data regions persistence will be disabled by default.
>>>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>> To resolve this, I suggest to
>>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>>> other defaults in DataStorageConfiguration.
>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>>>
>>>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>>> other defaults in DataStorageConfiguration.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>
>>>>>>> Vladimir,
>>>>>>>
>>>>>>> I like your approach because it's easier to implement.
>>>>>>>
>>>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>
>>>>>>>> Ivan,
>>>>>>>>
>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>> regions.
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Guys, I think I got the point now.
>>>>>>>>> Let's check the final design:
>>>>>>>>>
>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>>>> in
>>>>>>>>> default data region.
>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>>>> which
>>>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>> will be used by default.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>>>
>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>>>> region
>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>
>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>> globally.
>>>>>>>>>>>
>>>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
Missed the point with defaults. Makes sense to me now. So to wrap this up, if I want to enable the persistence globally and don’t have any regions configured explicitly I need to take the default region and switch the persistence on for it. Is my understanding correct?
—
Denis
> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> Denis, why do you need to access an instance of the default region bean? If you want to set any parameter, just instantiate new bean with this parameter set (like in XML snipped below). Other parameters will be automatically initialized with their default values.
>
> Best Regards,
> Ivan Rakov
>
> On 02.10.2017 19:28, Denis Magda wrote:
>>>> <property name="dataStorageConfiguration">
>>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>> <property name="defaultRegionConfiguration">
>>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>> </bean>
>>>> </property>
>>>> </bean>
>>>> </property>
>>> In other data regions persistence will be disabled by default.
>> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>>
>> —
>> Denis
>>
>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Agree with Alexey.
>>>
>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>>
>>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>>
>>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>>
>>>> <property name="dataStorageConfiguration">
>>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>> <property name="defaultRegionConfiguration">
>>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>> </bean>
>>>> </property>
>>>> </bean>
>>>> </property>
>>> In other data regions persistence will be disabled by default.
>>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>> To resolve this, I suggest to
>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>> other defaults in DataStorageConfiguration.
>>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>>
>>>> —
>>>> Denis
>>>>
>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>>
>>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>> other defaults in DataStorageConfiguration.
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>
>>>>>> Vladimir,
>>>>>>
>>>>>> I like your approach because it's easier to implement.
>>>>>>
>>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>>
>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>
>>>>>>> Ivan,
>>>>>>>
>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>> regions.
>>>>>>>
>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Guys, I think I got the point now.
>>>>>>>> Let's check the final design:
>>>>>>>>
>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>>> in
>>>>>>>> default data region.
>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>>> which
>>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>> efaultPersistenceEnabled*
>>>>>>>> will be used by default.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>>
>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>>> region
>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>
>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>> globally.
>>>>>>>>>>
>>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Denis, why do you need to access an instance of the default region bean?
If you want to set any parameter, just instantiate new bean with this
parameter set (like in XML snipped below). Other parameters will be
automatically initialized with their default values.
Best Regards,
Ivan Rakov
On 02.10.2017 19:28, Denis Magda wrote:
>>> <property name="dataStorageConfiguration">
>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>> <property name="defaultRegionConfiguration">
>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>> </bean>
>>> </property>
>>> </bean>
>>> </property>
>> In other data regions persistence will be disabled by default.
> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>
> —
> Denis
>
>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Agree with Alexey.
>>
>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>
>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>
>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>
>>> <property name="dataStorageConfiguration">
>>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>> <property name="defaultRegionConfiguration">
>>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>> </bean>
>>> </property>
>>> </bean>
>>> </property>
>> In other data regions persistence will be disabled by default.
>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 02.10.2017 18:35, Denis Magda wrote:
>>>> To resolve this, I suggest to
>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>> other defaults in DataStorageConfiguration.
>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>
>>> —
>>> Denis
>>>
>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>
>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>> talking about are actually inherited. To resolve this, I suggest to
>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>> other defaults in DataStorageConfiguration.
>>>>
>>>> Thoughts?
>>>>
>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>
>>>>> Vladimir,
>>>>>
>>>>> I like your approach because it's easier to implement.
>>>>>
>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>
>>>>>> Ivan,
>>>>>>
>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>> flag for default region only. It should not be propagated to custom
>>>>>> regions.
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Guys, I think I got the point now.
>>>>>>> Let's check the final design:
>>>>>>>
>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>> in
>>>>>>> default data region.
>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>> which
>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>> efaultPersistenceEnabled*
>>>>>>> will be used by default.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>
>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>
>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>> region
>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>
>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>> globally.
>>>>>>>>>
>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>
>>>>>>>>
>>>>>>>>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Mon, Oct 2, 2017 at 7:28 PM, Denis Magda <dm...@apache.org> wrote:
>
> >> <property name="dataStorageConfiguration">
> >> <bean class="org.apache.ignite.configuration.
> DataStorageConfiguration">
> >> <property name="systemCacheInitialSize" value="#{100 *
> 1024 * 1024}"/>
> >> <property name="defaultRegionConfiguration">
> >> <bean class="org.apache.ignite.configuration.
> DataRegionConfiguration">
> >> <property name="maxSize" value="#{5 * 1024 * 102 *
> 1024}"/>
> >> </bean>
> >> </property>
> >> </bean>
> >> </property>
> >
> > In other data regions persistence will be disabled by default.
>
> Ivan, how to get an instance to the default region bean and change a
> parameter? Obviously, if the goal is to enable the persistence I don’t want
> to create the default region bean from scratch.
>
Denis, I think this is the only way. If we don't create the
defaultRegionConfiguration, then we have to make sure that all the
properties are inherited from DataStorageConfiguration. From the feedback I
have seen so far, folks think that the inheritance can get really confusing
here.
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
>> <property name="dataStorageConfiguration">
>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>> <property name="defaultRegionConfiguration">
>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>> </bean>
>> </property>
>> </bean>
>> </property>
>
> In other data regions persistence will be disabled by default.
Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
—
Denis
> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> Agree with Alexey.
>
> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>
> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>
> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>
>> <property name="dataStorageConfiguration">
>> <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>> <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>> <property name="defaultRegionConfiguration">
>> <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>> <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>> </bean>
>> </property>
>> </bean>
>> </property>
>
> In other data regions persistence will be disabled by default.
> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>
> Best Regards,
> Ivan Rakov
>
> On 02.10.2017 18:35, Denis Magda wrote:
>>> To resolve this, I suggest to
>>> introduce just another field defaultRegionConfiguration and get rid of
>>> other defaults in DataStorageConfiguration.
>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>
>> —
>> Denis
>>
>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>
>>> Agree with Vladimir. If we are to implement this, we would either need to
>>> have a Boolean (non-primitive) for persistenceEnabled on
>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>> an overkill. On the other hand, one can assume that the defaults we are
>>> talking about are actually inherited. To resolve this, I suggest to
>>> introduce just another field defaultRegionConfiguration and get rid of
>>> other defaults in DataStorageConfiguration.
>>>
>>> Thoughts?
>>>
>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>
>>>> Vladimir,
>>>>
>>>> I like your approach because it's easier to implement.
>>>>
>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>> region. I'll add clarifying Javadoc at this place.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>
>>>>> Ivan,
>>>>>
>>>>> I do not think this is correct approach, because it will be hard to
>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>> flag for default region only. It should not be propagated to custom
>>>>> regions.
>>>>>
>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Guys, I think I got the point now.
>>>>>> Let's check the final design:
>>>>>>
>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>> property (default = false), which will be used for enabling persistence
>>>>>> in
>>>>>> default data region.
>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>> which
>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>> efaultPersistenceEnabled*
>>>>>> will be used by default.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>
>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>
>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>> region
>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>
>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>> globally.
>>>>>>>>
>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>
>>>>>>>
>>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Agree with Alexey.
Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled*
can confuse users who don't know that there's such thing as default data
region. They can decide they are inherited by all data regions where
size and persistence flag are not explicitly set.
Let's get rid of these properties and add *defaultRegionConfiguration*
property with explicit configuration of default data region.
Regarding XML configuration, changing size or persistence flag of
default data region will be just two lines longer (for bean description):
> <property name="dataStorageConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataStorageConfiguration">
> <property name="systemCacheInitialSize" value="#{100 *
> 1024 * 1024}"/>
> <property name="defaultRegionConfiguration">
> <bean
> class="org.apache.ignite.configuration.DataRegionConfiguration">
> <property name="maxSize" value="#{5 * 1024 * 102
> * 1024}"/>
> </bean>
> </property>
> </bean>
> </property>
In other data regions persistence will be disabled by default.
I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030
with these changes.
Best Regards,
Ivan Rakov
On 02.10.2017 18:35, Denis Magda wrote:
>> To resolve this, I suggest to
>> introduce just another field defaultRegionConfiguration and get rid of
>> other defaults in DataStorageConfiguration.
> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>
> —
> Denis
>
>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>
>> Agree with Vladimir. If we are to implement this, we would either need to
>> have a Boolean (non-primitive) for persistenceEnabled on
>> DataRegionConfiguration, or introduce an enum for this field which is also
>> an overkill. On the other hand, one can assume that the defaults we are
>> talking about are actually inherited. To resolve this, I suggest to
>> introduce just another field defaultRegionConfiguration and get rid of
>> other defaults in DataStorageConfiguration.
>>
>> Thoughts?
>>
>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>
>>> Vladimir,
>>>
>>> I like your approach because it's easier to implement.
>>>
>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>> flag and seeing that persistence is not enabled by default in custom memory
>>> region. I'll add clarifying Javadoc at this place.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>
>>>> Ivan,
>>>>
>>>> I do not think this is correct approach, because it will be hard to
>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>> flag for default region only. It should not be propagated to custom
>>>> regions.
>>>>
>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Guys, I think I got the point now.
>>>>> Let's check the final design:
>>>>>
>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>> property (default = false), which will be used for enabling persistence
>>>>> in
>>>>> default data region.
>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>> which
>>>>> will be used for enabling persistence in corresponding data region. If
>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>> efaultPersistenceEnabled*
>>>>> will be used by default.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>>
>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>
>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>
>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>
>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>> region
>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>
>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>> globally.
>>>>>>>
>>>>>> But we should also give users a way to switch the default behavior from
>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>
>>>>>>
>>>>>>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
> To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.
Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
—
Denis
> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>
> Agree with Vladimir. If we are to implement this, we would either need to
> have a Boolean (non-primitive) for persistenceEnabled on
> DataRegionConfiguration, or introduce an enum for this field which is also
> an overkill. On the other hand, one can assume that the defaults we are
> talking about are actually inherited. To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.
>
> Thoughts?
>
> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>
>> Vladimir,
>>
>> I like your approach because it's easier to implement.
>>
>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>> flag and seeing that persistence is not enabled by default in custom memory
>> region. I'll add clarifying Javadoc at this place.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>
>>> Ivan,
>>>
>>> I do not think this is correct approach, because it will be hard to
>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>> DataRegionConfiguration. I do not think we need default "persistence
>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>> flag for default region only. It should not be propagated to custom
>>> regions.
>>>
>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>>
>>> Guys, I think I got the point now.
>>>>
>>>> Let's check the final design:
>>>>
>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>> property (default = false), which will be used for enabling persistence
>>>> in
>>>> default data region.
>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>> which
>>>> will be used for enabling persistence in corresponding data region. If
>>>> value is not set, value of *DataStorageConfiguration::isD
>>>> efaultPersistenceEnabled*
>>>> will be used by default.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>>
>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>
>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>
>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>
>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>
>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>
>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>> region
>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>
>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>> globally.
>>>>>>
>>>>> But we should also give users a way to switch the default behavior from
>>>>> in-memory only (no-persistence) to persistence.
>>>>>
>>>>>
>>>>>
>>
Re: Persistence per memory policy configuration
Posted by Alexey Kuznetsov <ak...@apache.org>.
Guys,
Please, do not add Boolean (non-primitive) to configurations.
It is really the pain to handle them in tools like WebConsole.
Enum would be better.
I'm not insisting, but just a minor note.
On Mon, Oct 2, 2017 at 10:30 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:
> Agree with Vladimir. If we are to implement this, we would either need to
> have a Boolean (non-primitive) for persistenceEnabled on
> DataRegionConfiguration, or introduce an enum for this field which is also
> an overkill. On the other hand, one can assume that the defaults we are
> talking about are actually inherited. To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.
>
> Thoughts?
>
> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>
> > Vladimir,
> >
> > I like your approach because it's easier to implement.
> >
> > However, user may be confused by setting *isDefaultPersistenceEnabled*
> > flag and seeing that persistence is not enabled by default in custom
> memory
> > region. I'll add clarifying Javadoc at this place.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >
> >> Ivan,
> >>
> >> I do not think this is correct approach, because it will be hard to
> >> explain, and you will have to use "Boolean" instead of "boolean" for
> >> DataRegionConfiguration. I do not think we need default "persistence
> >> enabled" for all regions. Instead, we should have "persistence enabled"
> >> flag for default region only. It should not be propagated to custom
> >> regions.
> >>
> >> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
> >> wrote:
> >>
> >> Guys, I think I got the point now.
> >>>
> >>> Let's check the final design:
> >>>
> >>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
> >>> property (default = false), which will be used for enabling persistence
> >>> in
> >>> default data region.
> >>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
> >>> which
> >>> will be used for enabling persistence in corresponding data region. If
> >>> value is not set, value of *DataStorageConfiguration::isD
> >>> efaultPersistenceEnabled*
> >>> will be used by default.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>>
> >>>
> >>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>
> >>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
> >>>>
> >>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>>
> >>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>
> >>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will
> be
> >>>>> enabled globally if at least one memory region has this flag set.
> >>>>>
> >>>>> I’m confused. Why the persistence should be enabled *globally* if the
> >>>>> purpose is to have it set for a specific region? If it’s enabled for
> >>>>> region
> >>>>> A only, I don’t want to have it activated for region B.
> >>>>>
> >>>>> Yes, you are right. By default the persistence will be disabled
> >>>>> globally.
> >>>>>
> >>>> But we should also give users a way to switch the default behavior
> from
> >>>> in-memory only (no-persistence) to persistence.
> >>>>
> >>>>
> >>>>
> >
>
--
Alexey Kuznetsov
Re: Persistence per memory policy configuration
Posted by Alexey Goncharuk <al...@gmail.com>.
Agree with Vladimir. If we are to implement this, we would either need to
have a Boolean (non-primitive) for persistenceEnabled on
DataRegionConfiguration, or introduce an enum for this field which is also
an overkill. On the other hand, one can assume that the defaults we are
talking about are actually inherited. To resolve this, I suggest to
introduce just another field defaultRegionConfiguration and get rid of
other defaults in DataStorageConfiguration.
Thoughts?
2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
> Vladimir,
>
> I like your approach because it's easier to implement.
>
> However, user may be confused by setting *isDefaultPersistenceEnabled*
> flag and seeing that persistence is not enabled by default in custom memory
> region. I'll add clarifying Javadoc at this place.
>
> Best Regards,
> Ivan Rakov
>
>
> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>
>> Ivan,
>>
>> I do not think this is correct approach, because it will be hard to
>> explain, and you will have to use "Boolean" instead of "boolean" for
>> DataRegionConfiguration. I do not think we need default "persistence
>> enabled" for all regions. Instead, we should have "persistence enabled"
>> flag for default region only. It should not be propagated to custom
>> regions.
>>
>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>
>> Guys, I think I got the point now.
>>>
>>> Let's check the final design:
>>>
>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>> property (default = false), which will be used for enabling persistence
>>> in
>>> default data region.
>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>> which
>>> will be used for enabling persistence in corresponding data region. If
>>> value is not set, value of *DataStorageConfiguration::isD
>>> efaultPersistenceEnabled*
>>> will be used by default.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>>
>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>
>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>
>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>> enabled globally if at least one memory region has this flag set.
>>>>>
>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>> region
>>>>> A only, I don’t want to have it activated for region B.
>>>>>
>>>>> Yes, you are right. By default the persistence will be disabled
>>>>> globally.
>>>>>
>>>> But we should also give users a way to switch the default behavior from
>>>> in-memory only (no-persistence) to persistence.
>>>>
>>>>
>>>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Vladimir,
I like your approach because it's easier to implement.
However, user may be confused by setting *isDefaultPersistenceEnabled*
flag and seeing that persistence is not enabled by default in custom
memory region. I'll add clarifying Javadoc at this place.
Best Regards,
Ivan Rakov
On 02.10.2017 11:28, Vladimir Ozerov wrote:
> Ivan,
>
> I do not think this is correct approach, because it will be hard to
> explain, and you will have to use "Boolean" instead of "boolean" for
> DataRegionConfiguration. I do not think we need default "persistence
> enabled" for all regions. Instead, we should have "persistence enabled"
> flag for default region only. It should not be propagated to custom regions.
>
> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Guys, I think I got the point now.
>>
>> Let's check the final design:
>>
>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>> property (default = false), which will be used for enabling persistence in
>> default data region.
>> *DataRegionConfiguration* will have *isPersistenceEnabled* property, which
>> will be used for enabling persistence in corresponding data region. If
>> value is not set, value of *DataStorageConfiguration::isDefaultPersistenceEnabled*
>> will be used by default.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>>
>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>
>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>> 1) You're right. I forgot to include the main flag in
>>>>>
>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>> enabled globally if at least one memory region has this flag set.
>>>>
>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>> region
>>>> A only, I don’t want to have it activated for region B.
>>>>
>>>> Yes, you are right. By default the persistence will be disabled globally.
>>> But we should also give users a way to switch the default behavior from
>>> in-memory only (no-persistence) to persistence.
>>>
>>>
Re: Persistence per memory policy configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
Ivan,
I do not think this is correct approach, because it will be hard to
explain, and you will have to use "Boolean" instead of "boolean" for
DataRegionConfiguration. I do not think we need default "persistence
enabled" for all regions. Instead, we should have "persistence enabled"
flag for default region only. It should not be propagated to custom regions.
On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com> wrote:
> Guys, I think I got the point now.
>
> Let's check the final design:
>
> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
> property (default = false), which will be used for enabling persistence in
> default data region.
> *DataRegionConfiguration* will have *isPersistenceEnabled* property, which
> will be used for enabling persistence in corresponding data region. If
> value is not set, value of *DataStorageConfiguration::isDefaultPersistenceEnabled*
> will be used by default.
>
> Best Regards,
> Ivan Rakov
>
>
>
> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>
>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>
>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> 1) You're right. I forgot to include the main flag in
>>>>
>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>> enabled globally if at least one memory region has this flag set.
>>>
>>> I’m confused. Why the persistence should be enabled *globally* if the
>>> purpose is to have it set for a specific region? If it’s enabled for
>>> region
>>> A only, I don’t want to have it activated for region B.
>>>
>>> Yes, you are right. By default the persistence will be disabled globally.
>> But we should also give users a way to switch the default behavior from
>> in-memory only (no-persistence) to persistence.
>>
>>
>
Re: Persistence per memory policy configuration
Posted by Ivan Rakov <iv...@gmail.com>.
Guys, I think I got the point now.
Let's check the final design:
*DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
property (default = false), which will be used for enabling persistence
in default data region.
*DataRegionConfiguration* will have *isPersistenceEnabled* property,
which will be used for enabling persistence in corresponding data
region. If value is not set, value of
*DataStorageConfiguration::isDefaultPersistenceEnabled* will be used by
default.
Best Regards,
Ivan Rakov
On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>>
>> I’m confused. Why the persistence should be enabled *globally* if the
>> purpose is to have it set for a specific region? If it’s enabled for region
>> A only, I don’t want to have it activated for region B.
>>
> Yes, you are right. By default the persistence will be disabled globally.
> But we should also give users a way to switch the default behavior from
> in-memory only (no-persistence) to persistence.
>
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am not sure I like the name "isDefaultDataRegionPersistenceEnabled" - too
long. Can we just call it "isDefaultPersistenceEnabled"?
D.
On Sun, Oct 1, 2017 at 2:41 PM, Ivan Rakov <iv...@gmail.com> wrote:
> Denis,
>
> 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
> Regarding default data region, I've added *isDefaultDataRegionPersistenceEnabled
> *to the DataStorageConfiguration. Check the design draft again.
>
> 2) Of course, we have to maintain API compatibility. Deprecating old
> classes and introducing new is just what I meant.
>
> 3) We can't do that - MemoryMetrics are calculated per memory policy and
> PersistenceMetrics are calculated globally. It's a major change to
> implement it another way.
>
> By the way, let's assure the namings for new metrics classes.
> DataRegionMetrics instead of MemoryMetrics looks fine.
> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair enough
> as it will contain only metrics of persistent storage. Probably
> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
> keeping *PersistenceMetrics* are better.
>
> What do you think?
>
> Best Regards,
> Ivan Rakov
>
>
> On 29.09.2017 21:12, Denis Magda wrote:
>
>> Ivan,
>>
>> Several questions/concerns:
>>
>> 1. Looking at the new API I can’t grasp how to enable the persistence.
>> First, how can I enable it globally if there is only one default data
>> region defined. Second, how do I enable it per data region. Can’t find any
>> related switches in the draft.
>>
>> 2. We cannot renamed anything like you’re suggesting to do for
>> MemoryMetrics and their beans. We have to deprecate old and introduce new.
>>
>> 3. I think we should merge Memory and Persistence Metrics into
>> DataStorageMetrics for clarity.
>>
>> —
>> Denis
>>
>>
>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Guys,
>>>
>>> I've attached new configuration design draft to the ticket description:
>>> https://issues.apache.org/jira/browse/IGNITE-6030
>>> Please, take a look. Right now is the best time to change name of any
>>> property.
>>>
>>> And question about metrics: are we going to rename MemoryMetrics and
>>> PersistenceMetrics respectively (along with their MBeans)?
>>> It's not a problem to implement it at all. The only thing that concerns
>>> me is that we have to keep deprecated old classes in the codebase. Perhaps,
>>> MemoryMetrics/PersistenceMetrics are intuitive enough.
>>>
>>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>>
>>>> StorageRegion sounds like bad English to me.
>>>>
>>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>>
>>>> D.
>>>>
>>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
>>>> wrote:
>>>>
>>>> Guys,
>>>>>
>>>>> But what is exact desicion? :-) I saw two final options:
>>>>>
>>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>>
>>>>> Which one we choose?
>>>>>
>>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>>> wrote:
>>>>>
>>>>> I guess it is safe to assume that at this point we came to a consensus?
>>>>>>>
>>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>>
>>>>>> --Yakov
>>>>>>
>>>>>>
>
Re: Persistence per memory policy configuration
Posted by Vladimir Ozerov <vo...@gridgain.com>.
We merged memory and persistence on config level. So we should merge
metrics in the same way. DataRegionMetrics is absolutely normal name, even
if it contains only persistence-related stuff at the moment.
вс, 1 окт. 2017 г. в 14:41, Ivan Rakov <iv...@gmail.com>:
> Denis,
>
> 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
> Regarding default data region, I've added
> *isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration.
> Check the design draft again.
>
> 2) Of course, we have to maintain API compatibility. Deprecating old
> classes and introducing new is just what I meant.
>
> 3) We can't do that - MemoryMetrics are calculated per memory policy and
> PersistenceMetrics are calculated globally. It's a major change to
> implement it another way.
>
> By the way, let's assure the namings for new metrics classes.
> DataRegionMetrics instead of MemoryMetrics looks fine.
> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair
> enough as it will contain only metrics of persistent storage. Probably
> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
> keeping *PersistenceMetrics* are better.
>
> What do you think?
>
> Best Regards,
> Ivan Rakov
>
> On 29.09.2017 21:12, Denis Magda wrote:
> > Ivan,
> >
> > Several questions/concerns:
> >
> > 1. Looking at the new API I can’t grasp how to enable the persistence.
> First, how can I enable it globally if there is only one default data
> region defined. Second, how do I enable it per data region. Can’t find any
> related switches in the draft.
> >
> > 2. We cannot renamed anything like you’re suggesting to do for
> MemoryMetrics and their beans. We have to deprecate old and introduce new.
> >
> > 3. I think we should merge Memory and Persistence Metrics into
> DataStorageMetrics for clarity.
> >
> > —
> > Denis
> >
> >
> >> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>
> >> Guys,
> >>
> >> I've attached new configuration design draft to the ticket description:
> https://issues.apache.org/jira/browse/IGNITE-6030
> >> Please, take a look. Right now is the best time to change name of any
> property.
> >>
> >> And question about metrics: are we going to rename MemoryMetrics and
> PersistenceMetrics respectively (along with their MBeans)?
> >> It's not a problem to implement it at all. The only thing that concerns
> me is that we have to keep deprecated old classes in the codebase. Perhaps,
> MemoryMetrics/PersistenceMetrics are intuitive enough.
> >>
> >> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
> >>> StorageRegion sounds like bad English to me.
> >>>
> >>> I would go with DataStorageConfiguration and DataRegionConfiguration.
> >>>
> >>> D.
> >>>
> >>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> >>> wrote:
> >>>
> >>>> Guys,
> >>>>
> >>>> But what is exact desicion? :-) I saw two final options:
> >>>>
> >>>> 1) StorageConfiguration + StorageRegionConfiguration
> >>>> 2) DataStorageConfiguration + DataRegionConfiguration
> >>>>
> >>>> Which one we choose?
> >>>>
> >>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
> >>>> wrote:
> >>>>
> >>>>>> I guess it is safe to assume that at this point we came to a
> consensus?
> >>>>> Alex, I think so. Let's carve it in stone (code).
> >>>>>
> >>>>> --Yakov
> >>>>>
>
>
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
> On Oct 1, 2017, at 9:49 PM, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dmagda@apache.org <ma...@apache.org>> wrote:
>
>>
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>>
>> I’m confused. Why the persistence should be enabled *globally* if the
>> purpose is to have it set for a specific region? If it’s enabled for region
>> A only, I don’t want to have it activated for region B.
>>
>
> Yes, you are right. By default the persistence will be disabled globally.
> But we should also give users a way to switch the default behavior from
> in-memory only (no-persistence) to persistence.
Sure, that’s how it was discussed initially. Basically, at the API level we should have:
1. DataRegionConfigurations.isPersistenceEnabled - that turns on/off the persistence per region.
2. DataStorageConfiguration.isPersistenceEnabled - that does this globally except for regions disabled it explicitly via 1.
My confusion is caused by Ivan’s allegation that as soon as the persistence is enabled at least for one region via 1 the persistence will be globally activated for the rest (as I would enable it via 2). Ivan did I get your words in a right way?
—
Denis
Re: Persistence per memory policy configuration
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>
> > On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
>
> I’m confused. Why the persistence should be enabled *globally* if the
> purpose is to have it set for a specific region? If it’s enabled for region
> A only, I don’t want to have it activated for region B.
>
Yes, you are right. By default the persistence will be disabled globally.
But we should also give users a way to switch the default behavior from
in-memory only (no-persistence) to persistence.
Re: Persistence per memory policy configuration
Posted by Denis Magda <dm...@apache.org>.
> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
> 1) You're right. I forgot to include the main flag in DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be enabled globally if at least one memory region has this flag set.
I’m confused. Why the persistence should be enabled *globally* if the purpose is to have it set for a specific region? If it’s enabled for region A only, I don’t want to have it activated for region B.
—
Denis