Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Igniters,

I am currently reviewing a change allowing to enable persistence on a
per-memory-policy basis (thanks to K. Dudkov!) and have a question
regarding the changes in configuration.

The suggested change is to add a flag "persistenceEnabled" (defaults to
true) to the memory policy configuration. To keep configuration
compatibility, the logic is as follows:

If PersistentStoreConfiguration is set, then only memory policies with
persistenceEnabled=true flag will be persisted, which is consistent with
the current behavior. To disable persistence, persistenceEnabled flag
should be explicitly set to false.

If PersistentStoreConfiguration is not set, then all caches are stored
in-memory and persistenceEnabled is ignored.

While personally, I like this change, I would like to check if there are
any thoughts or objections to this approach.

--
Thanks,
AG

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

While we are contemplating the best way to introduce this feature I have a side note to consider. Can the user enable CacheStore (RDBMS, Cassandra) for caches that are not persisted in Ignite Persistence? That’s one of the use cases I hear a lot about - some of the caches need persisted in Ignite Persistence while the rest in a 3rd party storage.

Think it makes sense to support this use case as a part of this work.

—
Denis

> On Sep 12, 2017, at 2:24 AM, Alexey Goncharuk <al...@gmail.com> wrote:
> 
> Igniters,
> 
> I am currently reviewing a change allowing to enable persistence on a
> per-memory-policy basis (thanks to K. Dudkov!) and have a question
> regarding the changes in configuration.
> 
> The suggested change is to add a flag "persistenceEnabled" (defaults to
> true) to the memory policy configuration. To keep configuration
> compatibility, the logic is as follows:
> 
> If PersistentStoreConfiguration is set, then only memory policies with
> persistenceEnabled=true flag will be persisted, which is consistent with
> the current behavior. To disable persistence, persistenceEnabled flag
> should be explicitly set to false.
> 
> If PersistentStoreConfiguration is not set, then all caches are stored
> in-memory and persistenceEnabled is ignored.
> 
> While personally, I like this change, I would like to check if there are
> any thoughts or objections to this approach.
> 
> --
> Thanks,
> AG

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Sounds good. But can we see a full example of such configuration? E.g. we
have two caches, one in-memory, another persistent.

On Wed, Sep 13, 2017 at 10:54 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> I like the idea of having PersistentMemoryPolicy configuration class.
> Vladimir, Yakov, any objections?
>
> 2017-09-13 3:05 GMT+03:00 Denis Magda <dm...@apache.org>:
>
> > I would go for PersistentMemoryPolicyConfiguration way if it will be
> > expanded by storage related settings furthers.
> >
> > For instance, from the discussion on @user I see a raising demand for the
> > storage maximum size. If it’s reached the data will be evicted from there
> > too. Just as an example.
> >
> > —
> > Denis
> >
> > > On Sep 12, 2017, at 2:01 PM, Dmitriy Setrakyan <ds...@apache.org>
> > wrote:
> > >
> > > To extend on the idea of 2 different policies for memory and
> persistence,
> > > can we have 2 completely different configuration classes?
> > >
> > > - MemoryPolicy
> > > - PersistentMemoryPolicy (extends MemoryPolicy?)
> > >
> > > Every cache should be associated with either MemoryPolicy or
> > > PersistentMemoryPolicy, but not both. By default, caches on startup are
> > > associated with default MemoryPolicy. Users can always change it to
> some
> > > PersistentMemoryPolicy, if persistence needs to be enabled.
> > >
> > > If we agree on the above, do we really need a persistenceEnabled flag
> at
> > > all?
> > >
> > > D.
> > >
> > > On Tue, Sep 12, 2017 at 2:57 AM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > >> We surely can, but:
> > >> * we should then have two configuration settings for default memory
> > policy
> > >> size (one in-memory and one persisted)
> > >> * a user still may configure multiple custom memory policies. In this
> > >> case, the requirement to have this flag the same in a memory policy is
> > >> still valid, so a user still can get exceptions.
> > >>
> > >> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> > >>
> > >>> Alex,
> > >>>
> > >>> Can we have two default memory policies - one for in-memory and
> another
> > >> one
> > >>> for persistence cases?
> > >>>
> > >>> On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> > >>> alexey.goncharuk@gmail.com> wrote:
> > >>>
> > >>>> This is possible, but then if two caches belong to the same memory
> > >>> policy,
> > >>>> they must be both either persistence-enabled or
> persistence-disabled.
> > >> We
> > >>>> can add this validation, but I think this will lead to a greater
> > >>> confusion
> > >>>> for a user.
> > >>>>
> > >>>> 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> > >>>>
> > >>>>> Agree with Vladimir.
> > >>>>>
> > >>>>> Currently we enable persistence cluster-wide by setting
> > >>>>> IgniteConfiguration.persistentStoreConfiguration.
> > >>>>> Ideally CacheConfiguration.persistenceEnabled should be the only
> > >>>> setting I
> > >>>>> need to set.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Pavel
> > >>>>>
> > >>>>> On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> > >>> vozerov@gridgain.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> As a user I would definitely prefer to control persistence through
> > >>> flag
> > >>>>> on
> > >>>>>> cache configuration. I do not even want to know what "memory
> > >> policy"
> > >>>> is.
> > >>>>>> E.g. CacheConfiguration.persistenceEnabled.
> > >>>>>>
> > >>>>>> On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > >>>>>> alexey.goncharuk@gmail.com> wrote:
> > >>>>>>
> > >>>>>>> Igniters,
> > >>>>>>>
> > >>>>>>> I am currently reviewing a change allowing to enable persistence
> > >>> on a
> > >>>>>>> per-memory-policy basis (thanks to K. Dudkov!) and have a
> > >> question
> > >>>>>>> regarding the changes in configuration.
> > >>>>>>>
> > >>>>>>> The suggested change is to add a flag "persistenceEnabled"
> > >>> (defaults
> > >>>> to
> > >>>>>>> true) to the memory policy configuration. To keep configuration
> > >>>>>>> compatibility, the logic is as follows:
> > >>>>>>>
> > >>>>>>> If PersistentStoreConfiguration is set, then only memory policies
> > >>>> with
> > >>>>>>> persistenceEnabled=true flag will be persisted, which is
> > >> consistent
> > >>>>> with
> > >>>>>>> the current behavior. To disable persistence, persistenceEnabled
> > >>> flag
> > >>>>>>> should be explicitly set to false.
> > >>>>>>>
> > >>>>>>> If PersistentStoreConfiguration is not set, then all caches are
> > >>>> stored
> > >>>>>>> in-memory and persistenceEnabled is ignored.
> > >>>>>>>
> > >>>>>>> While personally, I like this change, I would like to check if
> > >>> there
> > >>>>> are
> > >>>>>>> any thoughts or objections to this approach.
> > >>>>>>>
> > >>>>>>> --
> > >>>>>>> Thanks,
> > >>>>>>> AG
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

> I like the idea of having PersistentMemoryPolicy configuration class.

I like the idea, too.

--Yakov

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

I like the idea of having PersistentMemoryPolicy configuration class.
Vladimir, Yakov, any objections?

2017-09-13 3:05 GMT+03:00 Denis Magda <dm...@apache.org>:

> I would go for PersistentMemoryPolicyConfiguration way if it will be
> expanded by storage related settings furthers.
>
> For instance, from the discussion on @user I see a raising demand for the
> storage maximum size. If it’s reached the data will be evicted from there
> too. Just as an example.
>
> —
> Denis
>
> > On Sep 12, 2017, at 2:01 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > To extend on the idea of 2 different policies for memory and persistence,
> > can we have 2 completely different configuration classes?
> >
> > - MemoryPolicy
> > - PersistentMemoryPolicy (extends MemoryPolicy?)
> >
> > Every cache should be associated with either MemoryPolicy or
> > PersistentMemoryPolicy, but not both. By default, caches on startup are
> > associated with default MemoryPolicy. Users can always change it to some
> > PersistentMemoryPolicy, if persistence needs to be enabled.
> >
> > If we agree on the above, do we really need a persistenceEnabled flag at
> > all?
> >
> > D.
> >
> > On Tue, Sep 12, 2017 at 2:57 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> >> We surely can, but:
> >> * we should then have two configuration settings for default memory
> policy
> >> size (one in-memory and one persisted)
> >> * a user still may configure multiple custom memory policies. In this
> >> case, the requirement to have this flag the same in a memory policy is
> >> still valid, so a user still can get exceptions.
> >>
> >> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> >>
> >>> Alex,
> >>>
> >>> Can we have two default memory policies - one for in-memory and another
> >> one
> >>> for persistence cases?
> >>>
> >>> On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> >>> alexey.goncharuk@gmail.com> wrote:
> >>>
> >>>> This is possible, but then if two caches belong to the same memory
> >>> policy,
> >>>> they must be both either persistence-enabled or persistence-disabled.
> >> We
> >>>> can add this validation, but I think this will lead to a greater
> >>> confusion
> >>>> for a user.
> >>>>
> >>>> 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> >>>>
> >>>>> Agree with Vladimir.
> >>>>>
> >>>>> Currently we enable persistence cluster-wide by setting
> >>>>> IgniteConfiguration.persistentStoreConfiguration.
> >>>>> Ideally CacheConfiguration.persistenceEnabled should be the only
> >>>> setting I
> >>>>> need to set.
> >>>>>
> >>>>> Thanks,
> >>>>> Pavel
> >>>>>
> >>>>> On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> >>> vozerov@gridgain.com>
> >>>>> wrote:
> >>>>>
> >>>>>> As a user I would definitely prefer to control persistence through
> >>> flag
> >>>>> on
> >>>>>> cache configuration. I do not even want to know what "memory
> >> policy"
> >>>> is.
> >>>>>> E.g. CacheConfiguration.persistenceEnabled.
> >>>>>>
> >>>>>> On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>
> >>>>>>> Igniters,
> >>>>>>>
> >>>>>>> I am currently reviewing a change allowing to enable persistence
> >>> on a
> >>>>>>> per-memory-policy basis (thanks to K. Dudkov!) and have a
> >> question
> >>>>>>> regarding the changes in configuration.
> >>>>>>>
> >>>>>>> The suggested change is to add a flag "persistenceEnabled"
> >>> (defaults
> >>>> to
> >>>>>>> true) to the memory policy configuration. To keep configuration
> >>>>>>> compatibility, the logic is as follows:
> >>>>>>>
> >>>>>>> If PersistentStoreConfiguration is set, then only memory policies
> >>>> with
> >>>>>>> persistenceEnabled=true flag will be persisted, which is
> >> consistent
> >>>>> with
> >>>>>>> the current behavior. To disable persistence, persistenceEnabled
> >>> flag
> >>>>>>> should be explicitly set to false.
> >>>>>>>
> >>>>>>> If PersistentStoreConfiguration is not set, then all caches are
> >>>> stored
> >>>>>>> in-memory and persistenceEnabled is ignored.
> >>>>>>>
> >>>>>>> While personally, I like this change, I would like to check if
> >>> there
> >>>>> are
> >>>>>>> any thoughts or objections to this approach.
> >>>>>>>
> >>>>>>> --
> >>>>>>> Thanks,
> >>>>>>> AG
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

I would go for PersistentMemoryPolicyConfiguration way if it will be expanded by storage related settings furthers.

For instance, from the discussion on @user I see a raising demand for the storage maximum size. If it’s reached the data will be evicted from there too. Just as an example.

—
Denis

> On Sep 12, 2017, at 2:01 PM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> To extend on the idea of 2 different policies for memory and persistence,
> can we have 2 completely different configuration classes?
> 
> - MemoryPolicy
> - PersistentMemoryPolicy (extends MemoryPolicy?)
> 
> Every cache should be associated with either MemoryPolicy or
> PersistentMemoryPolicy, but not both. By default, caches on startup are
> associated with default MemoryPolicy. Users can always change it to some
> PersistentMemoryPolicy, if persistence needs to be enabled.
> 
> If we agree on the above, do we really need a persistenceEnabled flag at
> all?
> 
> D.
> 
> On Tue, Sep 12, 2017 at 2:57 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
> 
>> We surely can, but:
>> * we should then have two configuration settings for default memory policy
>> size (one in-memory and one persisted)
>> * a user still may configure multiple custom memory policies. In this
>> case, the requirement to have this flag the same in a memory policy is
>> still valid, so a user still can get exceptions.
>> 
>> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
>> 
>>> Alex,
>>> 
>>> Can we have two default memory policies - one for in-memory and another
>> one
>>> for persistence cases?
>>> 
>>> On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com> wrote:
>>> 
>>>> This is possible, but then if two caches belong to the same memory
>>> policy,
>>>> they must be both either persistence-enabled or persistence-disabled.
>> We
>>>> can add this validation, but I think this will lead to a greater
>>> confusion
>>>> for a user.
>>>> 
>>>> 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
>>>> 
>>>>> Agree with Vladimir.
>>>>> 
>>>>> Currently we enable persistence cluster-wide by setting
>>>>> IgniteConfiguration.persistentStoreConfiguration.
>>>>> Ideally CacheConfiguration.persistenceEnabled should be the only
>>>> setting I
>>>>> need to set.
>>>>> 
>>>>> Thanks,
>>>>> Pavel
>>>>> 
>>>>> On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
>>> vozerov@gridgain.com>
>>>>> wrote:
>>>>> 
>>>>>> As a user I would definitely prefer to control persistence through
>>> flag
>>>>> on
>>>>>> cache configuration. I do not even want to know what "memory
>> policy"
>>>> is.
>>>>>> E.g. CacheConfiguration.persistenceEnabled.
>>>>>> 
>>>>>> On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>> 
>>>>>>> Igniters,
>>>>>>> 
>>>>>>> I am currently reviewing a change allowing to enable persistence
>>> on a
>>>>>>> per-memory-policy basis (thanks to K. Dudkov!) and have a
>> question
>>>>>>> regarding the changes in configuration.
>>>>>>> 
>>>>>>> The suggested change is to add a flag "persistenceEnabled"
>>> (defaults
>>>> to
>>>>>>> true) to the memory policy configuration. To keep configuration
>>>>>>> compatibility, the logic is as follows:
>>>>>>> 
>>>>>>> If PersistentStoreConfiguration is set, then only memory policies
>>>> with
>>>>>>> persistenceEnabled=true flag will be persisted, which is
>> consistent
>>>>> with
>>>>>>> the current behavior. To disable persistence, persistenceEnabled
>>> flag
>>>>>>> should be explicitly set to false.
>>>>>>> 
>>>>>>> If PersistentStoreConfiguration is not set, then all caches are
>>>> stored
>>>>>>> in-memory and persistenceEnabled is ignored.
>>>>>>> 
>>>>>>> While personally, I like this change, I would like to check if
>>> there
>>>>> are
>>>>>>> any thoughts or objections to this approach.
>>>>>>> 
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> AG
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

I like Vladimir's suggestion - StorageConfiguration -
StorageRegionConfiguration

--Yakov

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Vladimir,

*DataRegionMetrics* is for former memory metrics.
*DataStorageMetrics* will contain metrics about persistence.

If DataStorageMetrics is ok, let's go this way.

Best Regards,
Ivan Rakov

On 01.10.2017 15:19, Vladimir Ozerov wrote:
> We merged memory and persistence on config level. So we should merge
> metrics in the same way. DataRegionMetrics is absolutely normal name, even
> if it contains only persistence-related stuff at the moment.
>
> вс, 1 окт. 2017 г. в 14:41, Ivan Rakov <iv...@gmail.com>:
>
>> Denis,
>>
>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>> Regarding default data region, I've added
>> *isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration.
>> Check the design draft again.
>>
>> 2) Of course, we have to maintain API compatibility. Deprecating old
>> classes and introducing new is just what I meant.
>>
>> 3) We can't do that - MemoryMetrics are calculated per memory policy and
>> PersistenceMetrics are calculated globally. It's a major change to
>> implement it another way.
>>
>> By the way, let's assure the namings for new metrics classes.
>> DataRegionMetrics instead of MemoryMetrics looks fine.
>> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair
>> enough as it will contain only metrics of persistent storage. Probably
>> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
>> keeping *PersistenceMetrics* are better.
>>
>> What do you think?
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 29.09.2017 21:12, Denis Magda wrote:
>>> Ivan,
>>>
>>> Several questions/concerns:
>>>
>>> 1. Looking at the new API I can’t grasp how to enable the persistence.
>> First, how can I enable it globally if there is only one default data
>> region defined. Second, how do I enable it per data region. Can’t find any
>> related switches in the draft.
>>> 2. We cannot renamed anything like you’re suggesting to do for
>> MemoryMetrics and their beans. We have to deprecate old and introduce new.
>>> 3. I think we should merge Memory and Persistence Metrics into
>> DataStorageMetrics for clarity.
>>> —
>>> Denis
>>>
>>>
>>>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Guys,
>>>>
>>>> I've attached new configuration design draft to the ticket description:
>> https://issues.apache.org/jira/browse/IGNITE-6030
>>>> Please, take a look. Right now is the best time to change name of any
>> property.
>>>> And question about metrics: are we going to rename MemoryMetrics and
>> PersistenceMetrics respectively (along with their MBeans)?
>>>> It's not a problem to implement it at all. The only thing that concerns
>> me is that we have to keep deprecated old classes in the codebase. Perhaps,
>> MemoryMetrics/PersistenceMetrics are intuitive enough.
>>>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>>>> StorageRegion sounds like bad English to me.
>>>>>
>>>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>>>
>>>>> D.
>>>>>
>>>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vozerov@gridgain.com
>>>>> wrote:
>>>>>
>>>>>> Guys,
>>>>>>
>>>>>> But what is exact desicion? :-) I saw two final options:
>>>>>>
>>>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>>>
>>>>>> Which one we choose?
>>>>>>
>>>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>>> I guess it is safe to assume that at this point we came to a
>> consensus?
>>>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>>>
>>>>>>> --Yakov
>>>>>>>
>>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

The changes are implemented in ignite-6748 branch. TC run is pending.

2017-10-25 11:26 GMT+03:00 Alexey Goncharuk <al...@gmail.com>:

> Igniters,
>
> I've found one more shortcoming which we missed during the review. The
> checkpoint page buffer is attributed to a data region, so originally it was
> a mistake to place the checkpoint page buffer size to the global
> configuration. This property should go to the data region configuration.
>
> I will prepare a PR shortly so we can review the change.
>
> 2017-10-18 4:52 GMT+03:00 Denis Magda <dm...@gridgain.com>:
>
>> Ivan,
>>
>> Please don’t forget to update all the persistence and memory pools related
>> examples to the new configuration format. Let’s make sure non of our
>> example prints out the class deprecated warning.
>>
>> Denis
>>
>> On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>
>> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
>> > for our users to understand. We should also document this configuration
>> on
>> > readme.
>> >
>> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>> wrote:
>> >
>> > > Dmitriy,
>> > >
>> > > Please check description of https://issues.apache.org/jira
>> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
>> > >
>> > > Best Regards,
>> > > Ivan Rakov
>> > >
>> > >
>> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>> > >
>> > >> I am now confused. Can I please ask for the final configuration
>> again?
>> > >> What
>> > >> will it look like?
>> > >>
>> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> > >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
>> > >>
>> > >> Agree with Ivan. If we implemented backward compatibility, this
>> would be
>> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
>> is.
>> > >>>
>> > >>> As for the swap path, I see nothing wrong with having it for
>> in-memory
>> > >>> caches. This is a simple overflow mechanism that works fine if you
>> do
>> > not
>> > >>> need persistence guarantees.
>> > >>>
>> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>>:
>> > >>>
>> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
>> > >>>> *walArchivePath*.
>> > >>>>
>> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
>> path
>> > >>>>
>> > >>> for
>> > >>>
>> > >>>> WAL is implicitly changed, especially when we have separate option
>> for
>> > >>>>
>> > >>> it.
>> > >>>
>> > >>>> WAL and storage files are already located under same $IGNITE_HOME
>> > root.
>> > >>>>  From user perspective, there's no need to change root for all
>> > >>>> persistence-related directories as long as $IGNITE_HOME points to
>> the
>> > >>>> correct disk.
>> > >>>>  From developer perspective, this change breaks backwards
>> > compatibility.
>> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
>> > >>>> old-style and new-style paths) is complex and hard to maintain in
>> the
>> > >>>> codebase.
>> > >>>>
>> > >>>> Best Regards,
>> > >>>> Ivan Rakov
>> > >>>>
>> > >>>> My vote is for *storagePath* and keeping behavior as is.
>> > >>>>
>> > >>>>
>> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>> > >>>>
>> > >>>> Igniters, another thing to consider:
>> > >>>>>
>> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> > >>>>> since this is actually not a single file, but a directory path.
>> > >>>>>
>> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
>> > <javascript:;>>
>> > >>>>> wrote:
>> > >>>>>
>> > >>>>> Seems I've got what you’re talking about.
>> > >>>>>
>> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
>> saw
>> > >>>>>>
>> > >>>>> that
>> > >>>
>> > >>>> only data/indexes were placed to it while wal stayed somewhere in
>> my
>> > >>>>>>
>> > >>>>> work
>> > >>>
>> > >>>> dir. It works counterintuitive and causes non productive
>> discussions
>> > >>>>>>
>> > >>>>> like
>> > >>>
>> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> > >>>>>>
>> > >>>>> name
>> > >>>
>> > >>>> fits this behavior.
>> > >>>>>>
>> > >>>>>> My suggestion will be the following:
>> > >>>>>> - *persistencePath* refers to the path of all storage files
>> > >>>>>> (data/indexes,
>> > >>>>>> wal, archive). If the path is changed *all the files* will be
>> under
>> > >>>>>> the
>> > >>>>>> new
>> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>> > >>>>>> *explicitly*.
>> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
>> > >>>>>> setPersistencePath)
>> > >>>>>> - *setWalArchivePath* overrides the default location of the
>> archive
>> > >>>>>> (which
>> > >>>>>> is again has to be setPersistencePath).
>> > >>>>>>
>> > >>>>>> If we follow this approach the configuration and behavior becomes
>> > >>>>>>
>> > >>>>> vivid.
>> > >>>
>> > >>>> Thoughts?
>> > >>>>>>
>> > >>>>>> —
>> > >>>>>> Denis
>> > >>>>>>
>> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
>> > <javascript:;>>
>> > >>>>>> wrote:
>> > >>>>>>
>> > >>>>>>> Denis,
>> > >>>>>>>
>> > >>>>>>> Data/index storage and WAL are located under the same root by
>> > >>>>>>> default.
>> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>> > >>>>>>> properties
>> > >>>>>>>
>> > >>>>>>> can contain both absolute and relative paths. If paths are
>> > absolute,
>> > >>>>>> storage and WAL can reside on different devices, like this:
>> > >>>>>>
>> > >>>>>> storagePath: /storage1/NMVe_drive/storage
>> > >>>>>>>
>> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>> > >>>>>>>>
>> > >>>>>>>> We even recommend this in tuning guide:
>> > https://apacheignite.readme
>> > >>>>>>> .
>> > >>>>>>>
>> > >>>>>>> io/docs/durable-memory-tuning
>> > >>>>>>
>> > >>>>>> That's why I think *persistencePath* is misleading.
>> > >>>>>>>
>> > >>>>>>> Best Regards,
>> > >>>>>>> Ivan Rakov
>> > >>>>>>>
>> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> > >>>>>>>
>> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
>> dmagda@gridgain.com
>> > <javascript:;>>
>> > >>>>>>>>
>> > >>>>>>>> wrote:
>> > >>>>>>>   From what I see after running an example they are under the
>> same
>> > >>>>>>> root
>> > >>>>>>>
>> > >>>>>>>> folder and in different subdirectories. The root folder should
>> be
>> > >>>>>>>>>
>> > >>>>>>>>> defined
>> > >>>>>>>>
>> > >>>>>>> by setPersistencePath as I guess.
>> > >>>>>>>
>> > >>>>>>>> If that is the case, then you are right. Then we should not
>> have
>> > >>>>>>>>>
>> > >>>>>>>> storagePath or WalPath, and store them both under
>> > "persistencePath"
>> > >>>>>>>>
>> > >>>>>>>> root.
>> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
>> > this.
>> > >>>>>>>
>> > >>>>>>>>
>> > >>>>>>>>
>> > >
>> >
>>
>
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Igniters,

I've found one more shortcoming which we missed during the review. The
checkpoint page buffer is attributed to a data region, so originally it was
a mistake to place the checkpoint page buffer size to the global
configuration. This property should go to the data region configuration.

I will prepare a PR shortly so we can review the change.

2017-10-18 4:52 GMT+03:00 Denis Magda <dm...@gridgain.com>:

> Ivan,
>
> Please don’t forget to update all the persistence and memory pools related
> examples to the new configuration format. Let’s make sure non of our
> example prints out the class deprecated warning.
>
> Denis
>
> On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> > for our users to understand. We should also document this configuration
> on
> > readme.
> >
> > On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>> wrote:
> >
> > > Dmitriy,
> > >
> > > Please check description of https://issues.apache.org/jira
> > > /browse/IGNITE-6030, I've updated it with actual list of properties.
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > >
> > > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> > >
> > >> I am now confused. Can I please ask for the final configuration again?
> > >> What
> > >> will it look like?
> > >>
> > >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> > >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> > >>
> > >> Agree with Ivan. If we implemented backward compatibility, this would
> be
> > >>> completely counterintuitive behavior, so +1 to keep the behavior as
> is.
> > >>>
> > >>> As for the swap path, I see nothing wrong with having it for
> in-memory
> > >>> caches. This is a simple overflow mechanism that works fine if you do
> > not
> > >>> need persistence guarantees.
> > >>>
> > >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>>:
> > >>>
> > >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> > >>>> *walArchivePath*.
> > >>>>
> > >>>> Regarding persistencePath/storagePath, I don't like the idea when
> path
> > >>>>
> > >>> for
> > >>>
> > >>>> WAL is implicitly changed, especially when we have separate option
> for
> > >>>>
> > >>> it.
> > >>>
> > >>>> WAL and storage files are already located under same $IGNITE_HOME
> > root.
> > >>>>  From user perspective, there's no need to change root for all
> > >>>> persistence-related directories as long as $IGNITE_HOME points to
> the
> > >>>> correct disk.
> > >>>>  From developer perspective, this change breaks backwards
> > compatibility.
> > >>>> Maintaining backwards compatibility in fail-safe way (checking both
> > >>>> old-style and new-style paths) is complex and hard to maintain in
> the
> > >>>> codebase.
> > >>>>
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> My vote is for *storagePath* and keeping behavior as is.
> > >>>>
> > >>>>
> > >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > >>>>
> > >>>> Igniters, another thing to consider:
> > >>>>>
> > >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> > >>>>> since this is actually not a single file, but a directory path.
> > >>>>>
> > >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
> > <javascript:;>>
> > >>>>> wrote:
> > >>>>>
> > >>>>> Seems I've got what you’re talking about.
> > >>>>>
> > >>>>>> I’ve tried to change the root directory (*persistencePath*) and
> saw
> > >>>>>>
> > >>>>> that
> > >>>
> > >>>> only data/indexes were placed to it while wal stayed somewhere in my
> > >>>>>>
> > >>>>> work
> > >>>
> > >>>> dir. It works counterintuitive and causes non productive discussions
> > >>>>>>
> > >>>>> like
> > >>>
> > >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> > >>>>>>
> > >>>>> name
> > >>>
> > >>>> fits this behavior.
> > >>>>>>
> > >>>>>> My suggestion will be the following:
> > >>>>>> - *persistencePath* refers to the path of all storage files
> > >>>>>> (data/indexes,
> > >>>>>> wal, archive). If the path is changed *all the files* will be
> under
> > >>>>>> the
> > >>>>>> new
> > >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> > >>>>>> *explicitly*.
> > >>>>>> - *setWalPath* overrides the default location of WAL (which is
> > >>>>>> setPersistencePath)
> > >>>>>> - *setWalArchivePath* overrides the default location of the
> archive
> > >>>>>> (which
> > >>>>>> is again has to be setPersistencePath).
> > >>>>>>
> > >>>>>> If we follow this approach the configuration and behavior becomes
> > >>>>>>
> > >>>>> vivid.
> > >>>
> > >>>> Thoughts?
> > >>>>>>
> > >>>>>> —
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
> > <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Denis,
> > >>>>>>>
> > >>>>>>> Data/index storage and WAL are located under the same root by
> > >>>>>>> default.
> > >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> > >>>>>>> properties
> > >>>>>>>
> > >>>>>>> can contain both absolute and relative paths. If paths are
> > absolute,
> > >>>>>> storage and WAL can reside on different devices, like this:
> > >>>>>>
> > >>>>>> storagePath: /storage1/NMVe_drive/storage
> > >>>>>>>
> > >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> > >>>>>>>>
> > >>>>>>>> We even recommend this in tuning guide:
> > https://apacheignite.readme
> > >>>>>>> .
> > >>>>>>>
> > >>>>>>> io/docs/durable-memory-tuning
> > >>>>>>
> > >>>>>> That's why I think *persistencePath* is misleading.
> > >>>>>>>
> > >>>>>>> Best Regards,
> > >>>>>>> Ivan Rakov
> > >>>>>>>
> > >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>>>>>
> > >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <
> dmagda@gridgain.com
> > <javascript:;>>
> > >>>>>>>>
> > >>>>>>>> wrote:
> > >>>>>>>   From what I see after running an example they are under the
> same
> > >>>>>>> root
> > >>>>>>>
> > >>>>>>>> folder and in different subdirectories. The root folder should
> be
> > >>>>>>>>>
> > >>>>>>>>> defined
> > >>>>>>>>
> > >>>>>>> by setPersistencePath as I guess.
> > >>>>>>>
> > >>>>>>>> If that is the case, then you are right. Then we should not have
> > >>>>>>>>>
> > >>>>>>>> storagePath or WalPath, and store them both under
> > "persistencePath"
> > >>>>>>>>
> > >>>>>>>> root.
> > >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> > this.
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >
> >
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@gridgain.com>]

Ivan,

Please don’t forget to update all the persistence and memory pools related
examples to the new configuration format. Let’s make sure non of our
example prints out the class deprecated warning.

Denis

On Tuesday, October 17, 2017, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> Thanks Ivan! Let's make sure that every property gets sufficient javadoc
> for our users to understand. We should also document this configuration on
> readme.
>
> On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>> wrote:
>
> > Dmitriy,
> >
> > Please check description of https://issues.apache.org/jira
> > /browse/IGNITE-6030, I've updated it with actual list of properties.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> >
> >> I am now confused. Can I please ask for the final configuration again?
> >> What
> >> will it look like?
> >>
> >> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> >>
> >> Agree with Ivan. If we implemented backward compatibility, this would be
> >>> completely counterintuitive behavior, so +1 to keep the behavior as is.
> >>>
> >>> As for the swap path, I see nothing wrong with having it for in-memory
> >>> caches. This is a simple overflow mechanism that works fine if you do
> not
> >>> need persistence guarantees.
> >>>
> >>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>:
> >>>
> >>> *swapPath* is ok for me. It is also consistent with *walPath* and
> >>>> *walArchivePath*.
> >>>>
> >>>> Regarding persistencePath/storagePath, I don't like the idea when path
> >>>>
> >>> for
> >>>
> >>>> WAL is implicitly changed, especially when we have separate option for
> >>>>
> >>> it.
> >>>
> >>>> WAL and storage files are already located under same $IGNITE_HOME
> root.
> >>>>  From user perspective, there's no need to change root for all
> >>>> persistence-related directories as long as $IGNITE_HOME points to the
> >>>> correct disk.
> >>>>  From developer perspective, this change breaks backwards
> compatibility.
> >>>> Maintaining backwards compatibility in fail-safe way (checking both
> >>>> old-style and new-style paths) is complex and hard to maintain in the
> >>>> codebase.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> My vote is for *storagePath* and keeping behavior as is.
> >>>>
> >>>>
> >>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >>>>
> >>>> Igniters, another thing to consider:
> >>>>>
> >>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >>>>> since this is actually not a single file, but a directory path.
> >>>>>
> >>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dmagda@apache.org
> <javascript:;>>
> >>>>> wrote:
> >>>>>
> >>>>> Seems I've got what you’re talking about.
> >>>>>
> >>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
> >>>>>>
> >>>>> that
> >>>
> >>>> only data/indexes were placed to it while wal stayed somewhere in my
> >>>>>>
> >>>>> work
> >>>
> >>>> dir. It works counterintuitive and causes non productive discussions
> >>>>>>
> >>>>> like
> >>>
> >>>> we are in arguing about *persistencePath* or *storagePath*. Neither
> >>>>>>
> >>>>> name
> >>>
> >>>> fits this behavior.
> >>>>>>
> >>>>>> My suggestion will be the following:
> >>>>>> - *persistencePath* refers to the path of all storage files
> >>>>>> (data/indexes,
> >>>>>> wal, archive). If the path is changed *all the files* will be under
> >>>>>> the
> >>>>>> new
> >>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>>>>> *explicitly*.
> >>>>>> - *setWalPath* overrides the default location of WAL (which is
> >>>>>> setPersistencePath)
> >>>>>> - *setWalArchivePath* overrides the default location of the archive
> >>>>>> (which
> >>>>>> is again has to be setPersistencePath).
> >>>>>>
> >>>>>> If we follow this approach the configuration and behavior becomes
> >>>>>>
> >>>>> vivid.
> >>>
> >>>> Thoughts?
> >>>>>>
> >>>>>> —
> >>>>>> Denis
> >>>>>>
> >>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Denis,
> >>>>>>>
> >>>>>>> Data/index storage and WAL are located under the same root by
> >>>>>>> default.
> >>>>>>> However, this is not mandatory: *storagePath* and *walPath*
> >>>>>>> properties
> >>>>>>>
> >>>>>>> can contain both absolute and relative paths. If paths are
> absolute,
> >>>>>> storage and WAL can reside on different devices, like this:
> >>>>>>
> >>>>>> storagePath: /storage1/NMVe_drive/storage
> >>>>>>>
> >>>>>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>>>>
> >>>>>>>> We even recommend this in tuning guide:
> https://apacheignite.readme
> >>>>>>> .
> >>>>>>>
> >>>>>>> io/docs/durable-memory-tuning
> >>>>>>
> >>>>>> That's why I think *persistencePath* is misleading.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>>>>
> >>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dmagda@gridgain.com
> <javascript:;>>
> >>>>>>>>
> >>>>>>>> wrote:
> >>>>>>>   From what I see after running an example they are under the same
> >>>>>>> root
> >>>>>>>
> >>>>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>>>>
> >>>>>>>>> defined
> >>>>>>>>
> >>>>>>> by setPersistencePath as I guess.
> >>>>>>>
> >>>>>>>> If that is the case, then you are right. Then we should not have
> >>>>>>>>>
> >>>>>>>> storagePath or WalPath, and store them both under
> "persistencePath"
> >>>>>>>>
> >>>>>>>> root.
> >>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> >>>>>>>
> >>>>>>>>
> >>>>>>>>
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Thanks Ivan! Let's make sure that every property gets sufficient javadoc
for our users to understand. We should also document this configuration on
readme.

On Tue, Oct 17, 2017 at 3:06 PM, Ivan Rakov <iv...@gmail.com> wrote:

> Dmitriy,
>
> Please check description of https://issues.apache.org/jira
> /browse/IGNITE-6030, I've updated it with actual list of properties.
>
> Best Regards,
> Ivan Rakov
>
>
> On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
>
>> I am now confused. Can I please ask for the final configuration again?
>> What
>> will it look like?
>>
>> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
>> alexey.goncharuk@gmail.com> wrote:
>>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>>
>>> As for the swap path, I see nothing wrong with having it for in-memory
>>> caches. This is a simple overflow mechanism that works fine if you do not
>>> need persistence guarantees.
>>>
>>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>>> *walArchivePath*.
>>>>
>>>> Regarding persistencePath/storagePath, I don't like the idea when path
>>>>
>>> for
>>>
>>>> WAL is implicitly changed, especially when we have separate option for
>>>>
>>> it.
>>>
>>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>>>  From user perspective, there's no need to change root for all
>>>> persistence-related directories as long as $IGNITE_HOME points to the
>>>> correct disk.
>>>>  From developer perspective, this change breaks backwards compatibility.
>>>> Maintaining backwards compatibility in fail-safe way (checking both
>>>> old-style and new-style paths) is complex and hard to maintain in the
>>>> codebase.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> My vote is for *storagePath* and keeping behavior as is.
>>>>
>>>>
>>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>>
>>>> Igniters, another thing to consider:
>>>>>
>>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>>> since this is actually not a single file, but a directory path.
>>>>>
>>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org>
>>>>> wrote:
>>>>>
>>>>> Seems I've got what you’re talking about.
>>>>>
>>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>>>>>>
>>>>> that
>>>
>>>> only data/indexes were placed to it while wal stayed somewhere in my
>>>>>>
>>>>> work
>>>
>>>> dir. It works counterintuitive and causes non productive discussions
>>>>>>
>>>>> like
>>>
>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>>>>>>
>>>>> name
>>>
>>>> fits this behavior.
>>>>>>
>>>>>> My suggestion will be the following:
>>>>>> - *persistencePath* refers to the path of all storage files
>>>>>> (data/indexes,
>>>>>> wal, archive). If the path is changed *all the files* will be under
>>>>>> the
>>>>>> new
>>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>>> *explicitly*.
>>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>>> setPersistencePath)
>>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>>> (which
>>>>>> is again has to be setPersistencePath).
>>>>>>
>>>>>> If we follow this approach the configuration and behavior becomes
>>>>>>
>>>>> vivid.
>>>
>>>> Thoughts?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Denis,
>>>>>>>
>>>>>>> Data/index storage and WAL are located under the same root by
>>>>>>> default.
>>>>>>> However, this is not mandatory: *storagePath* and *walPath*
>>>>>>> properties
>>>>>>>
>>>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>>> storage and WAL can reside on different devices, like this:
>>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>>
>>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>>
>>>>>>>> We even recommend this in tuning guide: https://apacheignite.readme
>>>>>>> .
>>>>>>>
>>>>>>> io/docs/durable-memory-tuning
>>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>>>>
>>>>>>>> wrote:
>>>>>>>   From what I see after running an example they are under the same
>>>>>>> root
>>>>>>>
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>>
>>>>>>>>> defined
>>>>>>>>
>>>>>>> by setPersistencePath as I guess.
>>>>>>>
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>>>>
>>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>>
>>>>>>>> root.
>>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>
>>>>>>>>
>>>>>>>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Dmitriy,

Please check description of 
https://issues.apache.org/jira/browse/IGNITE-6030, I've updated it with 
actual list of properties.

Best Regards,
Ivan Rakov

On 17.10.2017 21:46, Dmitriy Setrakyan wrote:
> I am now confused. Can I please ask for the final configuration again? What
> will it look like?
>
> On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Agree with Ivan. If we implemented backward compatibility, this would be
>> completely counterintuitive behavior, so +1 to keep the behavior as is.
>>
>> As for the swap path, I see nothing wrong with having it for in-memory
>> caches. This is a simple overflow mechanism that works fine if you do not
>> need persistence guarantees.
>>
>> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>
>>> *swapPath* is ok for me. It is also consistent with *walPath* and
>>> *walArchivePath*.
>>>
>>> Regarding persistencePath/storagePath, I don't like the idea when path
>> for
>>> WAL is implicitly changed, especially when we have separate option for
>> it.
>>> WAL and storage files are already located under same $IGNITE_HOME root.
>>>  From user perspective, there's no need to change root for all
>>> persistence-related directories as long as $IGNITE_HOME points to the
>>> correct disk.
>>>  From developer perspective, this change breaks backwards compatibility.
>>> Maintaining backwards compatibility in fail-safe way (checking both
>>> old-style and new-style paths) is complex and hard to maintain in the
>>> codebase.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> My vote is for *storagePath* and keeping behavior as is.
>>>
>>>
>>> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>>>
>>>> Igniters, another thing to consider:
>>>>
>>>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>>>> since this is actually not a single file, but a directory path.
>>>>
>>>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> Seems I've got what you’re talking about.
>>>>> I’ve tried to change the root directory (*persistencePath*) and saw
>> that
>>>>> only data/indexes were placed to it while wal stayed somewhere in my
>> work
>>>>> dir. It works counterintuitive and causes non productive discussions
>> like
>>>>> we are in arguing about *persistencePath* or *storagePath*. Neither
>> name
>>>>> fits this behavior.
>>>>>
>>>>> My suggestion will be the following:
>>>>> - *persistencePath* refers to the path of all storage files
>>>>> (data/indexes,
>>>>> wal, archive). If the path is changed *all the files* will be under the
>>>>> new
>>>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>>>> *explicitly*.
>>>>> - *setWalPath* overrides the default location of WAL (which is
>>>>> setPersistencePath)
>>>>> - *setWalArchivePath* overrides the default location of the archive
>>>>> (which
>>>>> is again has to be setPersistencePath).
>>>>>
>>>>> If we follow this approach the configuration and behavior becomes
>> vivid.
>>>>> Thoughts?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>> Denis,
>>>>>>
>>>>>> Data/index storage and WAL are located under the same root by default.
>>>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>>>
>>>>> can contain both absolute and relative paths. If paths are absolute,
>>>>> storage and WAL can reside on different devices, like this:
>>>>>
>>>>>> storagePath: /storage1/NMVe_drive/storage
>>>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>>>
>>>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>>>
>>>>> io/docs/durable-memory-tuning
>>>>>
>>>>>> That's why I think *persistencePath* is misleading.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>>>
>>>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>>>
>>>>>> wrote:
>>>>>>   From what I see after running an example they are under the same root
>>>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>>>
>>>>>>> defined
>>>>>> by setPersistencePath as I guess.
>>>>>>>> If that is the case, then you are right. Then we should not have
>>>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>>>
>>>>>> root.
>>>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>>>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

I am now confused. Can I please ask for the final configuration again? What
will it look like?

On Tue, Oct 17, 2017 at 1:16 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Agree with Ivan. If we implemented backward compatibility, this would be
> completely counterintuitive behavior, so +1 to keep the behavior as is.
>
> As for the swap path, I see nothing wrong with having it for in-memory
> caches. This is a simple overflow mechanism that works fine if you do not
> need persistence guarantees.
>
> 2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> for
> > WAL is implicitly changed, especially when we have separate option for
> it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> > From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> > From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> >
> >> Igniters, another thing to consider:
> >>
> >> DataRegionConfiguration.SwapFilePath should be SwapPath,
> >> since this is actually not a single file, but a directory path.
> >>
> >> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
> >>
> >> Seems I've got what you’re talking about.
> >>>
> >>> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> >>> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >>> dir. It works counterintuitive and causes non productive discussions
> like
> >>> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> >>> fits this behavior.
> >>>
> >>> My suggestion will be the following:
> >>> - *persistencePath* refers to the path of all storage files
> >>> (data/indexes,
> >>> wal, archive). If the path is changed *all the files* will be under the
> >>> new
> >>> directory unless *setWalPath* and *setWalArchivePath* are set
> >>> *explicitly*.
> >>> - *setWalPath* overrides the default location of WAL (which is
> >>> setPersistencePath)
> >>> - *setWalArchivePath* overrides the default location of the archive
> >>> (which
> >>> is again has to be setPersistencePath).
> >>>
> >>> If we follow this approach the configuration and behavior becomes
> vivid.
> >>> Thoughts?
> >>>
> >>> —
> >>> Denis
> >>>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>>
> >>>> Denis,
> >>>>
> >>>> Data/index storage and WAL are located under the same root by default.
> >>>> However, this is not mandatory: *storagePath* and *walPath* properties
> >>>>
> >>> can contain both absolute and relative paths. If paths are absolute,
> >>> storage and WAL can reside on different devices, like this:
> >>>
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>>> walPath: /storage2/Big_SSD_drive/wal
> >>>>>
> >>>> We even recommend this in tuning guide: https://apacheignite.readme.
> >>>>
> >>> io/docs/durable-memory-tuning
> >>>
> >>>> That's why I think *persistencePath* is misleading.
> >>>>
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>>
> >>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> >>>>>
> >>>> wrote:
> >>>
> >>>>  From what I see after running an example they are under the same root
> >>>>>> folder and in different subdirectories. The root folder should be
> >>>>>>
> >>>>> defined
> >>>
> >>>> by setPersistencePath as I guess.
> >>>>>>
> >>>>>> If that is the case, then you are right. Then we should not have
> >>>>> storagePath or WalPath, and store them both under "persistencePath"
> >>>>>
> >>>> root.
> >>>
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>>
> >>>>>
> >>>
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Agree with Ivan. If we implemented backward compatibility, this would be
completely counterintuitive behavior, so +1 to keep the behavior as is.

As for the swap path, I see nothing wrong with having it for in-memory
caches. This is a simple overflow mechanism that works fine if you do not
need persistence guarantees.

2017-10-16 21:00 GMT+03:00 Ivan Rakov <iv...@gmail.com>:

> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path for
> WAL is implicitly changed, especially when we have separate option for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
> From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
> From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
>
>> Igniters, another thing to consider:
>>
>> DataRegionConfiguration.SwapFilePath should be SwapPath,
>> since this is actually not a single file, but a directory path.
>>
>> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>>
>> Seems I've got what you’re talking about.
>>>
>>> I’ve tried to change the root directory (*persistencePath*) and saw that
>>> only data/indexes were placed to it while wal stayed somewhere in my work
>>> dir. It works counterintuitive and causes non productive discussions like
>>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>>> fits this behavior.
>>>
>>> My suggestion will be the following:
>>> - *persistencePath* refers to the path of all storage files
>>> (data/indexes,
>>> wal, archive). If the path is changed *all the files* will be under the
>>> new
>>> directory unless *setWalPath* and *setWalArchivePath* are set
>>> *explicitly*.
>>> - *setWalPath* overrides the default location of WAL (which is
>>> setPersistencePath)
>>> - *setWalArchivePath* overrides the default location of the archive
>>> (which
>>> is again has to be setPersistencePath).
>>>
>>> If we follow this approach the configuration and behavior becomes vivid.
>>> Thoughts?
>>>
>>> —
>>> Denis
>>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Denis,
>>>>
>>>> Data/index storage and WAL are located under the same root by default.
>>>> However, this is not mandatory: *storagePath* and *walPath* properties
>>>>
>>> can contain both absolute and relative paths. If paths are absolute,
>>> storage and WAL can reside on different devices, like this:
>>>
>>>> storagePath: /storage1/NMVe_drive/storage
>>>>> walPath: /storage2/Big_SSD_drive/wal
>>>>>
>>>> We even recommend this in tuning guide: https://apacheignite.readme.
>>>>
>>> io/docs/durable-memory-tuning
>>>
>>>> That's why I think *persistencePath* is misleading.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>>
>>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>>>>>
>>>> wrote:
>>>
>>>>  From what I see after running an example they are under the same root
>>>>>> folder and in different subdirectories. The root folder should be
>>>>>>
>>>>> defined
>>>
>>>> by setPersistencePath as I guess.
>>>>>>
>>>>>> If that is the case, then you are right. Then we should not have
>>>>> storagePath or WalPath, and store them both under "persistencePath"
>>>>>
>>>> root.
>>>
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>>
>>>>>
>>>
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Igniters, we should not have Swap in the property names. Swap is something
that behaves completely differently from Ignite persistence, and I don't
think it should be present within Ignite altogether.

D.

On Mon, Oct 16, 2017 at 11:17 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> +1 to Ivan's suggestion.
>
> пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <iv...@gmail.com>:
>
> > *swapPath* is ok for me. It is also consistent with *walPath* and
> > *walArchivePath*.
> >
> > Regarding persistencePath/storagePath, I don't like the idea when path
> > for WAL is implicitly changed, especially when we have separate option
> > for it.
> > WAL and storage files are already located under same $IGNITE_HOME root.
> >  From user perspective, there's no need to change root for all
> > persistence-related directories as long as $IGNITE_HOME points to the
> > correct disk.
> >  From developer perspective, this change breaks backwards compatibility.
> > Maintaining backwards compatibility in fail-safe way (checking both
> > old-style and new-style paths) is complex and hard to maintain in the
> > codebase.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > My vote is for *storagePath* and keeping behavior as is.
> >
> > On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > > Igniters, another thing to consider:
> > >
> > > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > > since this is actually not a single file, but a directory path.
> > >
> > > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > >> Seems I've got what you’re talking about.
> > >>
> > >> I’ve tried to change the root directory (*persistencePath*) and saw
> that
> > >> only data/indexes were placed to it while wal stayed somewhere in my
> > work
> > >> dir. It works counterintuitive and causes non productive discussions
> > like
> > >> we are in arguing about *persistencePath* or *storagePath*. Neither
> name
> > >> fits this behavior.
> > >>
> > >> My suggestion will be the following:
> > >> - *persistencePath* refers to the path of all storage files
> > (data/indexes,
> > >> wal, archive). If the path is changed *all the files* will be under
> the
> > new
> > >> directory unless *setWalPath* and *setWalArchivePath* are set
> > *explicitly*.
> > >> - *setWalPath* overrides the default location of WAL (which is
> > >> setPersistencePath)
> > >> - *setWalArchivePath* overrides the default location of the archive
> > (which
> > >> is again has to be setPersistencePath).
> > >>
> > >> If we follow this approach the configuration and behavior becomes
> vivid.
> > >> Thoughts?
> > >>
> > >> —
> > >> Denis
> > >>
> > >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com>
> wrote:
> > >>>
> > >>> Denis,
> > >>>
> > >>> Data/index storage and WAL are located under the same root by
> default.
> > >>> However, this is not mandatory: *storagePath* and *walPath*
> properties
> > >> can contain both absolute and relative paths. If paths are absolute,
> > >> storage and WAL can reside on different devices, like this:
> > >>>> storagePath: /storage1/NMVe_drive/storage
> > >>>> walPath: /storage2/Big_SSD_drive/wal
> > >>> We even recommend this in tuning guide: https://apacheignite.readme.
> > >> io/docs/durable-memory-tuning
> > >>> That's why I think *persistencePath* is misleading.
> > >>>
> > >>> Best Regards,
> > >>> Ivan Rakov
> > >>>
> > >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> > >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> > >> wrote:
> > >>>>>  From what I see after running an example they are under the same
> > root
> > >>>>> folder and in different subdirectories. The root folder should be
> > >> defined
> > >>>>> by setPersistencePath as I guess.
> > >>>>>
> > >>>> If that is the case, then you are right. Then we should not have
> > >>>> storagePath or WalPath, and store them both under "persistencePath"
> > >> root.
> > >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm
> this.
> > >>>>
> > >>
> >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

+1 to Ivan's suggestion.

пн, 16 окт. 2017 г. в 21:00, Ivan Rakov <iv...@gmail.com>:

> *swapPath* is ok for me. It is also consistent with *walPath* and
> *walArchivePath*.
>
> Regarding persistencePath/storagePath, I don't like the idea when path
> for WAL is implicitly changed, especially when we have separate option
> for it.
> WAL and storage files are already located under same $IGNITE_HOME root.
>  From user perspective, there's no need to change root for all
> persistence-related directories as long as $IGNITE_HOME points to the
> correct disk.
>  From developer perspective, this change breaks backwards compatibility.
> Maintaining backwards compatibility in fail-safe way (checking both
> old-style and new-style paths) is complex and hard to maintain in the
> codebase.
>
> Best Regards,
> Ivan Rakov
>
> My vote is for *storagePath* and keeping behavior as is.
>
> On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> > Igniters, another thing to consider:
> >
> > DataRegionConfiguration.SwapFilePath should be SwapPath,
> > since this is actually not a single file, but a directory path.
> >
> > On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
> >
> >> Seems I've got what you’re talking about.
> >>
> >> I’ve tried to change the root directory (*persistencePath*) and saw that
> >> only data/indexes were placed to it while wal stayed somewhere in my
> work
> >> dir. It works counterintuitive and causes non productive discussions
> like
> >> we are in arguing about *persistencePath* or *storagePath*. Neither name
> >> fits this behavior.
> >>
> >> My suggestion will be the following:
> >> - *persistencePath* refers to the path of all storage files
> (data/indexes,
> >> wal, archive). If the path is changed *all the files* will be under the
> new
> >> directory unless *setWalPath* and *setWalArchivePath* are set
> *explicitly*.
> >> - *setWalPath* overrides the default location of WAL (which is
> >> setPersistencePath)
> >> - *setWalArchivePath* overrides the default location of the archive
> (which
> >> is again has to be setPersistencePath).
> >>
> >> If we follow this approach the configuration and behavior becomes vivid.
> >> Thoughts?
> >>
> >> —
> >> Denis
> >>
> >>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>
> >>> Denis,
> >>>
> >>> Data/index storage and WAL are located under the same root by default.
> >>> However, this is not mandatory: *storagePath* and *walPath* properties
> >> can contain both absolute and relative paths. If paths are absolute,
> >> storage and WAL can reside on different devices, like this:
> >>>> storagePath: /storage1/NMVe_drive/storage
> >>>> walPath: /storage2/Big_SSD_drive/wal
> >>> We even recommend this in tuning guide: https://apacheignite.readme.
> >> io/docs/durable-memory-tuning
> >>> That's why I think *persistencePath* is misleading.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> >> wrote:
> >>>>>  From what I see after running an example they are under the same
> root
> >>>>> folder and in different subdirectories. The root folder should be
> >> defined
> >>>>> by setPersistencePath as I guess.
> >>>>>
> >>>> If that is the case, then you are right. Then we should not have
> >>>> storagePath or WalPath, and store them both under "persistencePath"
> >> root.
> >>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>>>
> >>
>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

*swapPath* is ok for me. It is also consistent with *walPath* and 
*walArchivePath*.

Regarding persistencePath/storagePath, I don't like the idea when path 
for WAL is implicitly changed, especially when we have separate option 
for it.
WAL and storage files are already located under same $IGNITE_HOME root. 
 From user perspective, there's no need to change root for all 
persistence-related directories as long as $IGNITE_HOME points to the 
correct disk.
 From developer perspective, this change breaks backwards compatibility. 
Maintaining backwards compatibility in fail-safe way (checking both 
old-style and new-style paths) is complex and hard to maintain in the 
codebase.

Best Regards,
Ivan Rakov

My vote is for *storagePath* and keeping behavior as is.

On 16.10.2017 16:53, Pavel Tupitsyn wrote:
> Igniters, another thing to consider:
>
> DataRegionConfiguration.SwapFilePath should be SwapPath,
> since this is actually not a single file, but a directory path.
>
> On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Seems I've got what you’re talking about.
>>
>> I’ve tried to change the root directory (*persistencePath*) and saw that
>> only data/indexes were placed to it while wal stayed somewhere in my work
>> dir. It works counterintuitive and causes non productive discussions like
>> we are in arguing about *persistencePath* or *storagePath*. Neither name
>> fits this behavior.
>>
>> My suggestion will be the following:
>> - *persistencePath* refers to the path of all storage files (data/indexes,
>> wal, archive). If the path is changed *all the files* will be under the new
>> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
>> - *setWalPath* overrides the default location of WAL (which is
>> setPersistencePath)
>> - *setWalArchivePath* overrides the default location of the archive (which
>> is again has to be setPersistencePath).
>>
>> If we follow this approach the configuration and behavior becomes vivid.
>> Thoughts?
>>
>> —
>> Denis
>>
>>> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis,
>>>
>>> Data/index storage and WAL are located under the same root by default.
>>> However, this is not mandatory: *storagePath* and *walPath* properties
>> can contain both absolute and relative paths. If paths are absolute,
>> storage and WAL can reside on different devices, like this:
>>>> storagePath: /storage1/NMVe_drive/storage
>>>> walPath: /storage2/Big_SSD_drive/wal
>>> We even recommend this in tuning guide: https://apacheignite.readme.
>> io/docs/durable-memory-tuning
>>> That's why I think *persistencePath* is misleading.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>>>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
>> wrote:
>>>>>  From what I see after running an example they are under the same root
>>>>> folder and in different subdirectories. The root folder should be
>> defined
>>>>> by setPersistencePath as I guess.
>>>>>
>>>> If that is the case, then you are right. Then we should not have
>>>> storagePath or WalPath, and store them both under "persistencePath"
>> root.
>>>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>>>>
>>

Re: Persistence per memory policy configuration

[Pavel Tupitsyn <pt...@apache.org>]

Igniters, another thing to consider:

DataRegionConfiguration.SwapFilePath should be SwapPath,
since this is actually not a single file, but a directory path.

On Fri, Oct 13, 2017 at 7:53 PM, Denis Magda <dm...@apache.org> wrote:

> Seems I've got what you’re talking about.
>
> I’ve tried to change the root directory (*persistencePath*) and saw that
> only data/indexes were placed to it while wal stayed somewhere in my work
> dir. It works counterintuitive and causes non productive discussions like
> we are in arguing about *persistencePath* or *storagePath*. Neither name
> fits this behavior.
>
> My suggestion will be the following:
> - *persistencePath* refers to the path of all storage files (data/indexes,
> wal, archive). If the path is changed *all the files* will be under the new
> directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
> - *setWalPath* overrides the default location of WAL (which is
> setPersistencePath)
> - *setWalArchivePath* overrides the default location of the archive (which
> is again has to be setPersistencePath).
>
> If we follow this approach the configuration and behavior becomes vivid.
> Thoughts?
>
> —
> Denis
>
> > On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Denis,
> >
> > Data/index storage and WAL are located under the same root by default.
> > However, this is not mandatory: *storagePath* and *walPath* properties
> can contain both absolute and relative paths. If paths are absolute,
> storage and WAL can reside on different devices, like this:
> >
> >> storagePath: /storage1/NMVe_drive/storage
> >> walPath: /storage2/Big_SSD_drive/wal
> > We even recommend this in tuning guide: https://apacheignite.readme.
> io/docs/durable-memory-tuning
> >
> > That's why I think *persistencePath* is misleading.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> >> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com>
> wrote:
> >>
> >>> From what I see after running an example they are under the same root
> >>> folder and in different subdirectories. The root folder should be
> defined
> >>> by setPersistencePath as I guess.
> >>>
> >> If that is the case, then you are right. Then we should not have
> >> storagePath or WalPath, and store them both under "persistencePath"
> root.
> >> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
> >>
> >
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Seems I've got what you’re talking about. 

I’ve tried to change the root directory (*persistencePath*) and saw that only data/indexes were placed to it while wal stayed somewhere in my work dir. It works counterintuitive and causes non productive discussions like we are in arguing about *persistencePath* or *storagePath*. Neither name fits this behavior.

My suggestion will be the following:
- *persistencePath* refers to the path of all storage files (data/indexes, wal, archive). If the path is changed *all the files* will be under the new directory unless *setWalPath* and *setWalArchivePath* are set *explicitly*.
- *setWalPath* overrides the default location of WAL (which is setPersistencePath)
- *setWalArchivePath* overrides the default location of the archive (which is again has to be setPersistencePath).

If we follow this approach the configuration and behavior becomes vivid. Thoughts?

—
Denis

> On Oct 13, 2017, at 1:21 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Denis,
> 
> Data/index storage and WAL are located under the same root by default.
> However, this is not mandatory: *storagePath* and *walPath* properties can contain both absolute and relative paths. If paths are absolute, storage and WAL can reside on different devices, like this:
> 
>> storagePath: /storage1/NMVe_drive/storage
>> walPath: /storage2/Big_SSD_drive/wal
> We even recommend this in tuning guide: https://apacheignite.readme.io/docs/durable-memory-tuning
> 
> That's why I think *persistencePath* is misleading.
> 
> Best Regards,
> Ivan Rakov
> 
> On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
>> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:
>> 
>>> From what I see after running an example they are under the same root
>>> folder and in different subdirectories. The root folder should be defined
>>> by setPersistencePath as I guess.
>>> 
>> If that is the case, then you are right. Then we should not have
>> storagePath or WalPath, and store them both under "persistencePath" root.
>> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Denis,

Data/index storage and WAL are located under the same root by default.
However, this is not mandatory: *storagePath* and *walPath* properties 
can contain both absolute and relative paths. If paths are absolute, 
storage and WAL can reside on different devices, like this:

> storagePath: /storage1/NMVe_drive/storage
> walPath: /storage2/Big_SSD_drive/wal
We even recommend this in tuning guide: 
https://apacheignite.readme.io/docs/durable-memory-tuning

That's why I think *persistencePath* is misleading.

Best Regards,
Ivan Rakov

On 13.10.2017 5:03, Dmitriy Setrakyan wrote:
> On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:
>
>>  From what I see after running an example they are under the same root
>> folder and in different subdirectories. The root folder should be defined
>> by setPersistencePath as I guess.
>>
> If that is the case, then you are right. Then we should not have
> storagePath or WalPath, and store them both under "persistencePath" root.
> However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

On Thu, Oct 12, 2017 at 7:01 PM, Denis Magda <dm...@gridgain.com> wrote:

> From what I see after running an example they are under the same root
> folder and in different subdirectories. The root folder should be defined
> by setPersistencePath as I guess.
>

If that is the case, then you are right. Then we should not have
storagePath or WalPath, and store them both under "persistencePath" root.
However, I would need Alexey Goncharuk or Ivan Rakov to confirm this.

Re: Persistence per memory policy configuration

[Denis Magda <dm...@gridgain.com>]

From what I see after running an example they are under the same root
folder and in different subdirectories. The root folder should be defined
by setPersistencePath as I guess.

Denis

On Thursday, October 12, 2017, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Thu, Oct 12, 2017 at 4:05 PM, Denis Magda <dmagda@apache.org
> <javascript:;>> wrote:
>
> > Both terms “persistence” and “storage” mean same to me. If to choose
> > between them I would go for “persistence” because this is how
> > we name the feature - “Ignite Persistence” [1]
>
>
> > Furthermore, by default WAL files, the archive and data/index files are
> > located under the same root which is “setPersistencePath” directory.
> >
>
> Denis, to my knowledge, WAL can span multiple caches, no? How can it end up
> in the same folder with data and indexes, if those are per cache?
>
>
> >
> > [1] https://ignite.apache.org/features/persistence.html <
> > https://ignite.apache.org/features/persistence.html>
> >
> > —
> > Denis
> >
> > > On Oct 12, 2017, at 12:00 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>> wrote:
> > >
> > > Name *setPersistencePath* looks confusing to me because both WAL and
> > index/partition files storage provide persistence. That's why we
> separated
> > API methods as *setWalPath* and *setStoragePath*.
> > > I think, *setStoragePath* is good enough as long as it's supported by
> > explaining javadoc.*
> > > *
> > >
> > > Best Regards,
> > > Ivan Rakov
> > >
> > > On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> > >> Is the storage path the root folder for the persistence or only the
> root
> > >> path for the main storage?
> > >>
> > >> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dmagda@apache.org
> <javascript:;>> wrote:
> > >>
> > >>> Ivan,
> > >>>
> > >>> Instead of “setStoragePath” I would suggest “setPersistencePath”.
> Left
> > >>> some extra notes in the ticket.
> > >>>
> > >>
> > >>> —
> > >>> Denis
> > >>>
> > >>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <ivan.glukos@gmail.com
> <javascript:;>>
> > wrote:
> > >>>>
> > >>>> Vladimir,
> > >>>>
> > >>>> Thanks for focusing on existing namings. Most of your suggestions
> > really
> > >>> sound better. I've posted my thoughts under your comment.
> > >>>> By the way, we should decide two things:
> > >>>>
> > >>>> 1) Naming for methods for configuring store path. I suggest the
> > >>> following:
> > >>>> *setStoragePath* - for partition and index files
> > >>>> *setWalPath* - for WAL files
> > >>>> *walArchivePath* - for WAL archive files
> > >>>>
> > >>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same
> > with
> > >>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both
> options
> > >>> sounds ok to me, let's see what community thinks.
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
> > >>>>> Ivan,
> > >>>>>
> > >>>>> I left some comments in the ticket [1], please take a look.
> > >>>>>
> > >>>>> [1]
> > >>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
> > >>> focusedCommentId=16200050&page=com.atlassian.jira.
> > >>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> > >>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>>
> > >>> wrote:
> > >>>>>> Igniters,
> > >>>>>>
> > >>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> > >>> enqueued
> > >>>>>> for TC run.
> > >>>>>> PR: https://github.com/apache/ignite/pull/2828
> > >>>>>>
> > >>>>>> Everyone interested in new data storage configuration API, please
> > pay
> > >>>>>> attention and review.
> > >>>>>>
> > >>>>>>
> > >>>>>> Best Regards,
> > >>>>>> Ivan Rakov
> > >>>>>>
> > >>>>>>
> > >>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> > >>>>>>
> > >>>>>>> Sounds good to me.
> > >>>>>>>
> > >>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>
> > >
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Pavel,
> > >>>>>>>> Sounds reasonable.
> > >>>>>>>> I suggest to include both "data" and "configuration" to make it
> > even
> > >>> more
> > >>>>>>>> obvious:
> > >>>>>>>>
> > >>>>>>>> set/getDefaultDataRegionConfiguration
> > >>>>>>>> set/getDataRegionConfigurations
> > >>>>>>>>
> > >>>>>>>> Best Regards,
> > >>>>>>>> Ivan Rakov
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> > >>>>>>>>
> > >>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
> > >>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
> > >>>>>>>>> DataRegionConfiguration[] getDataRegions
> > >>>>>>>>>
> > >>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
> > >>>>>>>>> - or -
> > >>>>>>>>> defaultDataRegion + getDataRegions
> > >>>>>>>>>
> > >>>>>>>>> Thoughts?
> > >>>>>>>>>
> > >>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Denis,
> > >>>>>>>>>
> > >>>>>>>>>> Yes, you're right. All cache groups without specific data
> region
> > >>>>>>>>>> configured will be persistent.
> > >>>>>>>>>> And if you want to add another persistent data region, you
> > should
> > >>> set
> > >>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> > >>> explictly.
> > >>>>>>>>>> Best Regards,
> > >>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to
> > wrap
> > >>> this
> > >>>>>>>>>>> up, if I want to enable the persistence globally and don’t
> have
> > >>> any
> > >>>>>>>>>>> regions
> > >>>>>>>>>>> configured explicitly I need to take the default region and
> > >>> switch the
> > >>>>>>>>>>> persistence on for it. Is my understanding correct?
> > >>>>>>>>>>>
> > >>>>>>>>>>> —
> > >>>>>>>>>>> Denis
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <
> ivan.glukos@gmail.com <javascript:;>
> > >
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Denis, why do you need to access an instance of the default
> > region
> > >>>>>>>>>>>> bean?
> > >>>>>>>>>>>> If you want to set any parameter, just instantiate new bean
> > with
> > >>> this
> > >>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters
> > will
> > >>> be
> > >>>>>>>>>>>> automatically initialized with their default values.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>         <property name="dataStorageConfiguration">
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> > >>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> > >>>>>>>>>>>>>>> value="#{100
> > >>>>>>>>>>>>>>> *
> > >>>>>>>>>>>>>>> 1024 * 1024}"/>
> > >>>>>>>>>>>>>>>                 <property name="
> > defaultRegionConfiguration">
> > >>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> > >>>>>>>>>>>>>>>                         <property name="maxSize"
> > value="#{5 *
> > >>>>>>>>>>>>>>> 1024 *
> > >>>>>>>>>>>>>>> 102 * 1024}"/>
> > >>>>>>>>>>>>>>>                     </bean>
> > >>>>>>>>>>>>>>>                 </property>
> > >>>>>>>>>>>>>>>             </bean>
> > >>>>>>>>>>>>>>>         </property>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> > default.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean
> and
> > >>> change
> > >>>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the
> > persistence I
> > >>>>>>>>>>>>> don’t want
> > >>>>>>>>>>>>> to create the default region bean from scratch.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —
> > >>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Agree with Alexey.
> > >>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
> > >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>> can confuse users who don't know that there's such thing
> as
> > >>> default
> > >>>>>>>>>>>>>> data
> > >>>>>>>>>>>>>> region. They can decide they are inherited by all data
> > regions
> > >>>>>>>>>>>>>> where
> > >>>>>>>>>>>>>> size
> > >>>>>>>>>>>>>> and persistence flag are not explicitly set.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Let's get rid of these properties and add
> > >>>>>>>>>>>>>> *defaultRegionConfiguration*
> > >>>>>>>>>>>>>> property with explicit configuration of default data
> region.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence
> > flag
> > >>> of
> > >>>>>>>>>>>>>> default data region will be just two lines longer (for
> bean
> > >>>>>>>>>>>>>> description):
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>         <property name="dataStorageConfiguration">
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> > >>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> > >>>>>>>>>>>>>>> value="#{100
> > >>>>>>>>>>>>>>> *
> > >>>>>>>>>>>>>>> 1024 * 1024}"/>
> > >>>>>>>>>>>>>>>                 <property name="
> > defaultRegionConfiguration">
> > >>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> > >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> > >>>>>>>>>>>>>>>                         <property name="maxSize"
> > value="#{5 *
> > >>>>>>>>>>>>>>> 1024 *
> > >>>>>>>>>>>>>>> 102 * 1024}"/>
> > >>>>>>>>>>>>>>>                     </bean>
> > >>>>>>>>>>>>>>>                 </property>
> > >>>>>>>>>>>>>>>             </bean>
> > >>>>>>>>>>>>>>>         </property>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> > default.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> > >>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> To resolve this, I suggest to
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> and
> > >>> get
> > >>>>>>>>>>>>>>>> rid
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> > >>> file? I’m
> > >>>>>>>>>>>>>>> not
> > >>>>>>>>>>>>>>> an expert in Spring so how do I get
> > defaultRegionConfiguration
> > >>>>>>>>>>>>>>> bean
> > >>>>>>>>>>>>>>> first
> > >>>>>>>>>>>>>>> to change any parameter?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> —
> > >>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com <javascript:;>> wrote:
> > >>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we
> would
> > >>> either
> > >>>>>>>>>>>>>>>> need to
> > >>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> > >>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this
> > field
> > >>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>> is also
> > >>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> > >>> defaults
> > >>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>> are
> > >>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> > >>> suggest
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> > and
> > >>> get
> > >>>>>>>>>>>>>>>> rid
> > >>>>>>>>>>>>>>>> of
> > >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Thoughts?
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <
> > ivan.glukos@gmail.com <javascript:;>
> > >>>> :
> > >>>>>>>>>>>>>>>> Vladimir,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
> > >>>>>>>>>>>>>>>>> However, user may be confused by setting
> > >>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by
> > default
> > >>> in
> > >>>>>>>>>>>>>>>>> custom memory
> > >>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Ivan,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I do not think this is correct approach, because it
> will
> > be
> > >>> hard
> > >>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> > >>>>>>>>>>>>>>>>>> "boolean"
> > >>>>>>>>>>>>>>>>>> for
> > >>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need
> default
> > >>>>>>>>>>>>>>>>>> "persistence
> > >>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> > >>> "persistence
> > >>>>>>>>>>>>>>>>>> enabled"
> > >>>>>>>>>>>>>>>>>> flag for default region only. It should not be
> > propagated
> > >>> to
> > >>>>>>>>>>>>>>>>>> custom
> > >>>>>>>>>>>>>>>>>> regions.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> > >>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com <javascript:;>
> > >>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Let's check the final design:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> > >>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
> > >>> enabling
> > >>>>>>>>>>>>>>>>>>> persistence
> > >>>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>> default data region.
> > >>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have
> > *isPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> property,
> > >>>>>>>>>>>>>>>>>>> which
> > >>>>>>>>>>>>>>>>>>> will be used for enabling persistence in
> corresponding
> > >>> data
> > >>>>>>>>>>>>>>>>>>> region. If
> > >>>>>>>>>>>>>>>>>>> value is not set, value of
> > *DataStorageConfiguration::isD
> > >>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> > >>>>>>>>>>>>>>>>>>> will be used by default.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Best Regards,
> > >>>>>>>>>>>>>>>>>>> Ivan Rakov
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> > >>>>>>>>>>>>>>>>>>> dmagda@apache.org <javascript:;>>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> > >>> ivan.glukos@gmail.com <javascript:;>
> > >>>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag
> in
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> > >>> Persistence
> > >>>>>>>>>>>>>>>>>>>>>> will be
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has
> > this
> > >>>>>>>>>>>>>>>>>>>>>> flag
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> set.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> > >>>>>>>>>>>>>>>>>>>>> *globally*
> > >>>>>>>>>>>>>>>>>>>>> if the
> > >>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If
> > it’s
> > >>>>>>>>>>>>>>>>>>>>> enabled for
> > >>>>>>>>>>>>>>>>>>>>> region
> > >>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for
> region
> > B.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will
> > be
> > >>>>>>>>>>>>>>>>>>>>> disabled
> > >>>>>>>>>>>>>>>>>>>>> globally.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> > >>> default
> > >>>>>>>>>>>>>>>>>>>>> behavior from
> > >>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>
> > >>>
> > >
> >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

On Thu, Oct 12, 2017 at 4:05 PM, Denis Magda <dm...@apache.org> wrote:

> Both terms “persistence” and “storage” mean same to me. If to choose
> between them I would go for “persistence” because this is how
> we name the feature - “Ignite Persistence” [1]


> Furthermore, by default WAL files, the archive and data/index files are
> located under the same root which is “setPersistencePath” directory.
>

Denis, to my knowledge, WAL can span multiple caches, no? How can it end up
in the same folder with data and indexes, if those are per cache?


>
> [1] https://ignite.apache.org/features/persistence.html <
> https://ignite.apache.org/features/persistence.html>
>
> —
> Denis
>
> > On Oct 12, 2017, at 12:00 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Name *setPersistencePath* looks confusing to me because both WAL and
> index/partition files storage provide persistence. That's why we separated
> API methods as *setWalPath* and *setStoragePath*.
> > I think, *setStoragePath* is good enough as long as it's supported by
> explaining javadoc.*
> > *
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> >> Is the storage path the root folder for the persistence or only the root
> >> path for the main storage?
> >>
> >> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
> >>
> >>> Ivan,
> >>>
> >>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
> >>> some extra notes in the ticket.
> >>>
> >>
> >>> —
> >>> Denis
> >>>
> >>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com>
> wrote:
> >>>>
> >>>> Vladimir,
> >>>>
> >>>> Thanks for focusing on existing namings. Most of your suggestions
> really
> >>> sound better. I've posted my thoughts under your comment.
> >>>> By the way, we should decide two things:
> >>>>
> >>>> 1) Naming for methods for configuring store path. I suggest the
> >>> following:
> >>>> *setStoragePath* - for partition and index files
> >>>> *setWalPath* - for WAL files
> >>>> *walArchivePath* - for WAL archive files
> >>>>
> >>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same
> with
> >>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
> >>> sounds ok to me, let's see what community thinks.
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
> >>>>> Ivan,
> >>>>>
> >>>>> I left some comments in the ticket [1], please take a look.
> >>>>>
> >>>>> [1]
> >>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
> >>> focusedCommentId=16200050&page=com.atlassian.jira.
> >>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> >>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
> >>> wrote:
> >>>>>> Igniters,
> >>>>>>
> >>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> >>> enqueued
> >>>>>> for TC run.
> >>>>>> PR: https://github.com/apache/ignite/pull/2828
> >>>>>>
> >>>>>> Everyone interested in new data storage configuration API, please
> pay
> >>>>>> attention and review.
> >>>>>>
> >>>>>>
> >>>>>> Best Regards,
> >>>>>> Ivan Rakov
> >>>>>>
> >>>>>>
> >>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> >>>>>>
> >>>>>>> Sounds good to me.
> >>>>>>>
> >>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <ivan.glukos@gmail.com
> >
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Pavel,
> >>>>>>>> Sounds reasonable.
> >>>>>>>> I suggest to include both "data" and "configuration" to make it
> even
> >>> more
> >>>>>>>> obvious:
> >>>>>>>>
> >>>>>>>> set/getDefaultDataRegionConfiguration
> >>>>>>>> set/getDataRegionConfigurations
> >>>>>>>>
> >>>>>>>> Best Regards,
> >>>>>>>> Ivan Rakov
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> >>>>>>>>
> >>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
> >>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
> >>>>>>>>> DataRegionConfiguration[] getDataRegions
> >>>>>>>>>
> >>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
> >>>>>>>>> - or -
> >>>>>>>>> defaultDataRegion + getDataRegions
> >>>>>>>>>
> >>>>>>>>> Thoughts?
> >>>>>>>>>
> >>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <
> ivan.glukos@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Denis,
> >>>>>>>>>
> >>>>>>>>>> Yes, you're right. All cache groups without specific data region
> >>>>>>>>>> configured will be persistent.
> >>>>>>>>>> And if you want to add another persistent data region, you
> should
> >>> set
> >>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> >>> explictly.
> >>>>>>>>>> Best Regards,
> >>>>>>>>>> Ivan Rakov
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> >>>>>>>>>>
> >>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to
> wrap
> >>> this
> >>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
> >>> any
> >>>>>>>>>>> regions
> >>>>>>>>>>> configured explicitly I need to take the default region and
> >>> switch the
> >>>>>>>>>>> persistence on for it. Is my understanding correct?
> >>>>>>>>>>>
> >>>>>>>>>>> —
> >>>>>>>>>>> Denis
> >>>>>>>>>>>
> >>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <ivan.glukos@gmail.com
> >
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Denis, why do you need to access an instance of the default
> region
> >>>>>>>>>>>> bean?
> >>>>>>>>>>>> If you want to set any parameter, just instantiate new bean
> with
> >>> this
> >>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters
> will
> >>> be
> >>>>>>>>>>>> automatically initialized with their default values.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>
> >>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>         <property name="dataStorageConfiguration">
> >>>>>>>>>>>>
> >>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> >>>>>>>>>>>>>>> value="#{100
> >>>>>>>>>>>>>>> *
> >>>>>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>>>>                 <property name="
> defaultRegionConfiguration">
> >>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>>>>                         <property name="maxSize"
> value="#{5 *
> >>>>>>>>>>>>>>> 1024 *
> >>>>>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>>>>                     </bean>
> >>>>>>>>>>>>>>>                 </property>
> >>>>>>>>>>>>>>>             </bean>
> >>>>>>>>>>>>>>>         </property>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> default.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
> >>> change
> >>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the
> persistence I
> >>>>>>>>>>>>> don’t want
> >>>>>>>>>>>>> to create the default region bean from scratch.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <
> ivan.glukos@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Agree with Alexey.
> >>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
> >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
> >>> default
> >>>>>>>>>>>>>> data
> >>>>>>>>>>>>>> region. They can decide they are inherited by all data
> regions
> >>>>>>>>>>>>>> where
> >>>>>>>>>>>>>> size
> >>>>>>>>>>>>>> and persistence flag are not explicitly set.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Let's get rid of these properties and add
> >>>>>>>>>>>>>> *defaultRegionConfiguration*
> >>>>>>>>>>>>>> property with explicit configuration of default data region.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence
> flag
> >>> of
> >>>>>>>>>>>>>> default data region will be just two lines longer (for bean
> >>>>>>>>>>>>>> description):
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>         <property name="dataStorageConfiguration">
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> >>>>>>>>>>>>>>> value="#{100
> >>>>>>>>>>>>>>> *
> >>>>>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>>>>                 <property name="
> defaultRegionConfiguration">
> >>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> >>>>>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>>>>                         <property name="maxSize"
> value="#{5 *
> >>>>>>>>>>>>>>> 1024 *
> >>>>>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>>>>                     </bean>
> >>>>>>>>>>>>>>>                 </property>
> >>>>>>>>>>>>>>>             </bean>
> >>>>>>>>>>>>>>>         </property>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> In other data regions persistence will be disabled by
> default.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> >>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> To resolve this, I suggest to
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> >>> get
> >>>>>>>>>>>>>>>> rid
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> >>> file? I’m
> >>>>>>>>>>>>>>> not
> >>>>>>>>>>>>>>> an expert in Spring so how do I get
> defaultRegionConfiguration
> >>>>>>>>>>>>>>> bean
> >>>>>>>>>>>>>>> first
> >>>>>>>>>>>>>>> to change any parameter?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
> >>> either
> >>>>>>>>>>>>>>>> need to
> >>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> >>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this
> field
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>> is also
> >>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> >>> defaults
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> are
> >>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> >>> suggest
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration
> and
> >>> get
> >>>>>>>>>>>>>>>> rid
> >>>>>>>>>>>>>>>> of
> >>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <
> ivan.glukos@gmail.com
> >>>> :
> >>>>>>>>>>>>>>>> Vladimir,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
> >>>>>>>>>>>>>>>>> However, user may be confused by setting
> >>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by
> default
> >>> in
> >>>>>>>>>>>>>>>>> custom memory
> >>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Ivan,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will
> be
> >>> hard
> >>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> >>>>>>>>>>>>>>>>>> "boolean"
> >>>>>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
> >>>>>>>>>>>>>>>>>> "persistence
> >>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> >>> "persistence
> >>>>>>>>>>>>>>>>>> enabled"
> >>>>>>>>>>>>>>>>>> flag for default region only. It should not be
> propagated
> >>> to
> >>>>>>>>>>>>>>>>>> custom
> >>>>>>>>>>>>>>>>>> regions.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> >>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Let's check the final design:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> >>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
> >>> enabling
> >>>>>>>>>>>>>>>>>>> persistence
> >>>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>> default data region.
> >>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have
> *isPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> property,
> >>>>>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
> >>> data
> >>>>>>>>>>>>>>>>>>> region. If
> >>>>>>>>>>>>>>>>>>> value is not set, value of
> *DataStorageConfiguration::isD
> >>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> >>>>>>>>>>>>>>>>>>> will be used by default.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> >>>>>>>>>>>>>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> >>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> >>> Persistence
> >>>>>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has
> this
> >>>>>>>>>>>>>>>>>>>>>> flag
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> set.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> >>>>>>>>>>>>>>>>>>>>> *globally*
> >>>>>>>>>>>>>>>>>>>>> if the
> >>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If
> it’s
> >>>>>>>>>>>>>>>>>>>>> enabled for
> >>>>>>>>>>>>>>>>>>>>> region
> >>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region
> B.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will
> be
> >>>>>>>>>>>>>>>>>>>>> disabled
> >>>>>>>>>>>>>>>>>>>>> globally.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> >>> default
> >>>>>>>>>>>>>>>>>>>>> behavior from
> >>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>
> >
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Both terms “persistence” and “storage” mean same to me. If to choose between them I would go for “persistence” because this is how 
we name the feature - “Ignite Persistence” [1].

Furthermore, by default WAL files, the archive and data/index files are located under the same root which is “setPersistencePath” directory. 

[1] https://ignite.apache.org/features/persistence.html <https://ignite.apache.org/features/persistence.html>
 
—
Denis

> On Oct 12, 2017, at 12:00 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Name *setPersistencePath* looks confusing to me because both WAL and index/partition files storage provide persistence. That's why we separated API methods as *setWalPath* and *setStoragePath*.
> I think, *setStoragePath* is good enough as long as it's supported by explaining javadoc.*
> *
> 
> Best Regards,
> Ivan Rakov
> 
> On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
>> Is the storage path the root folder for the persistence or only the root
>> path for the main storage?
>> 
>> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
>> 
>>> Ivan,
>>> 
>>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
>>> some extra notes in the ticket.
>>> 
>> 
>>> —
>>> Denis
>>> 
>>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>> 
>>>> Vladimir,
>>>> 
>>>> Thanks for focusing on existing namings. Most of your suggestions really
>>> sound better. I've posted my thoughts under your comment.
>>>> By the way, we should decide two things:
>>>> 
>>>> 1) Naming for methods for configuring store path. I suggest the
>>> following:
>>>> *setStoragePath* - for partition and index files
>>>> *setWalPath* - for WAL files
>>>> *walArchivePath* - for WAL archive files
>>>> 
>>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
>>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
>>> sounds ok to me, let's see what community thinks.
>>>> Best Regards,
>>>> Ivan Rakov
>>>> 
>>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>>>>> Ivan,
>>>>> 
>>>>> I left some comments in the ticket [1], please take a look.
>>>>> 
>>>>> [1]
>>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
>>> focusedCommentId=16200050&page=com.atlassian.jira.
>>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>>>>> Igniters,
>>>>>> 
>>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
>>> enqueued
>>>>>> for TC run.
>>>>>> PR: https://github.com/apache/ignite/pull/2828
>>>>>> 
>>>>>> Everyone interested in new data storage configuration API, please pay
>>>>>> attention and review.
>>>>>> 
>>>>>> 
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>> 
>>>>>> 
>>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>>>> 
>>>>>>> Sounds good to me.
>>>>>>> 
>>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>> Pavel,
>>>>>>>> Sounds reasonable.
>>>>>>>> I suggest to include both "data" and "configuration" to make it even
>>> more
>>>>>>>> obvious:
>>>>>>>> 
>>>>>>>> set/getDefaultDataRegionConfiguration
>>>>>>>> set/getDataRegionConfigurations
>>>>>>>> 
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>>>> 
>>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>>>> 
>>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>>>>> - or -
>>>>>>>>> defaultDataRegion + getDataRegions
>>>>>>>>> 
>>>>>>>>> Thoughts?
>>>>>>>>> 
>>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>> Denis,
>>>>>>>>> 
>>>>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>>>>> configured will be persistent.
>>>>>>>>>> And if you want to add another persistent data region, you should
>>> set
>>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
>>> explictly.
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>>>> 
>>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
>>> this
>>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
>>> any
>>>>>>>>>>> regions
>>>>>>>>>>> configured explicitly I need to take the default region and
>>> switch the
>>>>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>>>> 
>>>>>>>>>>> —
>>>>>>>>>>> Denis
>>>>>>>>>>> 
>>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>> 
>>>>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>>>>> bean?
>>>>>>>>>>>> If you want to set any parameter, just instantiate new bean with
>>> this
>>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters will
>>> be
>>>>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>>>> 
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>> 
>>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>>>>> 
>>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>>> *
>>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>>                     </bean>
>>>>>>>>>>>>>>>                 </property>
>>>>>>>>>>>>>>>             </bean>
>>>>>>>>>>>>>>>         </property>
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
>>> change
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>>>>> don’t want
>>>>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> —
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
>>> default
>>>>>>>>>>>>>> data
>>>>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>>>>> where
>>>>>>>>>>>>>> size
>>>>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
>>> of
>>>>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>>>>> description):
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>>> *
>>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>>                     </bean>
>>>>>>>>>>>>>>>                 </property>
>>>>>>>>>>>>>>>             </bean>
>>>>>>>>>>>>>>>         </property>
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>>> get
>>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
>>> file? I’m
>>>>>>>>>>>>>>> not
>>>>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>>>>> bean
>>>>>>>>>>>>>>> first
>>>>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
>>> either
>>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> is also
>>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
>>> defaults
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
>>> suggest
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>>> get
>>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>>>> :
>>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
>>> in
>>>>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will be
>>> hard
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
>>> "persistence
>>>>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>>>>> flag for default region only. It should not be propagated
>>> to
>>>>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
>>> enabling
>>>>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
>>> data
>>>>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
>>> Persistence
>>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
>>> default
>>>>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Name *setPersistencePath* looks confusing to me because both WAL and 
index/partition files storage provide persistence. That's why we 
separated API methods as *setWalPath* and *setStoragePath*.
I think, *setStoragePath* is good enough as long as it's supported by 
explaining javadoc.*
*

Best Regards,
Ivan Rakov

On 12.10.2017 8:05, Dmitriy Setrakyan wrote:
> Is the storage path the root folder for the persistence or only the root
> path for the main storage?
>
> On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:
>
>> Ivan,
>>
>> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
>> some extra notes in the ticket.
>>
>
>> —
>> Denis
>>
>>> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Vladimir,
>>>
>>> Thanks for focusing on existing namings. Most of your suggestions really
>> sound better. I've posted my thoughts under your comment.
>>> By the way, we should decide two things:
>>>
>>> 1) Naming for methods for configuring store path. I suggest the
>> following:
>>> *setStoragePath* - for partition and index files
>>> *setWalPath* - for WAL files
>>> *walArchivePath* - for WAL archive files
>>>
>>> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
>> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
>> sounds ok to me, let's see what community thinks.
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>>>> Ivan,
>>>>
>>>> I left some comments in the ticket [1], please take a look.
>>>>
>>>> [1]
>>>> https://issues.apache.org/jira/browse/IGNITE-6030?
>> focusedCommentId=16200050&page=com.atlassian.jira.
>> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>>>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>>>> Igniters,
>>>>>
>>>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
>> enqueued
>>>>> for TC run.
>>>>> PR: https://github.com/apache/ignite/pull/2828
>>>>>
>>>>> Everyone interested in new data storage configuration API, please pay
>>>>> attention and review.
>>>>>
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>>>
>>>>>> Sounds good to me.
>>>>>>
>>>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Pavel,
>>>>>>> Sounds reasonable.
>>>>>>> I suggest to include both "data" and "configuration" to make it even
>> more
>>>>>>> obvious:
>>>>>>>
>>>>>>> set/getDefaultDataRegionConfiguration
>>>>>>> set/getDataRegionConfigurations
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>>>
>>>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>>>
>>>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>>>> - or -
>>>>>>>> defaultDataRegion + getDataRegions
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis,
>>>>>>>>
>>>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>>>> configured will be persistent.
>>>>>>>>> And if you want to add another persistent data region, you should
>> set
>>>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
>> explictly.
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
>> this
>>>>>>>>>> up, if I want to enable the persistence globally and don’t have
>> any
>>>>>>>>>> regions
>>>>>>>>>> configured explicitly I need to take the default region and
>> switch the
>>>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>>>> bean?
>>>>>>>>>>> If you want to set any parameter, just instantiate new bean with
>> this
>>>>>>>>>>> parameter set (like in XML snipped below). Other parameters will
>> be
>>>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>>>
>>>>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>>>>
>>>>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>> *
>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>                      </bean>
>>>>>>>>>>>>>>                  </property>
>>>>>>>>>>>>>>              </bean>
>>>>>>>>>>>>>>          </property>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Ivan, how to get an instance to the default region bean and
>> change
>>>>>>>>>>>>> a
>>>>>>>>>>>>>
>>>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>>>> don’t want
>>>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>>>
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> can confuse users who don't know that there's such thing as
>> default
>>>>>>>>>>>>> data
>>>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>>>> where
>>>>>>>>>>>>> size
>>>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
>> of
>>>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>>>> description):
>>>>>>>>>>>>>
>>>>>>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>>>>>>
>>>>>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>>>>> value="#{100
>>>>>>>>>>>>>> *
>>>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>>>>> 1024 *
>>>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>>>                      </bean>
>>>>>>>>>>>>>>                  </property>
>>>>>>>>>>>>>>              </bean>
>>>>>>>>>>>>>>          </property>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>>>
>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>> get
>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
>> file? I’m
>>>>>>>>>>>>>> not
>>>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>>>> bean
>>>>>>>>>>>>>> first
>>>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
>> either
>>>>>>>>>>>>>>> need to
>>>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> is also
>>>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
>> defaults
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
>> suggest
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
>> get
>>>>>>>>>>>>>>> rid
>>>>>>>>>>>>>>> of
>>>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
>>> :
>>>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
>> in
>>>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I do not think this is correct approach, because it will be
>> hard
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
>> "persistence
>>>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>>>> flag for default region only. It should not be propagated
>> to
>>>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>> property (default = false), which will be used for
>> enabling
>>>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
>> data
>>>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
>> Persistence
>>>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
>> default
>>>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Is the storage path the root folder for the persistence or only the root
path for the main storage?

On Wed, Oct 11, 2017 at 3:54 PM, Denis Magda <dm...@apache.org> wrote:

> Ivan,
>
> Instead of “setStoragePath” I would suggest “setPersistencePath”. Left
> some extra notes in the ticket.
>


> —
> Denis
>
> > On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > Vladimir,
> >
> > Thanks for focusing on existing namings. Most of your suggestions really
> sound better. I've posted my thoughts under your comment.
> >
> > By the way, we should decide two things:
> >
> > 1) Naming for methods for configuring store path. I suggest the
> following:
> >
> > *setStoragePath* - for partition and index files
> > *setWalPath* - for WAL files
> > *walArchivePath* - for WAL archive files
> >
> > 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with
> *checkpointingPageBufferSize* and *checkpointingThreads*). Both options
> sounds ok to me, let's see what community thinks.
> >
> > Best Regards,
> > Ivan Rakov
> >
> > On 11.10.2017 14:05, Vladimir Ozerov wrote:
> >> Ivan,
> >>
> >> I left some comments in the ticket [1], please take a look.
> >>
> >> [1]
> >> https://issues.apache.org/jira/browse/IGNITE-6030?
> focusedCommentId=16200050&page=com.atlassian.jira.
> plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
> >>
> >> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com>
> wrote:
> >>
> >>> Igniters,
> >>>
> >>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and
> enqueued
> >>> for TC run.
> >>> PR: https://github.com/apache/ignite/pull/2828
> >>>
> >>> Everyone interested in new data storage configuration API, please pay
> >>> attention and review.
> >>>
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>>
> >>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> >>>
> >>>> Sounds good to me.
> >>>>
> >>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
> >>>> wrote:
> >>>>
> >>>> Pavel,
> >>>>> Sounds reasonable.
> >>>>> I suggest to include both "data" and "configuration" to make it even
> more
> >>>>> obvious:
> >>>>>
> >>>>> set/getDefaultDataRegionConfiguration
> >>>>> set/getDataRegionConfigurations
> >>>>>
> >>>>> Best Regards,
> >>>>> Ivan Rakov
> >>>>>
> >>>>>
> >>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> >>>>>
> >>>>> Sorry that I'm late to the party, but this looks inconsistent:
> >>>>>> DataStorageConfiguration defaultRegionConfiguration
> >>>>>> DataRegionConfiguration[] getDataRegions
> >>>>>>
> >>>>>> defaultRegionConfiguration + getRegionConfigurations
> >>>>>> - or -
> >>>>>> defaultDataRegion + getDataRegions
> >>>>>>
> >>>>>> Thoughts?
> >>>>>>
> >>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Denis,
> >>>>>>
> >>>>>>> Yes, you're right. All cache groups without specific data region
> >>>>>>> configured will be persistent.
> >>>>>>> And if you want to add another persistent data region, you should
> set
> >>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration*
> explictly.
> >>>>>>>
> >>>>>>> Best Regards,
> >>>>>>> Ivan Rakov
> >>>>>>>
> >>>>>>>
> >>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
> >>>>>>>
> >>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap
> this
> >>>>>>>
> >>>>>>>> up, if I want to enable the persistence globally and don’t have
> any
> >>>>>>>> regions
> >>>>>>>> configured explicitly I need to take the default region and
> switch the
> >>>>>>>> persistence on for it. Is my understanding correct?
> >>>>>>>>
> >>>>>>>> —
> >>>>>>>> Denis
> >>>>>>>>
> >>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Denis, why do you need to access an instance of the default region
> >>>>>>>>> bean?
> >>>>>>>>> If you want to set any parameter, just instantiate new bean with
> this
> >>>>>>>>> parameter set (like in XML snipped below). Other parameters will
> be
> >>>>>>>>> automatically initialized with their default values.
> >>>>>>>>>
> >>>>>>>>> Best Regards,
> >>>>>>>>> Ivan Rakov
> >>>>>>>>>
> >>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
> >>>>>>>>>
> >>>>>>>>>         <property name="dataStorageConfiguration">
> >>>>>>>>>
> >>>>>>>>>>             <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> >>>>>>>>>>>> value="#{100
> >>>>>>>>>>>> *
> >>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
> >>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
> >>>>>>>>>>>> 1024 *
> >>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>                     </bean>
> >>>>>>>>>>>>                 </property>
> >>>>>>>>>>>>             </bean>
> >>>>>>>>>>>>         </property>
> >>>>>>>>>>>>
> >>>>>>>>>>>> In other data regions persistence will be disabled by default.
> >>>>>>>>>>>>
> >>>>>>>>>>> Ivan, how to get an instance to the default region bean and
> change
> >>>>>>>>>>> a
> >>>>>>>>>>>
> >>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
> >>>>>>>>>> don’t want
> >>>>>>>>>> to create the default region bean from scratch.
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Agree with Alexey.
> >>>>>>>>>>> Properties like *defaultDataRegionSize*,
> >>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>> can confuse users who don't know that there's such thing as
> default
> >>>>>>>>>>> data
> >>>>>>>>>>> region. They can decide they are inherited by all data regions
> >>>>>>>>>>> where
> >>>>>>>>>>> size
> >>>>>>>>>>> and persistence flag are not explicitly set.
> >>>>>>>>>>>
> >>>>>>>>>>> Let's get rid of these properties and add
> >>>>>>>>>>> *defaultRegionConfiguration*
> >>>>>>>>>>> property with explicit configuration of default data region.
> >>>>>>>>>>>
> >>>>>>>>>>> Regarding XML configuration, changing size or persistence flag
> of
> >>>>>>>>>>> default data region will be just two lines longer (for bean
> >>>>>>>>>>> description):
> >>>>>>>>>>>
> >>>>>>>>>>>         <property name="dataStorageConfiguration">
> >>>>>>>>>>>
> >>>>>>>>>>>             <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataStorageConfiguration">
> >>>>>>>>>>>>                 <property name="systemCacheInitialSize"
> >>>>>>>>>>>> value="#{100
> >>>>>>>>>>>> *
> >>>>>>>>>>>> 1024 * 1024}"/>
> >>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
> >>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
> >>>>>>>>>>>> guration.DataRegionConfiguration">
> >>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
> >>>>>>>>>>>> 1024 *
> >>>>>>>>>>>> 102 * 1024}"/>
> >>>>>>>>>>>>                     </bean>
> >>>>>>>>>>>>                 </property>
> >>>>>>>>>>>>             </bean>
> >>>>>>>>>>>>         </property>
> >>>>>>>>>>>>
> >>>>>>>>>>>> In other data regions persistence will be disabled by default.
> >>>>>>>>>>>>
> >>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
> >>>>>>>>>>> /browse/IGNITE-6030 with these changes.
> >>>>>>>>>>>
> >>>>>>>>>>> Best Regards,
> >>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>
> >>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> To resolve this, I suggest to
> >>>>>>>>>>>
> >>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> get
> >>>>>>>>>>>>> rid
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML
> file? I’m
> >>>>>>>>>>>>>
> >>>>>>>>>>>> not
> >>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
> >>>>>>>>>>>> bean
> >>>>>>>>>>>> first
> >>>>>>>>>>>> to change any parameter?
> >>>>>>>>>>>>
> >>>>>>>>>>>> —
> >>>>>>>>>>>> Denis
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
> >>>>>>>>>>>>
> >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would
> either
> >>>>>>>>>>>>> need to
> >>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
> >>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
> >>>>>>>>>>>>> which
> >>>>>>>>>>>>> is also
> >>>>>>>>>>>>> an overkill. On the other hand, one can assume that the
> defaults
> >>>>>>>>>>>>> we
> >>>>>>>>>>>>> are
> >>>>>>>>>>>>> talking about are actually inherited. To resolve this, I
> suggest
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and
> get
> >>>>>>>>>>>>> rid
> >>>>>>>>>>>>> of
> >>>>>>>>>>>>> other defaults in DataStorageConfiguration.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <ivan.glukos@gmail.com
> >:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Vladimir,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I like your approach because it's easier to implement.
> >>>>>>>>>>>>>> However, user may be confused by setting
> >>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default
> in
> >>>>>>>>>>>>>> custom memory
> >>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Ivan,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I do not think this is correct approach, because it will be
> hard
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
> >>>>>>>>>>>>>>> "boolean"
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
> >>>>>>>>>>>>>>> "persistence
> >>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have
> "persistence
> >>>>>>>>>>>>>>> enabled"
> >>>>>>>>>>>>>>> flag for default region only. It should not be propagated
> to
> >>>>>>>>>>>>>>> custom
> >>>>>>>>>>>>>>> regions.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
> >>>>>>>>>>>>>>> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Guys, I think I got the point now.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Let's check the final design:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
> >>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
> >>>>>>>>>>>>>>>> property (default = false), which will be used for
> enabling
> >>>>>>>>>>>>>>>> persistence
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>> default data region.
> >>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
> >>>>>>>>>>>>>>>> property,
> >>>>>>>>>>>>>>>> which
> >>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding
> data
> >>>>>>>>>>>>>>>> region. If
> >>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
> >>>>>>>>>>>>>>>> efaultPersistenceEnabled*
> >>>>>>>>>>>>>>>> will be used by default.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Best Regards,
> >>>>>>>>>>>>>>>> Ivan Rakov
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
> >>>>>>>>>>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <
> ivan.glukos@gmail.com
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*.
> Persistence
> >>>>>>>>>>>>>>>>>>> will be
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
> >>>>>>>>>>>>>>>>>>> flag
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> set.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
> >>>>>>>>>>>>>>>>>> *globally*
> >>>>>>>>>>>>>>>>>> if the
> >>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
> >>>>>>>>>>>>>>>>>> enabled for
> >>>>>>>>>>>>>>>>>> region
> >>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
> >>>>>>>>>>>>>>>>>> disabled
> >>>>>>>>>>>>>>>>>> globally.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> But we should also give users a way to switch the
> default
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> behavior from
> >>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Ivan,

Instead of “setStoragePath” I would suggest “setPersistencePath”. Left some extra notes in the ticket.

—
Denis

> On Oct 11, 2017, at 4:30 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Vladimir,
> 
> Thanks for focusing on existing namings. Most of your suggestions really sound better. I've posted my thoughts under your comment.
> 
> By the way, we should decide two things:
> 
> 1) Naming for methods for configuring store path. I suggest the following:
> 
> *setStoragePath* - for partition and index files
> *setWalPath* - for WAL files
> *walArchivePath* - for WAL archive files
> 
> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with *checkpointingPageBufferSize* and *checkpointingThreads*). Both options sounds ok to me, let's see what community thinks.
> 
> Best Regards,
> Ivan Rakov
> 
> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>> Ivan,
>> 
>> I left some comments in the ticket [1], please take a look.
>> 
>> [1]
>> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>> 
>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:
>> 
>>> Igniters,
>>> 
>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>>> for TC run.
>>> PR: https://github.com/apache/ignite/pull/2828
>>> 
>>> Everyone interested in new data storage configuration API, please pay
>>> attention and review.
>>> 
>>> 
>>> Best Regards,
>>> Ivan Rakov
>>> 
>>> 
>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>> 
>>>> Sounds good to me.
>>>> 
>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>> 
>>>> Pavel,
>>>>> Sounds reasonable.
>>>>> I suggest to include both "data" and "configuration" to make it even more
>>>>> obvious:
>>>>> 
>>>>> set/getDefaultDataRegionConfiguration
>>>>> set/getDataRegionConfigurations
>>>>> 
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>> 
>>>>> 
>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>> 
>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>> 
>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>> - or -
>>>>>> defaultDataRegion + getDataRegions
>>>>>> 
>>>>>> Thoughts?
>>>>>> 
>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>> Denis,
>>>>>> 
>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>> configured will be persistent.
>>>>>>> And if you want to add another persistent data region, you should set
>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>>> 
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>> 
>>>>>>> 
>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>> 
>>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>>> 
>>>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>>>> regions
>>>>>>>> configured explicitly I need to take the default region and switch the
>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>> 
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>> 
>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>> 
>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>> bean?
>>>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>>>> automatically initialized with their default values.
>>>>>>>>> 
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>> 
>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>> 
>>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>> 
>>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>                     </bean>
>>>>>>>>>>>>                 </property>
>>>>>>>>>>>>             </bean>
>>>>>>>>>>>>         </property>
>>>>>>>>>>>> 
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>> 
>>>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>>>> a
>>>>>>>>>>> 
>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>> don’t want
>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>> 
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>> 
>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>>>> data
>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>> where
>>>>>>>>>>> size
>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>> 
>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>> 
>>>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>> description):
>>>>>>>>>>> 
>>>>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>>>> 
>>>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>                     </bean>
>>>>>>>>>>>>                 </property>
>>>>>>>>>>>>             </bean>
>>>>>>>>>>>>         </property>
>>>>>>>>>>>> 
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>> 
>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>> 
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>> 
>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>> 
>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>> 
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>>> 
>>>>>>>>>>>> not
>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>> bean
>>>>>>>>>>>> first
>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>> 
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>> 
>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>> 
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>> which
>>>>>>>>>>>>> is also
>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>>>> we
>>>>>>>>>>>>> are
>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>>>> to
>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Typo in my previous reply: *walArchivePath* -> *setWalArchivePath*

Best Regards,
Ivan Rakov

On 11.10.2017 14:30, Ivan Rakov wrote:
> Vladimir,
>
> Thanks for focusing on existing namings. Most of your suggestions 
> really sound better. I've posted my thoughts under your comment.
>
> By the way, we should decide two things:
>
> 1) Naming for methods for configuring store path. I suggest the 
> following:
>
> *setStoragePath* - for partition and index files
> *setWalPath* - for WAL files
> *walArchivePath* - for WAL archive files
>
> 2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same 
> with *checkpointingPageBufferSize* and *checkpointingThreads*). Both 
> options sounds ok to me, let's see what community thinks.
>
> Best Regards,
> Ivan Rakov
>
> On 11.10.2017 14:05, Vladimir Ozerov wrote:
>> Ivan,
>>
>> I left some comments in the ticket [1], please take a look.
>>
>> [1]
>> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050 
>>
>>
>> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> 
>> wrote:
>>
>>> Igniters,
>>>
>>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>>> for TC run.
>>> PR: https://github.com/apache/ignite/pull/2828
>>>
>>> Everyone interested in new data storage configuration API, please pay
>>> attention and review.
>>>
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>>
>>>> Sounds good to me.
>>>>
>>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Pavel,
>>>>> Sounds reasonable.
>>>>> I suggest to include both "data" and "configuration" to make it 
>>>>> even more
>>>>> obvious:
>>>>>
>>>>> set/getDefaultDataRegionConfiguration
>>>>> set/getDataRegionConfigurations
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>>
>>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>>> DataRegionConfiguration[] getDataRegions
>>>>>>
>>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>>> - or -
>>>>>> defaultDataRegion + getDataRegions
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Denis,
>>>>>>
>>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>>> configured will be persistent.
>>>>>>> And if you want to add another persistent data region, you 
>>>>>>> should set
>>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* 
>>>>>>> explictly.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>>
>>>>>>> Missed the point with defaults. Makes sense to me now. So to 
>>>>>>> wrap this
>>>>>>>
>>>>>>>> up, if I want to enable the persistence globally and don’t have 
>>>>>>>> any
>>>>>>>> regions
>>>>>>>> configured explicitly I need to take the default region and 
>>>>>>>> switch the
>>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>>> bean?
>>>>>>>>> If you want to set any parameter, just instantiate new bean 
>>>>>>>>> with this
>>>>>>>>> parameter set (like in XML snipped below). Other parameters 
>>>>>>>>> will be
>>>>>>>>> automatically initialized with their default values.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>>
>>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>                      </bean>
>>>>>>>>>>>>                  </property>
>>>>>>>>>>>>              </bean>
>>>>>>>>>>>>          </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> Ivan, how to get an instance to the default region bean and 
>>>>>>>>>>> change
>>>>>>>>>>> a
>>>>>>>>>>>
>>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>>> don’t want
>>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Agree with Alexey.
>>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> can confuse users who don't know that there's such thing as 
>>>>>>>>>>> default
>>>>>>>>>>> data
>>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>>> where
>>>>>>>>>>> size
>>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>>
>>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>>
>>>>>>>>>>> Regarding XML configuration, changing size or persistence 
>>>>>>>>>>> flag of
>>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>>> description):
>>>>>>>>>>>
>>>>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>>>>
>>>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>>> value="#{100
>>>>>>>>>>>> *
>>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>>> 1024 *
>>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>>                      </bean>
>>>>>>>>>>>>                  </property>
>>>>>>>>>>>>              </bean>
>>>>>>>>>>>>          </property>
>>>>>>>>>>>>
>>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>>
>>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>>
>>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>>
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and 
>>>>>>>>>>>> get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML 
>>>>>>>>>>>>> file? I’m
>>>>>>>>>>>>>
>>>>>>>>>>>> not
>>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>>> bean
>>>>>>>>>>>> first
>>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>>
>>>>>>>>>>>> —
>>>>>>>>>>>> Denis
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>>
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would 
>>>>>>>>>>>>> either
>>>>>>>>>>>>> need to
>>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>>> which
>>>>>>>>>>>>> is also
>>>>>>>>>>>>> an overkill. On the other hand, one can assume that the 
>>>>>>>>>>>>> defaults
>>>>>>>>>>>>> we
>>>>>>>>>>>>> are
>>>>>>>>>>>>> talking about are actually inherited. To resolve this, I 
>>>>>>>>>>>>> suggest
>>>>>>>>>>>>> to
>>>>>>>>>>>>> introduce just another field defaultRegionConfiguration 
>>>>>>>>>>>>> and get
>>>>>>>>>>>>> rid
>>>>>>>>>>>>> of
>>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov 
>>>>>>>>>>>>> <iv...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> flag and seeing that persistence is not enabled by 
>>>>>>>>>>>>>> default in
>>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I do not think this is correct approach, because it will 
>>>>>>>>>>>>>> be hard
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have 
>>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>>> flag for default region only. It should not be 
>>>>>>>>>>>>>>> propagated to
>>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>>> property (default = false), which will be used for 
>>>>>>>>>>>>>>>> enabling
>>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding 
>>>>>>>>>>>>>>>> data
>>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov 
>>>>>>>>>>>>>>>> <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. 
>>>>>>>>>>>>>>>>> Persistence
>>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> But we should also give users a way to switch the 
>>>>>>>>>>>>>>>>>> default
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Vladimir,

Thanks for focusing on existing namings. Most of your suggestions really 
sound better. I've posted my thoughts under your comment.

By the way, we should decide two things:

1) Naming for methods for configuring store path. I suggest the following:

*setStoragePath* - for partition and index files
*setWalPath* - for WAL files
*walArchivePath* - for WAL archive files

2) Renaming *checkpointingFrequency* to *checkpointFrequency* (same with 
*checkpointingPageBufferSize* and *checkpointingThreads*). Both options 
sounds ok to me, let's see what community thinks.

Best Regards,
Ivan Rakov

On 11.10.2017 14:05, Vladimir Ozerov wrote:
> Ivan,
>
> I left some comments in the ticket [1], please take a look.
>
> [1]
> https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050
>
> On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Igniters,
>>
>> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
>> for TC run.
>> PR: https://github.com/apache/ignite/pull/2828
>>
>> Everyone interested in new data storage configuration API, please pay
>> attention and review.
>>
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>>
>>> Sounds good to me.
>>>
>>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>>
>>> Pavel,
>>>> Sounds reasonable.
>>>> I suggest to include both "data" and "configuration" to make it even more
>>>> obvious:
>>>>
>>>> set/getDefaultDataRegionConfiguration
>>>> set/getDataRegionConfigurations
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>>
>>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>> DataStorageConfiguration defaultRegionConfiguration
>>>>> DataRegionConfiguration[] getDataRegions
>>>>>
>>>>> defaultRegionConfiguration + getRegionConfigurations
>>>>> - or -
>>>>> defaultDataRegion + getDataRegions
>>>>>
>>>>> Thoughts?
>>>>>
>>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Denis,
>>>>>
>>>>>> Yes, you're right. All cache groups without specific data region
>>>>>> configured will be persistent.
>>>>>> And if you want to add another persistent data region, you should set
>>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>>
>>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>>
>>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>>
>>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>>> regions
>>>>>>> configured explicitly I need to take the default region and switch the
>>>>>>> persistence on for it. Is my understanding correct?
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>>> bean?
>>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>>> automatically initialized with their default values.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>>
>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>
>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>> value="#{100
>>>>>>>>>>> *
>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>> 1024 *
>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>                      </bean>
>>>>>>>>>>>                  </property>
>>>>>>>>>>>              </bean>
>>>>>>>>>>>          </property>
>>>>>>>>>>>
>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>
>>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>>> a
>>>>>>>>>>
>>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>>> don’t want
>>>>>>>>> to create the default region bean from scratch.
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Agree with Alexey.
>>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>>> data
>>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>>> where
>>>>>>>>>> size
>>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>>
>>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>>
>>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>>> description):
>>>>>>>>>>
>>>>>>>>>>          <property name="dataStorageConfiguration">
>>>>>>>>>>
>>>>>>>>>>              <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>>                  <property name="systemCacheInitialSize"
>>>>>>>>>>> value="#{100
>>>>>>>>>>> *
>>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>>                  <property name="defaultRegionConfiguration">
>>>>>>>>>>>                      <bean class="org.apache.ignite.confi
>>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>>                          <property name="maxSize" value="#{5 *
>>>>>>>>>>> 1024 *
>>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>>                      </bean>
>>>>>>>>>>>                  </property>
>>>>>>>>>>>              </bean>
>>>>>>>>>>>          </property>
>>>>>>>>>>>
>>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>>
>>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>>
>>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>>
>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>> rid
>>>>>>>>>>>> of
>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>
>>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>>
>>>>>>>>>>> not
>>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>>> bean
>>>>>>>>>>> first
>>>>>>>>>>> to change any parameter?
>>>>>>>>>>>
>>>>>>>>>>> —
>>>>>>>>>>> Denis
>>>>>>>>>>>
>>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>>
>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>>> need to
>>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>>> which
>>>>>>>>>>>> is also
>>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>>> we
>>>>>>>>>>>> are
>>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>>> to
>>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>>> rid
>>>>>>>>>>>> of
>>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>> Vladimir,
>>>>>>>>>>>>
>>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>>> custom memory
>>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>>> custom
>>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>>> which
>>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Ivan,

I left some comments in the ticket [1], please take a look.

[1]
https://issues.apache.org/jira/browse/IGNITE-6030?focusedCommentId=16200050&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16200050

On Wed, Oct 11, 2017 at 12:04 PM, Ivan Rakov <iv...@gmail.com> wrote:

> Igniters,
>
> https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued
> for TC run.
> PR: https://github.com/apache/ignite/pull/2828
>
> Everyone interested in new data storage configuration API, please pay
> attention and review.
>
>
> Best Regards,
> Ivan Rakov
>
>
> On 09.10.2017 12:40, Pavel Tupitsyn wrote:
>
>> Sounds good to me.
>>
>> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>
>> Pavel,
>>>
>>> Sounds reasonable.
>>> I suggest to include both "data" and "configuration" to make it even more
>>> obvious:
>>>
>>> set/getDefaultDataRegionConfiguration
>>> set/getDataRegionConfigurations
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>>
>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>>
>>>> DataStorageConfiguration defaultRegionConfiguration
>>>> DataRegionConfiguration[] getDataRegions
>>>>
>>>> defaultRegionConfiguration + getRegionConfigurations
>>>> - or -
>>>> defaultDataRegion + getDataRegions
>>>>
>>>> Thoughts?
>>>>
>>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Denis,
>>>>
>>>>> Yes, you're right. All cache groups without specific data region
>>>>> configured will be persistent.
>>>>> And if you want to add another persistent data region, you should set
>>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>>
>>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>>
>>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>>> regions
>>>>>> configured explicitly I need to take the default region and switch the
>>>>>> persistence on for it. Is my understanding correct?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Denis, why do you need to access an instance of the default region
>>>>>>> bean?
>>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>>> automatically initialized with their default values.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>>
>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>
>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>
>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>> value="#{100
>>>>>>>>>> *
>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>> 1024 *
>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>                     </bean>
>>>>>>>>>>                 </property>
>>>>>>>>>>             </bean>
>>>>>>>>>>         </property>
>>>>>>>>>>
>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>
>>>>>>>>> Ivan, how to get an instance to the default region bean and change
>>>>>>>>> a
>>>>>>>>>
>>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>>> don’t want
>>>>>>>> to create the default region bean from scratch.
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Agree with Alexey.
>>>>>>>>>
>>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>>> data
>>>>>>>>> region. They can decide they are inherited by all data regions
>>>>>>>>> where
>>>>>>>>> size
>>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>>
>>>>>>>>> Let's get rid of these properties and add
>>>>>>>>> *defaultRegionConfiguration*
>>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>>
>>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>>> description):
>>>>>>>>>
>>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>>
>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>>                 <property name="systemCacheInitialSize"
>>>>>>>>>> value="#{100
>>>>>>>>>> *
>>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>>                         <property name="maxSize" value="#{5 *
>>>>>>>>>> 1024 *
>>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>>                     </bean>
>>>>>>>>>>                 </property>
>>>>>>>>>>             </bean>
>>>>>>>>>>         </property>
>>>>>>>>>>
>>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>>>>
>>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>>
>>>>>>>>> To resolve this, I suggest to
>>>>>>>>>
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>> rid
>>>>>>>>>>> of
>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>
>>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>>>>
>>>>>>>>>> not
>>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration
>>>>>>>>>> bean
>>>>>>>>>> first
>>>>>>>>>> to change any parameter?
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>>
>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>>> need to
>>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field
>>>>>>>>>>> which
>>>>>>>>>>> is also
>>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults
>>>>>>>>>>> we
>>>>>>>>>>> are
>>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest
>>>>>>>>>>> to
>>>>>>>>>>> introduce just another field defaultRegionConfiguration and get
>>>>>>>>>>> rid
>>>>>>>>>>> of
>>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>>
>>>>>>>>>>> Thoughts?
>>>>>>>>>>>
>>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>> Vladimir,
>>>>>>>>>>>
>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>>
>>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>>> custom memory
>>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>
>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>>> to
>>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of
>>>>>>>>>>>>> "boolean"
>>>>>>>>>>>>> for
>>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>>> "persistence
>>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>>> enabled"
>>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>>> custom
>>>>>>>>>>>>> regions.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>>> persistence
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>>> property,
>>>>>>>>>>>>>> which
>>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>>> region. If
>>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <
>>>>>>>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this
>>>>>>>>>>>>>>>>> flag
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled
>>>>>>>>>>>>>>>> *globally*
>>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Igniters,

https://issues.apache.org/jira/browse/IGNITE-6030 is ready and enqueued 
for TC run.
PR: https://github.com/apache/ignite/pull/2828

Everyone interested in new data storage configuration API, please pay 
attention and review.


Best Regards,
Ivan Rakov

On 09.10.2017 12:40, Pavel Tupitsyn wrote:
> Sounds good to me.
>
> On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Pavel,
>>
>> Sounds reasonable.
>> I suggest to include both "data" and "configuration" to make it even more
>> obvious:
>>
>> set/getDefaultDataRegionConfiguration
>> set/getDataRegionConfigurations
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>>
>>> Sorry that I'm late to the party, but this looks inconsistent:
>>>
>>> DataStorageConfiguration defaultRegionConfiguration
>>> DataRegionConfiguration[] getDataRegions
>>>
>>> defaultRegionConfiguration + getRegionConfigurations
>>> - or -
>>> defaultDataRegion + getDataRegions
>>>
>>> Thoughts?
>>>
>>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis,
>>>> Yes, you're right. All cache groups without specific data region
>>>> configured will be persistent.
>>>> And if you want to add another persistent data region, you should set
>>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>>
>>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>>
>>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>>> up, if I want to enable the persistence globally and don’t have any
>>>>> regions
>>>>> configured explicitly I need to take the default region and switch the
>>>>> persistence on for it. Is my understanding correct?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>
>>>>>> Denis, why do you need to access an instance of the default region
>>>>>> bean?
>>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>>> automatically initialized with their default values.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>>
>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>                 <property name="systemCacheInitialSize" value="#{100
>>>>>>>>> *
>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>                         <property name="maxSize" value="#{5 * 1024 *
>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>                     </bean>
>>>>>>>>>                 </property>
>>>>>>>>>             </bean>
>>>>>>>>>         </property>
>>>>>>>>>
>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>>> don’t want
>>>>>>> to create the default region bean from scratch.
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>
>>>>>>>> Agree with Alexey.
>>>>>>>>
>>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>>> data
>>>>>>>> region. They can decide they are inherited by all data regions where
>>>>>>>> size
>>>>>>>> and persistence flag are not explicitly set.
>>>>>>>>
>>>>>>>> Let's get rid of these properties and add
>>>>>>>> *defaultRegionConfiguration*
>>>>>>>> property with explicit configuration of default data region.
>>>>>>>>
>>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>>> default data region will be just two lines longer (for bean
>>>>>>>> description):
>>>>>>>>
>>>>>>>>         <property name="dataStorageConfiguration">
>>>>>>>>
>>>>>>>>>             <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>>                 <property name="systemCacheInitialSize" value="#{100
>>>>>>>>> *
>>>>>>>>> 1024 * 1024}"/>
>>>>>>>>>                 <property name="defaultRegionConfiguration">
>>>>>>>>>                     <bean class="org.apache.ignite.confi
>>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>>                         <property name="maxSize" value="#{5 * 1024 *
>>>>>>>>> 102 * 1024}"/>
>>>>>>>>>                     </bean>
>>>>>>>>>                 </property>
>>>>>>>>>             </bean>
>>>>>>>>>         </property>
>>>>>>>>>
>>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>>
>>>>>>>> To resolve this, I suggest to
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>>> of
>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>
>>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>>> not
>>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean
>>>>>>>>> first
>>>>>>>>> to change any parameter?
>>>>>>>>>
>>>>>>>>> —
>>>>>>>>> Denis
>>>>>>>>>
>>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>>
>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>>> need to
>>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>>>> is also
>>>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>>>> are
>>>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>>> of
>>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>>
>>>>>>>>>> Thoughts?
>>>>>>>>>>
>>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>> Vladimir,
>>>>>>>>>>
>>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>>
>>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>>> custom memory
>>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>>
>>>>>>>>>>> Ivan,
>>>>>>>>>>>
>>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>>> to
>>>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>>>> for
>>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>>> "persistence
>>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>>> enabled"
>>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>>> custom
>>>>>>>>>>>> regions.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>>
>>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>>> persistence
>>>>>>>>>>>>> in
>>>>>>>>>>>>> default data region.
>>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>>> property,
>>>>>>>>>>>>> which
>>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>>> region. If
>>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>>> region
>>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>

Re: Persistence per memory policy configuration

[Pavel Tupitsyn <pt...@apache.org>]

Sounds good to me.

On Mon, Oct 9, 2017 at 12:35 PM, Ivan Rakov <iv...@gmail.com> wrote:

> Pavel,
>
> Sounds reasonable.
> I suggest to include both "data" and "configuration" to make it even more
> obvious:
>
> set/getDefaultDataRegionConfiguration
> set/getDataRegionConfigurations
>
> Best Regards,
> Ivan Rakov
>
>
> On 09.10.2017 10:51, Pavel Tupitsyn wrote:
>
>> Sorry that I'm late to the party, but this looks inconsistent:
>>
>> DataStorageConfiguration defaultRegionConfiguration
>> DataRegionConfiguration[] getDataRegions
>>
>> defaultRegionConfiguration + getRegionConfigurations
>> - or -
>> defaultDataRegion + getDataRegions
>>
>> Thoughts?
>>
>> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Denis,
>>>
>>> Yes, you're right. All cache groups without specific data region
>>> configured will be persistent.
>>> And if you want to add another persistent data region, you should set
>>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 02.10.2017 21:01, Denis Magda wrote:
>>>
>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>>> up, if I want to enable the persistence globally and don’t have any
>>>> regions
>>>> configured explicitly I need to take the default region and switch the
>>>> persistence on for it. Is my understanding correct?
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>>> Denis, why do you need to access an instance of the default region
>>>>> bean?
>>>>> If you want to set any parameter, just instantiate new bean with this
>>>>> parameter set (like in XML snipped below). Other parameters will be
>>>>> automatically initialized with their default values.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>
>>>>>        <property name="dataStorageConfiguration">
>>>>>>
>>>>>>>            <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>                <property name="systemCacheInitialSize" value="#{100
>>>>>>>> *
>>>>>>>> 1024 * 1024}"/>
>>>>>>>>                <property name="defaultRegionConfiguration">
>>>>>>>>                    <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>                        <property name="maxSize" value="#{5 * 1024 *
>>>>>>>> 102 * 1024}"/>
>>>>>>>>                    </bean>
>>>>>>>>                </property>
>>>>>>>>            </bean>
>>>>>>>>        </property>
>>>>>>>>
>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>>
>>>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>>> parameter? Obviously, if the goal is to enable the persistence I
>>>>>> don’t want
>>>>>> to create the default region bean from scratch.
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>
>>>>>>> Agree with Alexey.
>>>>>>>
>>>>>>> Properties like *defaultDataRegionSize*,
>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>> can confuse users who don't know that there's such thing as default
>>>>>>> data
>>>>>>> region. They can decide they are inherited by all data regions where
>>>>>>> size
>>>>>>> and persistence flag are not explicitly set.
>>>>>>>
>>>>>>> Let's get rid of these properties and add
>>>>>>> *defaultRegionConfiguration*
>>>>>>> property with explicit configuration of default data region.
>>>>>>>
>>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>>> default data region will be just two lines longer (for bean
>>>>>>> description):
>>>>>>>
>>>>>>>        <property name="dataStorageConfiguration">
>>>>>>>
>>>>>>>>            <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataStorageConfiguration">
>>>>>>>>                <property name="systemCacheInitialSize" value="#{100
>>>>>>>> *
>>>>>>>> 1024 * 1024}"/>
>>>>>>>>                <property name="defaultRegionConfiguration">
>>>>>>>>                    <bean class="org.apache.ignite.confi
>>>>>>>> guration.DataRegionConfiguration">
>>>>>>>>                        <property name="maxSize" value="#{5 * 1024 *
>>>>>>>> 102 * 1024}"/>
>>>>>>>>                    </bean>
>>>>>>>>                </property>
>>>>>>>>            </bean>
>>>>>>>>        </property>
>>>>>>>>
>>>>>>>> In other data regions persistence will be disabled by default.
>>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>>
>>>>>>> To resolve this, I suggest to
>>>>>>>>
>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>> of
>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>
>>>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m
>>>>>>>> not
>>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean
>>>>>>>> first
>>>>>>>> to change any parameter?
>>>>>>>>
>>>>>>>> —
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>>
>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>>> need to
>>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>>> is also
>>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>>> are
>>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>>> of
>>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>>
>>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>>
>>>>>>>>> Vladimir,
>>>>>>>>>
>>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>>
>>>>>>>>>> However, user may be confused by setting
>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>>> custom memory
>>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>>
>>>>>>>>>> Ivan,
>>>>>>>>>>
>>>>>>>>>>> I do not think this is correct approach, because it will be hard
>>>>>>>>>>> to
>>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>>> for
>>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>>> "persistence
>>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>>> enabled"
>>>>>>>>>>> flag for default region only. It should not be propagated to
>>>>>>>>>>> custom
>>>>>>>>>>> regions.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <
>>>>>>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>>
>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>>
>>>>>>>>>>>> *DataStorageConfiguration* will have
>>>>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>>> persistence
>>>>>>>>>>>> in
>>>>>>>>>>>> default data region.
>>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>>> property,
>>>>>>>>>>>> which
>>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>>> region. If
>>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>>> will be used by default.
>>>>>>>>>>>>
>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>
>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>>> set.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>>> if the
>>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>>> region
>>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yes, you are right. By default the persistence will be
>>>>>>>>>>>>>> disabled
>>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>>>>
>>>>>>>>>>>>> behavior from
>>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Pavel,

Sounds reasonable.
I suggest to include both "data" and "configuration" to make it even 
more obvious:

set/getDefaultDataRegionConfiguration
set/getDataRegionConfigurations

Best Regards,
Ivan Rakov

On 09.10.2017 10:51, Pavel Tupitsyn wrote:
> Sorry that I'm late to the party, but this looks inconsistent:
>
> DataStorageConfiguration defaultRegionConfiguration
> DataRegionConfiguration[] getDataRegions
>
> defaultRegionConfiguration + getRegionConfigurations
> - or -
> defaultDataRegion + getDataRegions
>
> Thoughts?
>
> On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Denis,
>>
>> Yes, you're right. All cache groups without specific data region
>> configured will be persistent.
>> And if you want to add another persistent data region, you should set
>> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>> On 02.10.2017 21:01, Denis Magda wrote:
>>
>>> Missed the point with defaults. Makes sense to me now. So to wrap this
>>> up, if I want to enable the persistence globally and don’t have any regions
>>> configured explicitly I need to take the default region and switch the
>>> persistence on for it. Is my understanding correct?
>>>
>>> —
>>> Denis
>>>
>>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>> Denis, why do you need to access an instance of the default region bean?
>>>> If you want to set any parameter, just instantiate new bean with this
>>>> parameter set (like in XML snipped below). Other parameters will be
>>>> automatically initialized with their default values.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>
>>>>>        <property name="dataStorageConfiguration">
>>>>>>>            <bean class="org.apache.ignite.confi
>>>>>>> guration.DataStorageConfiguration">
>>>>>>>                <property name="systemCacheInitialSize" value="#{100 *
>>>>>>> 1024 * 1024}"/>
>>>>>>>                <property name="defaultRegionConfiguration">
>>>>>>>                    <bean class="org.apache.ignite.confi
>>>>>>> guration.DataRegionConfiguration">
>>>>>>>                        <property name="maxSize" value="#{5 * 1024 *
>>>>>>> 102 * 1024}"/>
>>>>>>>                    </bean>
>>>>>>>                </property>
>>>>>>>            </bean>
>>>>>>>        </property>
>>>>>>>
>>>>>> In other data regions persistence will be disabled by default.
>>>>>>
>>>>> Ivan, how to get an instance to the default region bean and change a
>>>>> parameter? Obviously, if the goal is to enable the persistence I don’t want
>>>>> to create the default region bean from scratch.
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>> Agree with Alexey.
>>>>>>
>>>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled*
>>>>>> can confuse users who don't know that there's such thing as default data
>>>>>> region. They can decide they are inherited by all data regions where size
>>>>>> and persistence flag are not explicitly set.
>>>>>>
>>>>>> Let's get rid of these properties and add *defaultRegionConfiguration*
>>>>>> property with explicit configuration of default data region.
>>>>>>
>>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>>> default data region will be just two lines longer (for bean description):
>>>>>>
>>>>>>        <property name="dataStorageConfiguration">
>>>>>>>            <bean class="org.apache.ignite.confi
>>>>>>> guration.DataStorageConfiguration">
>>>>>>>                <property name="systemCacheInitialSize" value="#{100 *
>>>>>>> 1024 * 1024}"/>
>>>>>>>                <property name="defaultRegionConfiguration">
>>>>>>>                    <bean class="org.apache.ignite.confi
>>>>>>> guration.DataRegionConfiguration">
>>>>>>>                        <property name="maxSize" value="#{5 * 1024 *
>>>>>>> 102 * 1024}"/>
>>>>>>>                    </bean>
>>>>>>>                </property>
>>>>>>>            </bean>
>>>>>>>        </property>
>>>>>>>
>>>>>> In other data regions persistence will be disabled by default.
>>>>>> I've updated draft in https://issues.apache.org/jira
>>>>>> /browse/IGNITE-6030 with these changes.
>>>>>>
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>>
>>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>>
>>>>>>> To resolve this, I suggest to
>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>> of
>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>
>>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not
>>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean first
>>>>>>> to change any parameter?
>>>>>>>
>>>>>>> —
>>>>>>> Denis
>>>>>>>
>>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>>> need to
>>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>>> is also
>>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>>> are
>>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>>> of
>>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>>
>>>>>>>> Vladimir,
>>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>>
>>>>>>>>> However, user may be confused by setting
>>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>>> custom memory
>>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>>
>>>>>>>>> Ivan,
>>>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>>> for
>>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>>> "persistence
>>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>>> enabled"
>>>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>>>> regions.
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>>
>>>>>>>>>>> Let's check the final design:
>>>>>>>>>>>
>>>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>>> persistence
>>>>>>>>>>> in
>>>>>>>>>>> default data region.
>>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>>> property,
>>>>>>>>>>> which
>>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>>> region. If
>>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>>> will be used by default.
>>>>>>>>>>>
>>>>>>>>>>> Best Regards,
>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>>> will be
>>>>>>>>>>>>>>
>>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>>> set.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>>> if the
>>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>>> enabled for
>>>>>>>>>>>>> region
>>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>>>> globally.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>>> behavior from
>>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>

Re: Persistence per memory policy configuration

[Pavel Tupitsyn <pt...@apache.org>]

Sorry that I'm late to the party, but this looks inconsistent:

DataStorageConfiguration defaultRegionConfiguration
DataRegionConfiguration[] getDataRegions

defaultRegionConfiguration + getRegionConfigurations
- or -
defaultDataRegion + getDataRegions

Thoughts?

On Mon, Oct 2, 2017 at 9:10 PM, Ivan Rakov <iv...@gmail.com> wrote:

> Denis,
>
> Yes, you're right. All cache groups without specific data region
> configured will be persistent.
> And if you want to add another persistent data region, you should set
> *isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.
>
> Best Regards,
> Ivan Rakov
>
>
> On 02.10.2017 21:01, Denis Magda wrote:
>
>> Missed the point with defaults. Makes sense to me now. So to wrap this
>> up, if I want to enable the persistence globally and don’t have any regions
>> configured explicitly I need to take the default region and switch the
>> persistence on for it. Is my understanding correct?
>>
>> —
>> Denis
>>
>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Denis, why do you need to access an instance of the default region bean?
>>> If you want to set any parameter, just instantiate new bean with this
>>> parameter set (like in XML snipped below). Other parameters will be
>>> automatically initialized with their default values.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On 02.10.2017 19:28, Denis Magda wrote:
>>>
>>>>       <property name="dataStorageConfiguration">
>>>>>>           <bean class="org.apache.ignite.confi
>>>>>> guration.DataStorageConfiguration">
>>>>>>               <property name="systemCacheInitialSize" value="#{100 *
>>>>>> 1024 * 1024}"/>
>>>>>>               <property name="defaultRegionConfiguration">
>>>>>>                   <bean class="org.apache.ignite.confi
>>>>>> guration.DataRegionConfiguration">
>>>>>>                       <property name="maxSize" value="#{5 * 1024 *
>>>>>> 102 * 1024}"/>
>>>>>>                   </bean>
>>>>>>               </property>
>>>>>>           </bean>
>>>>>>       </property>
>>>>>>
>>>>> In other data regions persistence will be disabled by default.
>>>>>
>>>> Ivan, how to get an instance to the default region bean and change a
>>>> parameter? Obviously, if the goal is to enable the persistence I don’t want
>>>> to create the default region bean from scratch.
>>>>
>>>> —
>>>> Denis
>>>>
>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>
>>>>> Agree with Alexey.
>>>>>
>>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled*
>>>>> can confuse users who don't know that there's such thing as default data
>>>>> region. They can decide they are inherited by all data regions where size
>>>>> and persistence flag are not explicitly set.
>>>>>
>>>>> Let's get rid of these properties and add *defaultRegionConfiguration*
>>>>> property with explicit configuration of default data region.
>>>>>
>>>>> Regarding XML configuration, changing size or persistence flag of
>>>>> default data region will be just two lines longer (for bean description):
>>>>>
>>>>>       <property name="dataStorageConfiguration">
>>>>>>           <bean class="org.apache.ignite.confi
>>>>>> guration.DataStorageConfiguration">
>>>>>>               <property name="systemCacheInitialSize" value="#{100 *
>>>>>> 1024 * 1024}"/>
>>>>>>               <property name="defaultRegionConfiguration">
>>>>>>                   <bean class="org.apache.ignite.confi
>>>>>> guration.DataRegionConfiguration">
>>>>>>                       <property name="maxSize" value="#{5 * 1024 *
>>>>>> 102 * 1024}"/>
>>>>>>                   </bean>
>>>>>>               </property>
>>>>>>           </bean>
>>>>>>       </property>
>>>>>>
>>>>> In other data regions persistence will be disabled by default.
>>>>> I've updated draft in https://issues.apache.org/jira
>>>>> /browse/IGNITE-6030 with these changes.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>
>>>>>> To resolve this, I suggest to
>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>> of
>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>
>>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not
>>>>>> an expert in Spring so how do I get defaultRegionConfiguration bean first
>>>>>> to change any parameter?
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <
>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>
>>>>>>> Agree with Vladimir. If we are to implement this, we would either
>>>>>>> need to
>>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>>> DataRegionConfiguration, or introduce an enum for this field which
>>>>>>> is also
>>>>>>> an overkill. On the other hand, one can assume that the defaults we
>>>>>>> are
>>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>>> introduce just another field defaultRegionConfiguration and get rid
>>>>>>> of
>>>>>>> other defaults in DataStorageConfiguration.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>>
>>>>>>> Vladimir,
>>>>>>>>
>>>>>>>> I like your approach because it's easier to implement.
>>>>>>>>
>>>>>>>> However, user may be confused by setting
>>>>>>>> *isDefaultPersistenceEnabled*
>>>>>>>> flag and seeing that persistence is not enabled by default in
>>>>>>>> custom memory
>>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>>
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>>
>>>>>>>>
>>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>>
>>>>>>>> Ivan,
>>>>>>>>>
>>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean"
>>>>>>>>> for
>>>>>>>>> DataRegionConfiguration. I do not think we need default
>>>>>>>>> "persistence
>>>>>>>>> enabled" for all regions. Instead, we should have "persistence
>>>>>>>>> enabled"
>>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>>> regions.
>>>>>>>>>
>>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <ivan.glukos@gmail.com
>>>>>>>>> >
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Guys, I think I got the point now.
>>>>>>>>>
>>>>>>>>>> Let's check the final design:
>>>>>>>>>>
>>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>>> property (default = false), which will be used for enabling
>>>>>>>>>> persistence
>>>>>>>>>> in
>>>>>>>>>> default data region.
>>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled*
>>>>>>>>>> property,
>>>>>>>>>> which
>>>>>>>>>> will be used for enabling persistence in corresponding data
>>>>>>>>>> region. If
>>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>>> will be used by default.
>>>>>>>>>>
>>>>>>>>>> Best Regards,
>>>>>>>>>> Ivan Rakov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>>
>>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence
>>>>>>>>>>>>> will be
>>>>>>>>>>>>>
>>>>>>>>>>>> enabled globally if at least one memory region has this flag
>>>>>>>>>>>> set.
>>>>>>>>>>>>
>>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally*
>>>>>>>>>>>> if the
>>>>>>>>>>>> purpose is to have it set for a specific region? If it’s
>>>>>>>>>>>> enabled for
>>>>>>>>>>>> region
>>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>>
>>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>>> globally.
>>>>>>>>>>>>
>>>>>>>>>>>> But we should also give users a way to switch the default
>>>>>>>>>>> behavior from
>>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Denis,

Yes, you're right. All cache groups without specific data region 
configured will be persistent.
And if you want to add another persistent data region, you should set 
*isPeristenceEnabled* flag in its *DataRegionConfiguration* explictly.

Best Regards,
Ivan Rakov

On 02.10.2017 21:01, Denis Magda wrote:
> Missed the point with defaults. Makes sense to me now. So to wrap this up, if I want to enable the persistence globally and don’t have any regions configured explicitly I need to take the default region and switch the persistence on for it. Is my understanding correct?
>
> —
> Denis
>
>> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Denis, why do you need to access an instance of the default region bean? If you want to set any parameter, just instantiate new bean with this parameter set (like in XML snipped below). Other parameters will be automatically initialized with their default values.
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 02.10.2017 19:28, Denis Magda wrote:
>>>>>       <property name="dataStorageConfiguration">
>>>>>           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>>               <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>>               <property name="defaultRegionConfiguration">
>>>>>                   <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>>                       <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>>                   </bean>
>>>>>               </property>
>>>>>           </bean>
>>>>>       </property>
>>>> In other data regions persistence will be disabled by default.
>>> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>>>
>>> —
>>> Denis
>>>
>>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> Agree with Alexey.
>>>>
>>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>>>
>>>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>>>
>>>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>>>
>>>>>       <property name="dataStorageConfiguration">
>>>>>           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>>               <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>>               <property name="defaultRegionConfiguration">
>>>>>                   <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>>                       <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>>                   </bean>
>>>>>               </property>
>>>>>           </bean>
>>>>>       </property>
>>>> In other data regions persistence will be disabled by default.
>>>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>>>
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>>> To resolve this, I suggest to
>>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>>> other defaults in DataStorageConfiguration.
>>>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>>>
>>>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>>> other defaults in DataStorageConfiguration.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>>>
>>>>>>> Vladimir,
>>>>>>>
>>>>>>> I like your approach because it's easier to implement.
>>>>>>>
>>>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>>>
>>>>>>>> Ivan,
>>>>>>>>
>>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>>> regions.
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Guys, I think I got the point now.
>>>>>>>>> Let's check the final design:
>>>>>>>>>
>>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>>>> in
>>>>>>>>> default data region.
>>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>>>> which
>>>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>>> efaultPersistenceEnabled*
>>>>>>>>> will be used by default.
>>>>>>>>>
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>>>
>>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>>>> region
>>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>>>
>>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>>> globally.
>>>>>>>>>>>
>>>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Missed the point with defaults. Makes sense to me now. So to wrap this up, if I want to enable the persistence globally and don’t have any regions configured explicitly I need to take the default region and switch the persistence on for it. Is my understanding correct?

—
Denis

> On Oct 2, 2017, at 10:57 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Denis, why do you need to access an instance of the default region bean? If you want to set any parameter, just instantiate new bean with this parameter set (like in XML snipped below). Other parameters will be automatically initialized with their default values.
> 
> Best Regards,
> Ivan Rakov
> 
> On 02.10.2017 19:28, Denis Magda wrote:
>>>>      <property name="dataStorageConfiguration">
>>>>          <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>              <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>              <property name="defaultRegionConfiguration">
>>>>                  <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>                      <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>                  </bean>
>>>>              </property>
>>>>          </bean>
>>>>      </property>
>>> In other data regions persistence will be disabled by default.
>> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>> 
>> —
>> Denis
>> 
>>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>> 
>>> Agree with Alexey.
>>> 
>>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>> 
>>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>> 
>>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>> 
>>>>      <property name="dataStorageConfiguration">
>>>>          <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>>              <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>>              <property name="defaultRegionConfiguration">
>>>>                  <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>>                      <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>>                  </bean>
>>>>              </property>
>>>>          </bean>
>>>>      </property>
>>> In other data regions persistence will be disabled by default.
>>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>> 
>>> Best Regards,
>>> Ivan Rakov
>>> 
>>> On 02.10.2017 18:35, Denis Magda wrote:
>>>>> To resolve this, I suggest to
>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>> other defaults in DataStorageConfiguration.
>>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>> 
>>>> —
>>>> Denis
>>>> 
>>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>> 
>>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>>> talking about are actually inherited. To resolve this, I suggest to
>>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>>> other defaults in DataStorageConfiguration.
>>>>> 
>>>>> Thoughts?
>>>>> 
>>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>> 
>>>>>> Vladimir,
>>>>>> 
>>>>>> I like your approach because it's easier to implement.
>>>>>> 
>>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>> 
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>> 
>>>>>> 
>>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>> 
>>>>>>> Ivan,
>>>>>>> 
>>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>>> flag for default region only. It should not be propagated to custom
>>>>>>> regions.
>>>>>>> 
>>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>>> wrote:
>>>>>>> 
>>>>>>> Guys, I think I got the point now.
>>>>>>>> Let's check the final design:
>>>>>>>> 
>>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>>> in
>>>>>>>> default data region.
>>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>>> which
>>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>>> efaultPersistenceEnabled*
>>>>>>>> will be used by default.
>>>>>>>> 
>>>>>>>> Best Regards,
>>>>>>>> Ivan Rakov
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>> 
>>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>> 
>>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>> 
>>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>>> region
>>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>> 
>>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>>> globally.
>>>>>>>>>> 
>>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Denis, why do you need to access an instance of the default region bean? 
If you want to set any parameter, just instantiate new bean with this 
parameter set (like in XML snipped below). Other parameters will be 
automatically initialized with their default values.

Best Regards,
Ivan Rakov

On 02.10.2017 19:28, Denis Magda wrote:
>>>       <property name="dataStorageConfiguration">
>>>           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>               <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>               <property name="defaultRegionConfiguration">
>>>                   <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>                       <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>                   </bean>
>>>               </property>
>>>           </bean>
>>>       </property>
>> In other data regions persistence will be disabled by default.
> Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.
>
> —
> Denis
>
>> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Agree with Alexey.
>>
>> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
>>
>> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
>>
>> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
>>
>>>       <property name="dataStorageConfiguration">
>>>           <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>>               <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>>               <property name="defaultRegionConfiguration">
>>>                   <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>>                       <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>>                   </bean>
>>>               </property>
>>>           </bean>
>>>       </property>
>> In other data regions persistence will be disabled by default.
>> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
>>
>> Best Regards,
>> Ivan Rakov
>>
>> On 02.10.2017 18:35, Denis Magda wrote:
>>>> To resolve this, I suggest to
>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>> other defaults in DataStorageConfiguration.
>>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>>>
>>> —
>>> Denis
>>>
>>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>>>
>>>> Agree with Vladimir. If we are to implement this, we would either need to
>>>> have a Boolean (non-primitive) for persistenceEnabled on
>>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>>> an overkill. On the other hand, one can assume that the defaults we are
>>>> talking about are actually inherited. To resolve this, I suggest to
>>>> introduce just another field defaultRegionConfiguration and get rid of
>>>> other defaults in DataStorageConfiguration.
>>>>
>>>> Thoughts?
>>>>
>>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>>>
>>>>> Vladimir,
>>>>>
>>>>> I like your approach because it's easier to implement.
>>>>>
>>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>>> region. I'll add clarifying Javadoc at this place.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>>>
>>>>>> Ivan,
>>>>>>
>>>>>> I do not think this is correct approach, because it will be hard to
>>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>>> flag for default region only. It should not be propagated to custom
>>>>>> regions.
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> Guys, I think I got the point now.
>>>>>>> Let's check the final design:
>>>>>>>
>>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>>> property (default = false), which will be used for enabling persistence
>>>>>>> in
>>>>>>> default data region.
>>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>>> which
>>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>>> efaultPersistenceEnabled*
>>>>>>> will be used by default.
>>>>>>>
>>>>>>> Best Regards,
>>>>>>> Ivan Rakov
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>>>
>>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>>>
>>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>>> region
>>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>>>
>>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>>> globally.
>>>>>>>>>
>>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>>>
>>>>>>>>
>>>>>>>>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

On Mon, Oct 2, 2017 at 7:28 PM, Denis Magda <dm...@apache.org> wrote:

>
> >>      <property name="dataStorageConfiguration">
> >>          <bean class="org.apache.ignite.configuration.
> DataStorageConfiguration">
> >>              <property name="systemCacheInitialSize" value="#{100 *
> 1024 * 1024}"/>
> >>              <property name="defaultRegionConfiguration">
> >>                  <bean class="org.apache.ignite.configuration.
> DataRegionConfiguration">
> >>                      <property name="maxSize" value="#{5 * 1024 * 102 *
> 1024}"/>
> >>                  </bean>
> >>              </property>
> >>          </bean>
> >>      </property>
> >
> > In other data regions persistence will be disabled by default.
>
> Ivan, how to get an instance to the default region bean and change a
> parameter? Obviously, if the goal is to enable the persistence I don’t want
> to create the default region bean from scratch.
>

Denis, I think this is the only way. If we don't create the
defaultRegionConfiguration, then we have to make sure that all the
properties are inherited from DataStorageConfiguration. From the feedback I
have seen so far, folks think that the inheritance can get really confusing
here.

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

>>      <property name="dataStorageConfiguration">
>>          <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>              <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>              <property name="defaultRegionConfiguration">
>>                  <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>                      <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>                  </bean>
>>              </property>
>>          </bean>
>>      </property>
> 
> In other data regions persistence will be disabled by default.

Ivan, how to get an instance to the default region bean and change a parameter? Obviously, if the goal is to enable the persistence I don’t want to create the default region bean from scratch.

—
Denis

> On Oct 2, 2017, at 9:11 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Agree with Alexey.
> 
> Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* can confuse users who don't know that there's such thing as default data region. They can decide they are inherited by all data regions where size and persistence flag are not explicitly set.
> 
> Let's get rid of these properties and add *defaultRegionConfiguration* property with explicit configuration of default data region.
> 
> Regarding XML configuration, changing size or persistence flag of default data region will be just two lines longer (for bean description):
> 
>>      <property name="dataStorageConfiguration">
>>          <bean class="org.apache.ignite.configuration.DataStorageConfiguration">
>>              <property name="systemCacheInitialSize" value="#{100 * 1024 * 1024}"/>
>>              <property name="defaultRegionConfiguration">
>>                  <bean class="org.apache.ignite.configuration.DataRegionConfiguration">
>>                      <property name="maxSize" value="#{5 * 1024 * 102 * 1024}"/>
>>                  </bean>
>>              </property>
>>          </bean>
>>      </property>
> 
> In other data regions persistence will be disabled by default.
> I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 with these changes.
> 
> Best Regards,
> Ivan Rakov
> 
> On 02.10.2017 18:35, Denis Magda wrote:
>>> To resolve this, I suggest to
>>> introduce just another field defaultRegionConfiguration and get rid of
>>> other defaults in DataStorageConfiguration.
>> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>> 
>> —
>> Denis
>> 
>>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>> 
>>> Agree with Vladimir. If we are to implement this, we would either need to
>>> have a Boolean (non-primitive) for persistenceEnabled on
>>> DataRegionConfiguration, or introduce an enum for this field which is also
>>> an overkill. On the other hand, one can assume that the defaults we are
>>> talking about are actually inherited. To resolve this, I suggest to
>>> introduce just another field defaultRegionConfiguration and get rid of
>>> other defaults in DataStorageConfiguration.
>>> 
>>> Thoughts?
>>> 
>>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>> 
>>>> Vladimir,
>>>> 
>>>> I like your approach because it's easier to implement.
>>>> 
>>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>>> flag and seeing that persistence is not enabled by default in custom memory
>>>> region. I'll add clarifying Javadoc at this place.
>>>> 
>>>> Best Regards,
>>>> Ivan Rakov
>>>> 
>>>> 
>>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>> 
>>>>> Ivan,
>>>>> 
>>>>> I do not think this is correct approach, because it will be hard to
>>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>>> flag for default region only. It should not be propagated to custom
>>>>> regions.
>>>>> 
>>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>>> wrote:
>>>>> 
>>>>> Guys, I think I got the point now.
>>>>>> Let's check the final design:
>>>>>> 
>>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>>> property (default = false), which will be used for enabling persistence
>>>>>> in
>>>>>> default data region.
>>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>>> which
>>>>>> will be used for enabling persistence in corresponding data region. If
>>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>>> efaultPersistenceEnabled*
>>>>>> will be used by default.
>>>>>> 
>>>>>> Best Regards,
>>>>>> Ivan Rakov
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>> 
>>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>> 
>>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>> 
>>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>>> region
>>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>> 
>>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>>> globally.
>>>>>>>> 
>>>>>>> But we should also give users a way to switch the default behavior from
>>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>> 
>>>>>>> 
>>>>>>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Agree with Alexey.

Properties like *defaultDataRegionSize*, *isDefaultPersistenceEnabled* 
can confuse users who don't know that there's such thing as default data 
region. They can decide they are inherited by all data regions where 
size and persistence flag are not explicitly set.

Let's get rid of these properties and add *defaultRegionConfiguration* 
property with explicit configuration of default data region.

Regarding XML configuration, changing size or persistence flag of 
default data region will be just two lines longer (for bean description):

>      <property name="dataStorageConfiguration">
>          <bean 
> class="org.apache.ignite.configuration.DataStorageConfiguration">
>              <property name="systemCacheInitialSize" value="#{100 * 
> 1024 * 1024}"/>
>              <property name="defaultRegionConfiguration">
>                  <bean 
> class="org.apache.ignite.configuration.DataRegionConfiguration">
>                      <property name="maxSize" value="#{5 * 1024 * 102 
> * 1024}"/>
>                  </bean>
>              </property>
>          </bean>
>      </property>

In other data regions persistence will be disabled by default.
I've updated draft in https://issues.apache.org/jira/browse/IGNITE-6030 
with these changes.

Best Regards,
Ivan Rakov

On 02.10.2017 18:35, Denis Magda wrote:
>> To resolve this, I suggest to
>> introduce just another field defaultRegionConfiguration and get rid of
>> other defaults in DataStorageConfiguration.
> Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?
>
> —
> Denis
>
>> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
>>
>> Agree with Vladimir. If we are to implement this, we would either need to
>> have a Boolean (non-primitive) for persistenceEnabled on
>> DataRegionConfiguration, or introduce an enum for this field which is also
>> an overkill. On the other hand, one can assume that the defaults we are
>> talking about are actually inherited. To resolve this, I suggest to
>> introduce just another field defaultRegionConfiguration and get rid of
>> other defaults in DataStorageConfiguration.
>>
>> Thoughts?
>>
>> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>>
>>> Vladimir,
>>>
>>> I like your approach because it's easier to implement.
>>>
>>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>>> flag and seeing that persistence is not enabled by default in custom memory
>>> region. I'll add clarifying Javadoc at this place.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>>>
>>>> Ivan,
>>>>
>>>> I do not think this is correct approach, because it will be hard to
>>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>>> DataRegionConfiguration. I do not think we need default "persistence
>>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>>> flag for default region only. It should not be propagated to custom
>>>> regions.
>>>>
>>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>>> wrote:
>>>>
>>>> Guys, I think I got the point now.
>>>>> Let's check the final design:
>>>>>
>>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>>> property (default = false), which will be used for enabling persistence
>>>>> in
>>>>> default data region.
>>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>>> which
>>>>> will be used for enabling persistence in corresponding data region. If
>>>>> value is not set, value of *DataStorageConfiguration::isD
>>>>> efaultPersistenceEnabled*
>>>>> will be used by default.
>>>>>
>>>>> Best Regards,
>>>>> Ivan Rakov
>>>>>
>>>>>
>>>>>
>>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>>>
>>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>>>
>>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>>>
>>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>>> region
>>>>>>> A only, I don’t want to have it activated for region B.
>>>>>>>
>>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>>> globally.
>>>>>>>
>>>>>> But we should also give users a way to switch the default behavior from
>>>>>> in-memory only (no-persistence) to persistence.
>>>>>>
>>>>>>
>>>>>>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

> To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.

Won’t it complicate the configuration from a Spring XML file? I’m not an expert in Spring so how do I get defaultRegionConfiguration bean first to change any parameter?

—
Denis

> On Oct 2, 2017, at 8:30 AM, Alexey Goncharuk <al...@gmail.com> wrote:
> 
> Agree with Vladimir. If we are to implement this, we would either need to
> have a Boolean (non-primitive) for persistenceEnabled on
> DataRegionConfiguration, or introduce an enum for this field which is also
> an overkill. On the other hand, one can assume that the defaults we are
> talking about are actually inherited. To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.
> 
> Thoughts?
> 
> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
> 
>> Vladimir,
>> 
>> I like your approach because it's easier to implement.
>> 
>> However, user may be confused by setting *isDefaultPersistenceEnabled*
>> flag and seeing that persistence is not enabled by default in custom memory
>> region. I'll add clarifying Javadoc at this place.
>> 
>> Best Regards,
>> Ivan Rakov
>> 
>> 
>> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>> 
>>> Ivan,
>>> 
>>> I do not think this is correct approach, because it will be hard to
>>> explain, and you will have to use "Boolean" instead of "boolean" for
>>> DataRegionConfiguration. I do not think we need default "persistence
>>> enabled" for all regions. Instead, we should have "persistence enabled"
>>> flag for default region only. It should not be propagated to custom
>>> regions.
>>> 
>>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>>> wrote:
>>> 
>>> Guys, I think I got the point now.
>>>> 
>>>> Let's check the final design:
>>>> 
>>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>>> property (default = false), which will be used for enabling persistence
>>>> in
>>>> default data region.
>>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>>> which
>>>> will be used for enabling persistence in corresponding data region. If
>>>> value is not set, value of *DataStorageConfiguration::isD
>>>> efaultPersistenceEnabled*
>>>> will be used by default.
>>>> 
>>>> Best Regards,
>>>> Ivan Rakov
>>>> 
>>>> 
>>>> 
>>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>> 
>>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>> 
>>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>> 
>>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>> 
>>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>>> enabled globally if at least one memory region has this flag set.
>>>>>> 
>>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>>> region
>>>>>> A only, I don’t want to have it activated for region B.
>>>>>> 
>>>>>> Yes, you are right. By default the persistence will be disabled
>>>>>> globally.
>>>>>> 
>>>>> But we should also give users a way to switch the default behavior from
>>>>> in-memory only (no-persistence) to persistence.
>>>>> 
>>>>> 
>>>>> 
>> 

Re: Persistence per memory policy configuration

[Alexey Kuznetsov <ak...@apache.org>]

Guys,

Please, do not add  Boolean (non-primitive) to configurations.
It is really the pain to handle them in tools like WebConsole.
Enum would be better.

I'm not insisting, but just a minor note.


On Mon, Oct 2, 2017 at 10:30 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Agree with Vladimir. If we are to implement this, we would either need to
> have a Boolean (non-primitive) for persistenceEnabled on
> DataRegionConfiguration, or introduce an enum for this field which is also
> an overkill. On the other hand, one can assume that the defaults we are
> talking about are actually inherited. To resolve this, I suggest to
> introduce just another field defaultRegionConfiguration and get rid of
> other defaults in DataStorageConfiguration.
>
> Thoughts?
>
> 2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:
>
> > Vladimir,
> >
> > I like your approach because it's easier to implement.
> >
> > However, user may be confused by setting *isDefaultPersistenceEnabled*
> > flag and seeing that persistence is not enabled by default in custom
> memory
> > region. I'll add clarifying Javadoc at this place.
> >
> > Best Regards,
> > Ivan Rakov
> >
> >
> > On 02.10.2017 11:28, Vladimir Ozerov wrote:
> >
> >> Ivan,
> >>
> >> I do not think this is correct approach, because it will be hard to
> >> explain, and you will have to use "Boolean" instead of "boolean" for
> >> DataRegionConfiguration. I do not think we need default "persistence
> >> enabled" for all regions. Instead, we should have "persistence enabled"
> >> flag for default region only. It should not be propagated to custom
> >> regions.
> >>
> >> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
> >> wrote:
> >>
> >> Guys, I think I got the point now.
> >>>
> >>> Let's check the final design:
> >>>
> >>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
> >>> property (default = false), which will be used for enabling persistence
> >>> in
> >>> default data region.
> >>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
> >>> which
> >>> will be used for enabling persistence in corresponding data region. If
> >>> value is not set, value of *DataStorageConfiguration::isD
> >>> efaultPersistenceEnabled*
> >>> will be used by default.
> >>>
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>>
> >>>
> >>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> >>>
> >>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
> >>>>
> >>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>>>
> >>>>> 1) You're right. I forgot to include the main flag in
> >>>>>>
> >>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will
> be
> >>>>> enabled globally if at least one memory region has this flag set.
> >>>>>
> >>>>> I’m confused. Why the persistence should be enabled *globally* if the
> >>>>> purpose is to have it set for a specific region? If it’s enabled for
> >>>>> region
> >>>>> A only, I don’t want to have it activated for region B.
> >>>>>
> >>>>> Yes, you are right. By default the persistence will be disabled
> >>>>> globally.
> >>>>>
> >>>> But we should also give users a way to switch the default behavior
> from
> >>>> in-memory only (no-persistence) to persistence.
> >>>>
> >>>>
> >>>>
> >
>



-- 
Alexey Kuznetsov

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Agree with Vladimir. If we are to implement this, we would either need to
have a Boolean (non-primitive) for persistenceEnabled on
DataRegionConfiguration, or introduce an enum for this field which is also
an overkill. On the other hand, one can assume that the defaults we are
talking about are actually inherited. To resolve this, I suggest to
introduce just another field defaultRegionConfiguration and get rid of
other defaults in DataStorageConfiguration.

Thoughts?

2017-10-02 15:19 GMT+03:00 Ivan Rakov <iv...@gmail.com>:

> Vladimir,
>
> I like your approach because it's easier to implement.
>
> However, user may be confused by setting *isDefaultPersistenceEnabled*
> flag and seeing that persistence is not enabled by default in custom memory
> region. I'll add clarifying Javadoc at this place.
>
> Best Regards,
> Ivan Rakov
>
>
> On 02.10.2017 11:28, Vladimir Ozerov wrote:
>
>> Ivan,
>>
>> I do not think this is correct approach, because it will be hard to
>> explain, and you will have to use "Boolean" instead of "boolean" for
>> DataRegionConfiguration. I do not think we need default "persistence
>> enabled" for all regions. Instead, we should have "persistence enabled"
>> flag for default region only. It should not be propagated to custom
>> regions.
>>
>> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com>
>> wrote:
>>
>> Guys, I think I got the point now.
>>>
>>> Let's check the final design:
>>>
>>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>>> property (default = false), which will be used for enabling persistence
>>> in
>>> default data region.
>>> *DataRegionConfiguration* will have *isPersistenceEnabled* property,
>>> which
>>> will be used for enabling persistence in corresponding data region. If
>>> value is not set, value of *DataStorageConfiguration::isD
>>> efaultPersistenceEnabled*
>>> will be used by default.
>>>
>>> Best Regards,
>>> Ivan Rakov
>>>
>>>
>>>
>>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>>
>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>>> 1) You're right. I forgot to include the main flag in
>>>>>>
>>>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>>> enabled globally if at least one memory region has this flag set.
>>>>>
>>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>>> region
>>>>> A only, I don’t want to have it activated for region B.
>>>>>
>>>>> Yes, you are right. By default the persistence will be disabled
>>>>> globally.
>>>>>
>>>> But we should also give users a way to switch the default behavior from
>>>> in-memory only (no-persistence) to persistence.
>>>>
>>>>
>>>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Vladimir,

I like your approach because it's easier to implement.

However, user may be confused by setting *isDefaultPersistenceEnabled* 
flag and seeing that persistence is not enabled by default in custom 
memory region. I'll add clarifying Javadoc at this place.

Best Regards,
Ivan Rakov

On 02.10.2017 11:28, Vladimir Ozerov wrote:
> Ivan,
>
> I do not think this is correct approach, because it will be hard to
> explain, and you will have to use "Boolean" instead of "boolean" for
> DataRegionConfiguration. I do not think we need default "persistence
> enabled" for all regions. Instead, we should have "persistence enabled"
> flag for default region only. It should not be propagated to custom regions.
>
> On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com> wrote:
>
>> Guys, I think I got the point now.
>>
>> Let's check the final design:
>>
>> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
>> property (default = false), which will be used for enabling persistence in
>> default data region.
>> *DataRegionConfiguration* will have *isPersistenceEnabled* property, which
>> will be used for enabling persistence in corresponding data region. If
>> value is not set, value of *DataStorageConfiguration::isDefaultPersistenceEnabled*
>> will be used by default.
>>
>> Best Regards,
>> Ivan Rakov
>>
>>
>>
>> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>>
>>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>>
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>> 1) You're right. I forgot to include the main flag in
>>>>>
>>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>>> enabled globally if at least one memory region has this flag set.
>>>>
>>>> I’m confused. Why the persistence should be enabled *globally* if the
>>>> purpose is to have it set for a specific region? If it’s enabled for
>>>> region
>>>> A only, I don’t want to have it activated for region B.
>>>>
>>>> Yes, you are right. By default the persistence will be disabled globally.
>>> But we should also give users a way to switch the default behavior from
>>> in-memory only (no-persistence) to persistence.
>>>
>>>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Ivan,

I do not think this is correct approach, because it will be hard to
explain, and you will have to use "Boolean" instead of "boolean" for
DataRegionConfiguration. I do not think we need default "persistence
enabled" for all regions. Instead, we should have "persistence enabled"
flag for default region only. It should not be propagated to custom regions.

On Mon, Oct 2, 2017 at 11:14 AM, Ivan Rakov <iv...@gmail.com> wrote:

> Guys, I think I got the point now.
>
> Let's check the final design:
>
> *DataStorageConfiguration* will have *isDefaultPersistenceEnabled*
> property (default = false), which will be used for enabling persistence in
> default data region.
> *DataRegionConfiguration* will have *isPersistenceEnabled* property, which
> will be used for enabling persistence in corresponding data region. If
> value is not set, value of *DataStorageConfiguration::isDefaultPersistenceEnabled*
> will be used by default.
>
> Best Regards,
> Ivan Rakov
>
>
>
> On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
>
>> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>>
>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>>
>>>> 1) You're right. I forgot to include the main flag in
>>>>
>>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>>> enabled globally if at least one memory region has this flag set.
>>>
>>> I’m confused. Why the persistence should be enabled *globally* if the
>>> purpose is to have it set for a specific region? If it’s enabled for
>>> region
>>> A only, I don’t want to have it activated for region B.
>>>
>>> Yes, you are right. By default the persistence will be disabled globally.
>> But we should also give users a way to switch the default behavior from
>> in-memory only (no-persistence) to persistence.
>>
>>
>

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Guys, I think I got the point now.

Let's check the final design:

*DataStorageConfiguration* will have *isDefaultPersistenceEnabled* 
property (default = false), which will be used for enabling persistence 
in default data region.
*DataRegionConfiguration* will have *isPersistenceEnabled* property, 
which will be used for enabling persistence in corresponding data 
region. If value is not set, value of 
*DataStorageConfiguration::isDefaultPersistenceEnabled* will be used by 
default.

Best Regards,
Ivan Rakov


On 02.10.2017 7:49, Dmitriy Setrakyan wrote:
> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:
>
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>>
>> I’m confused. Why the persistence should be enabled *globally* if the
>> purpose is to have it set for a specific region? If it’s enabled for region
>> A only, I don’t want to have it activated for region B.
>>
> Yes, you are right. By default the persistence will be disabled globally.
> But we should also give users a way to switch the default behavior from
> in-memory only (no-persistence) to persistence.
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

I am not sure I like the name "isDefaultDataRegionPersistenceEnabled" - too
long. Can we just call it "isDefaultPersistenceEnabled"?

D.

On Sun, Oct 1, 2017 at 2:41 PM, Ivan Rakov <iv...@gmail.com> wrote:

> Denis,
>
> 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
> Regarding default data region, I've added *isDefaultDataRegionPersistenceEnabled
> *to the DataStorageConfiguration. Check the design draft again.
>
> 2) Of course, we have to maintain API compatibility. Deprecating old
> classes and introducing new is just what I meant.
>
> 3) We can't do that - MemoryMetrics are calculated per memory policy and
> PersistenceMetrics are calculated globally. It's a major change to
> implement it another way.
>
> By the way, let's assure the namings for new metrics classes.
> DataRegionMetrics instead of MemoryMetrics looks fine.
> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair enough
> as it will contain only metrics of persistent storage. Probably
> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
> keeping *PersistenceMetrics* are better.
>
> What do you think?
>
> Best Regards,
> Ivan Rakov
>
>
> On 29.09.2017 21:12, Denis Magda wrote:
>
>> Ivan,
>>
>> Several questions/concerns:
>>
>> 1. Looking at the new API I can’t grasp how to enable the persistence.
>> First, how can I enable it globally if there is only one default data
>> region defined. Second, how do I enable it per data region. Can’t find any
>> related switches in the draft.
>>
>> 2. We cannot renamed anything like you’re suggesting to do for
>> MemoryMetrics and their beans. We have to deprecate old and introduce new.
>>
>> 3. I think we should merge Memory and Persistence Metrics into
>> DataStorageMetrics for clarity.
>>
>> —
>> Denis
>>
>>
>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>>
>>> Guys,
>>>
>>> I've attached new configuration design draft to the ticket description:
>>> https://issues.apache.org/jira/browse/IGNITE-6030
>>> Please, take a look. Right now is the best time to change name of any
>>> property.
>>>
>>> And question about metrics: are we going to rename MemoryMetrics and
>>> PersistenceMetrics respectively (along with their MBeans)?
>>> It's not a problem to implement it at all. The only thing that concerns
>>> me is that we have to keep deprecated old classes in the codebase. Perhaps,
>>> MemoryMetrics/PersistenceMetrics are intuitive enough.
>>>
>>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>>
>>>> StorageRegion sounds like bad English to me.
>>>>
>>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>>
>>>> D.
>>>>
>>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
>>>> wrote:
>>>>
>>>> Guys,
>>>>>
>>>>> But what is exact desicion? :-) I saw two final options:
>>>>>
>>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>>
>>>>> Which one we choose?
>>>>>
>>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>>> wrote:
>>>>>
>>>>> I guess it is safe to assume that at this point we came to a consensus?
>>>>>>>
>>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>>
>>>>>> --Yakov
>>>>>>
>>>>>>
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

We merged memory and persistence on config level. So we should merge
metrics in the same way. DataRegionMetrics is absolutely normal name, even
if it contains only persistence-related stuff at the moment.

вс, 1 окт. 2017 г. в 14:41, Ivan Rakov <iv...@gmail.com>:

> Denis,
>
> 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
> Regarding default data region, I've added
> *isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration.
> Check the design draft again.
>
> 2) Of course, we have to maintain API compatibility. Deprecating old
> classes and introducing new is just what I meant.
>
> 3) We can't do that - MemoryMetrics are calculated per memory policy and
> PersistenceMetrics are calculated globally. It's a major change to
> implement it another way.
>
> By the way, let's assure the namings for new metrics classes.
> DataRegionMetrics instead of MemoryMetrics looks fine.
> About PersistenceMetrics - name "*DataStorageMetrics*" is not fair
> enough as it will contain only metrics of persistent storage. Probably
> *DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even
> keeping *PersistenceMetrics* are better.
>
> What do you think?
>
> Best Regards,
> Ivan Rakov
>
> On 29.09.2017 21:12, Denis Magda wrote:
> > Ivan,
> >
> > Several questions/concerns:
> >
> > 1. Looking at the new API I can’t grasp how to enable the persistence.
> First, how can I enable it globally if there is only one default data
> region defined. Second, how do I enable it per data region. Can’t find any
> related switches in the draft.
> >
> > 2. We cannot renamed anything like you’re suggesting to do for
> MemoryMetrics and their beans. We have to deprecate old and introduce new.
> >
> > 3. I think we should merge Memory and Persistence Metrics into
> DataStorageMetrics for clarity.
> >
> > —
> > Denis
> >
> >
> >> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >>
> >> Guys,
> >>
> >> I've attached new configuration design draft to the ticket description:
> https://issues.apache.org/jira/browse/IGNITE-6030
> >> Please, take a look. Right now is the best time to change name of any
> property.
> >>
> >> And question about metrics: are we going to rename MemoryMetrics and
> PersistenceMetrics respectively (along with their MBeans)?
> >> It's not a problem to implement it at all. The only thing that concerns
> me is that we have to keep deprecated old classes in the codebase. Perhaps,
> MemoryMetrics/PersistenceMetrics are intuitive enough.
> >>
> >> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
> >>> StorageRegion sounds like bad English to me.
> >>>
> >>> I would go with DataStorageConfiguration and DataRegionConfiguration.
> >>>
> >>> D.
> >>>
> >>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> >>> wrote:
> >>>
> >>>> Guys,
> >>>>
> >>>> But what is exact desicion? :-) I saw two final options:
> >>>>
> >>>> 1) StorageConfiguration + StorageRegionConfiguration
> >>>> 2) DataStorageConfiguration + DataRegionConfiguration
> >>>>
> >>>> Which one we choose?
> >>>>
> >>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
> >>>> wrote:
> >>>>
> >>>>>> I guess it is safe to assume that at this point we came to a
> consensus?
> >>>>> Alex, I think so. Let's carve it in stone (code).
> >>>>>
> >>>>> --Yakov
> >>>>>
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

> On Oct 1, 2017, at 9:49 PM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dmagda@apache.org <ma...@apache.org>> wrote:
> 
>> 
>>> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <ivan.glukos@gmail.com <ma...@gmail.com>> wrote:
>>> 
>>> 1) You're right. I forgot to include the main flag in
>> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
>> enabled globally if at least one memory region has this flag set.
>> 
>> I’m confused. Why the persistence should be enabled *globally* if the
>> purpose is to have it set for a specific region? If it’s enabled for region
>> A only, I don’t want to have it activated for region B.
>> 
> 
> Yes, you are right. By default the persistence will be disabled globally.
> But we should also give users a way to switch the default behavior from
> in-memory only (no-persistence) to persistence.

Sure, that’s how it was discussed initially. Basically, at the API level we should have:

1. DataRegionConfigurations.isPersistenceEnabled - that turns on/off the persistence per region.

2. DataStorageConfiguration.isPersistenceEnabled - that does this globally except for regions disabled it explicitly via 1.

My confusion is caused by Ivan’s allegation that as soon as the persistence is enabled at least for one region via 1 the persistence will be globally activated for the rest (as I would enable it via 2). Ivan did I get your words in a right way?

—
Denis

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

On Mon, Oct 2, 2017 at 7:46 AM, Denis Magda <dm...@apache.org> wrote:

>
> > On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
> >
> > 1) You're right. I forgot to include the main flag in
> DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be
> enabled globally if at least one memory region has this flag set.
>
> I’m confused. Why the persistence should be enabled *globally* if the
> purpose is to have it set for a specific region? If it’s enabled for region
> A only, I don’t want to have it activated for region B.
>

Yes, you are right. By default the persistence will be disabled globally.
But we should also give users a way to switch the default behavior from
in-memory only (no-persistence) to persistence.

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

> On Oct 1, 2017, at 4:41 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> 1) You're right. I forgot to include the main flag in DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be enabled globally if at least one memory region has this flag set.

I’m confused. Why the persistence should be enabled *globally* if the purpose is to have it set for a specific region? If it’s enabled for region A only, I don’t want to have it activated for region B.

—
Denis
 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Denis,

1) You're right. I forgot to include the main flag in 
DataRegionConfiguration - *isPersistenceEnabled*. Persistence will be 
enabled globally if at least one memory region has this flag set.
Regarding default data region, I've added 
*isDefaultDataRegionPersistenceEnabled *to the DataStorageConfiguration. 
Check the design draft again.

2) Of course, we have to maintain API compatibility. Deprecating old 
classes and introducing new is just what I meant.

3) We can't do that - MemoryMetrics are calculated per memory policy and 
PersistenceMetrics are calculated globally. It's a major change to 
implement it another way.

By the way, let's assure the namings for new metrics classes.
DataRegionMetrics instead of MemoryMetrics looks fine.
About PersistenceMetrics - name "*DataStorageMetrics*" is not fair 
enough as it will contain only metrics of persistent storage. Probably 
*DataStoragePersistenceMetrics*,*PersistentDataStorageMetrics *or even 
keeping *PersistenceMetrics* are better.

What do you think?

Best Regards,
Ivan Rakov

On 29.09.2017 21:12, Denis Magda wrote:
> Ivan,
>
> Several questions/concerns:
>
> 1. Looking at the new API I can’t grasp how to enable the persistence. First, how can I enable it globally if there is only one default data region defined. Second, how do I enable it per data region. Can’t find any related switches in the draft.
>
> 2. We cannot renamed anything like you’re suggesting to do for MemoryMetrics and their beans. We have to deprecate old and introduce new.
>
> 3. I think we should merge Memory and Persistence Metrics into DataStorageMetrics for clarity.
>
> —
> Denis
>
>
>> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
>>
>> Guys,
>>
>> I've attached new configuration design draft to the ticket description: https://issues.apache.org/jira/browse/IGNITE-6030
>> Please, take a look. Right now is the best time to change name of any property.
>>
>> And question about metrics: are we going to rename MemoryMetrics and PersistenceMetrics respectively (along with their MBeans)?
>> It's not a problem to implement it at all. The only thing that concerns me is that we have to keep deprecated old classes in the codebase. Perhaps, MemoryMetrics/PersistenceMetrics are intuitive enough.
>>
>> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>>> StorageRegion sounds like bad English to me.
>>>
>>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>>>
>>> D.
>>>
>>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
>>> wrote:
>>>
>>>> Guys,
>>>>
>>>> But what is exact desicion? :-) I saw two final options:
>>>>
>>>> 1) StorageConfiguration + StorageRegionConfiguration
>>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>>>
>>>> Which one we choose?
>>>>
>>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>>> wrote:
>>>>
>>>>>> I guess it is safe to assume that at this point we came to a consensus?
>>>>> Alex, I think so. Let's carve it in stone (code).
>>>>>
>>>>> --Yakov
>>>>>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Ivan,

Several questions/concerns:

1. Looking at the new API I can’t grasp how to enable the persistence. First, how can I enable it globally if there is only one default data region defined. Second, how do I enable it per data region. Can’t find any related switches in the draft.

2. We cannot renamed anything like you’re suggesting to do for MemoryMetrics and their beans. We have to deprecate old and introduce new. 

3. I think we should merge Memory and Persistence Metrics into DataStorageMetrics for clarity.

—
Denis


> On Sep 29, 2017, at 6:29 AM, Ivan Rakov <iv...@gmail.com> wrote:
> 
> Guys,
> 
> I've attached new configuration design draft to the ticket description: https://issues.apache.org/jira/browse/IGNITE-6030
> Please, take a look. Right now is the best time to change name of any property.
> 
> And question about metrics: are we going to rename MemoryMetrics and PersistenceMetrics respectively (along with their MBeans)?
> It's not a problem to implement it at all. The only thing that concerns me is that we have to keep deprecated old classes in the codebase. Perhaps, MemoryMetrics/PersistenceMetrics are intuitive enough.
> 
> On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
>> StorageRegion sounds like bad English to me.
>> 
>> I would go with DataStorageConfiguration and DataRegionConfiguration.
>> 
>> D.
>> 
>> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
>> wrote:
>> 
>>> Guys,
>>> 
>>> But what is exact desicion? :-) I saw two final options:
>>> 
>>> 1) StorageConfiguration + StorageRegionConfiguration
>>> 2) DataStorageConfiguration + DataRegionConfiguration
>>> 
>>> Which one we choose?
>>> 
>>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>>> wrote:
>>> 
>>>>> I guess it is safe to assume that at this point we came to a consensus?
>>>> Alex, I think so. Let's carve it in stone (code).
>>>> 
>>>> --Yakov
>>>> 
> 

Re: Persistence per memory policy configuration

[Ivan Rakov <iv...@gmail.com>]

Guys,

I've attached new configuration design draft to the ticket description: 
https://issues.apache.org/jira/browse/IGNITE-6030
Please, take a look. Right now is the best time to change name of any 
property.

And question about metrics: are we going to rename MemoryMetrics and 
PersistenceMetrics respectively (along with their MBeans)?
It's not a problem to implement it at all. The only thing that concerns 
me is that we have to keep deprecated old classes in the codebase. 
Perhaps, MemoryMetrics/PersistenceMetrics are intuitive enough.

On 29.09.2017 3:16, Dmitriy Setrakyan wrote:
> StorageRegion sounds like bad English to me.
>
> I would go with DataStorageConfiguration and DataRegionConfiguration.
>
> D.
>
> On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
>> Guys,
>>
>> But what is exact desicion? :-) I saw two final options:
>>
>> 1) StorageConfiguration + StorageRegionConfiguration
>> 2) DataStorageConfiguration + DataRegionConfiguration
>>
>> Which one we choose?
>>
>> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
>> wrote:
>>
>>>> I guess it is safe to assume that at this point we came to a consensus?
>>> Alex, I think so. Let's carve it in stone (code).
>>>
>>> --Yakov
>>>

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

StorageConfiguration + StorageRegionConfiguration

--Yakov

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

StorageRegion sounds like bad English to me.

I would go with DataStorageConfiguration and DataRegionConfiguration.

D.

On Thu, Sep 28, 2017 at 7:24 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> Guys,
>
> But what is exact desicion? :-) I saw two final options:
>
> 1) StorageConfiguration + StorageRegionConfiguration
> 2) DataStorageConfiguration + DataRegionConfiguration
>
> Which one we choose?
>
> On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org>
> wrote:
>
> > > I guess it is safe to assume that at this point we came to a consensus?
> >
> > Alex, I think so. Let's carve it in stone (code).
> >
> > --Yakov
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Guys,

But what is exact desicion? :-) I saw two final options:

1) StorageConfiguration + StorageRegionConfiguration
2) DataStorageConfiguration + DataRegionConfiguration

Which one we choose?

On Thu, Sep 28, 2017 at 5:10 PM, Yakov Zhdanov <yz...@apache.org> wrote:

> > I guess it is safe to assume that at this point we came to a consensus?
>
> Alex, I think so. Let's carve it in stone (code).
>
> --Yakov
>

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

> I guess it is safe to assume that at this point we came to a consensus?

Alex, I think so. Let's carve it in stone (code).

--Yakov

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

My vote also goes for 1. I guess it is safe to assume that at this point we
came to a consensus?

2017-09-27 21:52 GMT+03:00 Denis Magda <dm...@apache.org>:

> Vote for 1.
>
> —
> Denis
>
> > On Sep 26, 2017, at 11:23 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> >
> > Folks,
> >
> > Let me summarize current naming ideas one more time:
> >
> > 1) [StorageConfiguration - StorageRegionConfiguration]
> > 2) [DurableMemoryConfiguration - DataRegionConfiguration]
> > 3) [DurableMemoryConfiguration - DurableMemoryRegionConfiguration] -
> out of
> > question, as "durable memory region" is too misleading.
> >
> > My vote for p.1. Short, simple and intuitive.
> >
> > Vladimir.
> >
> > On Tue, Sep 26, 2017 at 10:16 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> >> On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov <dp...@gmail.com>
> >> wrote:
> >>
> >>> Hi Dmitriy, thank you for reply. Do you agree Memory Policy already
> >> became
> >>> Ignite's term? We call this configuration now
> >> MemoryPolicy(Configuration),
> >>> can we call new configuration elments by their existings name? We can
> >> avoid
> >>> introduction of second Ignite's term in that case.
> >>>
> >>
> >> The refactoring is about merging memory and persistence configuration
> under
> >> the same umbrella. The term "MemoryPolicy" does not make sense anymore,
> >> given that it now also includes persistent configuration as well.
> >>
> >>
> >>>
> >>> вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> >>>
> >>>> Dmitriy, we are not renaming classes, we are refactoring them. Prior
> to
> >>>> this design, it was impossible to set persistence configuration on
> >>>> per-cache basis. With this new design, users will be able to configure
> >>> some
> >>>> caches to be in-memory only and others to be on disk.
> >>>>
> >>>> Given that we are already refactoring, it only makes sense to pick
> >>> better,
> >>>> more appropriate names.
> >>>>
> >>>> D.
> >>>>
> >>>> On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dpavlov.spb@gmail.com
> >
> >>>> wrote:
> >>>>
> >>>>> Vladimir, it is not clear for me, why we need to rename existing
> >>>>> configuration classes. Could you explain? And if we can't get
> >> consensus
> >>>>> now, should we pospond solution?
> >>>>>
> >>>>> My idea is that user needs this feature more than elegant names in
> >>>>> configuration.
> >>>>>
> >>>>> Moreover once MemoryPolicyConfiguration was introduced as Ignite term
> >>> it
> >>>> is
> >>>>> simpler to keep it as is, than create new terms.
> >>>>>
> >>>>> Sincerely,
> >>>>> Dmitriy Pavlov
> >>>>>
> >>>>> вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vozerov@gridgain.com
> >>> :
> >>>>>
> >>>>>> I do not understand why we should delay with renames. Yes, it will
> >>>> cause
> >>>>>> questions, so we will have to put additional efforts to docs and
> >>>>> JavaDocs.
> >>>>>> But the earlier we do that, the better.
> >>>>>>
> >>>>>> On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <
> >>> dpavlov.spb@gmail.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi Igniters, sorry for late response. I didn't catch idea of
> >>>> renaming.
> >>>>>>> PersistentStoreConfiguration is intuitive, and
> >>>>> MemoryPolicyConfiguration
> >>>>>> is
> >>>>>>> intuitive also.
> >>>>>>>
> >>>>>>> If we rename these classes now, it will bring more questions to
> >>> user
> >>>>>> list.
> >>>>>>> Users may be confused by old and new names and by trying to match
> >>> it.
> >>>>>> More
> >>>>>>> issues can came from XML configs that users already have.
> >>>>>>>
> >>>>>>> Can we postpone the renaming? I suggest to finish 'persistence
> >> per
> >>>>> memory
> >>>>>>> policy' task without renaming, and create separate JIRA issue for
> >>>>>> creating
> >>>>>>> future decision?
> >>>>>>>
> >>>>>>> вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com
> >>>>>>>> :
> >>>>>>>
> >>>>>>>> I do not like DurableMemoryConfiguration, because it's quite
> >>>>> confusing
> >>>>>> -
> >>>>>>> we
> >>>>>>>> configure in-memory caches using DurableMemory class, which
> >>>>> immediately
> >>>>>>>> suggests that everything will be persisted. I am not sure if
> >> this
> >>>> is
> >>>>> a
> >>>>>>>> right wording choice for the documentation either. I would go
> >>> with
> >>>>>>>> DataStoreConfiguration and DataRegionConfiguration.
> >>>>>>>>
> >>>>>>>> --AG
> >>>>>>>>
> >>>>>>>> 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org
> >>>>> :
> >>>>>>>>
> >>>>>>>>> Given that we already have a notion of CacheStore which comes
> >>>> from
> >>>>>>> JCache
> >>>>>>>>> spec, I think having other stores may get confusing. I like
> >>>>>>>>> DurableMemoryConfiguration.
> >>>>>>>>>
> >>>>>>>>> Other opinions?
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> >>>>>>> vozerov@gridgain.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Dima, let's finalize the design first.
> >>>>>>>>>>
> >>>>>>>>>> As I understand, we are happy with idea to merge
> >>>>>> MemoryConfiguration
> >>>>>>>>>> and PersistentStoreConfiguration
> >>>>>>>>>> into something what I called DataConfiguration, and to
> >> rename
> >>>>>>>>>> MemoryPolicyConfiguration to DataRegionConfiguration.
> >>>>>>>>>>
> >>>>>>>>>> The only outstanding qurestion is whether DataConfiguration
> >>> is
> >>>> a
> >>>>>> good
> >>>>>>>>> name.
> >>>>>>>>>> I am not very happy with it, so let's think of other
> >>>>> alternatives.
> >>>>>>>> Quick
> >>>>>>>>>> ideas:
> >>>>>>>>>> 1) StoreConfiguration - looks perfect to me - short and
> >>>>>>>> self-describing,
> >>>>>>>>>> but clashes a bit with existing CacheStore
> >>>>>>>>>> 2) DataStoreConfiguration - same as p.1, but the word
> >> "data"
> >>> is
> >>>>> not
> >>>>>>>>>> necessary IMO
> >>>>>>>>>> 3) PageStoreConfiguration? GIves a hint to our page-based
> >>>>>>> architecture.
> >>>>>>>>>> 4) DurableMemoryConfiguration - aligns well with our docs,
> >>> but
> >>>> I
> >>>>> do
> >>>>>>> not
> >>>>>>>>>> like it - too long and misleading
> >>>>>>>>>>
> >>>>>>>>>> Any other ideas?
> >>>>>>>>>>
> >>>>>>>>>> I would prefer to have either [StoreConfiguration +
> >>>>>>>>>> StoreRegionConfiguration] or [PageStoreConfiguration and
> >>>>>>>>>> PageStoreRegionConfiguration]. Looks clean and simple.
> >>>>>>>>>>
> >>>>>>>>>> Vladimir.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Vladimir,
> >>>>>>>>>>>
> >>>>>>>>>>> Can you please add the configuration example in the
> >> ticket?
> >>>>>>>>>>>
> >>>>>>>>>>> D.
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> >>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Guys,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I suggest we finalize the configuration changes in the
> >>>>> original
> >>>>>>>>> ticket
> >>>>>>>>>>>> then: https://issues.apache.org/
> >> jira/browse/IGNITE-6030
> >>>> and
> >>>>>>>> proceed
> >>>>>>>>>> with
> >>>>>>>>>>>> the changes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> >>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>> :
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Can we specify what metrics will look like? I think
> >> we
> >>>>> should
> >>>>>>> not
> >>>>>>>>>> just
> >>>>>>>>>>>>> blindly merge them.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> >>>>>>>>>>> vozerov@gridgain.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Denis,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Makes sense. Thanks for catching it!
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> >>>>>>>> dmagda@apache.org>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> If we’re taking the consolidation path for Memory
> >>> and
> >>>>>>>>> Persistence
> >>>>>>>>>>>>>>> configurations then it makes sense to merge
> >>>>> MemoryMetrics
> >>>>>>> [1]
> >>>>>>>>> and
> >>>>>>>>>>>>>>> PersistenceMetrics [2] plus their JMX beans.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Agree?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> [1]
> >>>>>> https://ignite.apache.org/releases/latest/javadoc/org/
> >>>>>>>>>>>>>>> apache/ignite/MemoryMetrics.html <
> >>>>>>> https://ignite.apache.org/
> >>>>>>>>>>>>>>> releases/latest/javadoc/org/
> >>>>> apache/ignite/MemoryMetrics.
> >>>>>>> html>
> >>>>>>>>>>>>>>> [2]
> >>>>>> https://ignite.apache.org/releases/latest/javadoc/org/
> >>>>>>>>>>>>> apache/ignite/
> >>>>>>>>>>>>>>> PersistenceMetrics.html
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Sep 22, 2017, at 10:18 PM, Dmitriy
> >> Setrakyan <
> >>>>>>>>>>>>> dsetrakyan@apache.org
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I like it.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alexey G, can you please chime in?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Fri, Sep 22, 2017 at 11:33 AM, Vladimir
> >>> Ozerov <
> >>>>>>>>>>>>>> vozerov@gridgain.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Here is my proposal:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> 1) MemoryPolicyConfiguration is renamed to
> >>>>>>>>>>>>> *DataRegionConfiguration*.
> >>>>>>>>>>>>>>>>> 2) PersistenceConfiguration is merged with
> >>>>>>>>> MemoryConfiguration
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> renamed
> >>>>>>>>>>>>>>>>> to ... *DataStorageConfiguration*! It has:
> >>> common
> >>>>>> memory
> >>>>>>>>>>> settings
> >>>>>>>>>>>>>> (e.g.
> >>>>>>>>>>>>>>>>> default data region), persistence settings
> >> (e.g.
> >>>>> WAL)
> >>>>>>> and
> >>>>>>>> a
> >>>>>>>>>> list
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> DataRegionConfiguration beans.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> What we have in the end:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> <property name="dataConfiguration">
> >>>>>>>>>>>>>>>>>   <bean class="o.a.i.DataConfiguration">
> >>>>>>>>>>>>>>>>>       <property name="pageSize" value="8192"
> >> />
> >>>>>>>>>>>>>>>>>       <property name="persistentStorePath"
> >>>>>>>> value="/my/path"
> >>>>>>>>>> />
> >>>>>>>>>>>>>>>>>       <property name="dataRegions">
> >>>>>>>>>>>>>>>>>           <list>
> >>>>>>>>>>>>>>>>>               <bean
> >>>>>>>> class="o.a.i.DataRegionConfiguration">
> >>>>>>>>>>>>>>>>>                   <property name="name"
> >>>>>>> value="VOLATILE"
> >>>>>>>> />
> >>>>>>>>>>>>>>>>>                   <property name="maxSize"
> >>>>>>>>>>> value="1_000_000_000"
> >>>>>>>>>>>> />
> >>>>>>>>>>>>>>>>>               </bean>
> >>>>>>>>>>>>>>>>>               <bean
> >>>>>>>> class="o.a.i.DataRegionConfiguration">
> >>>>>>>>>>>>>>>>>                   <property name="name"
> >>>>>>>> value="PERSISTENT"
> >>>>>>>>> />
> >>>>>>>>>>>>>>>>>                   <property name="maxSize"
> >>>>>>>>>>> value="1_000_000_000"
> >>>>>>>>>>>> />
> >>>>>>>>>>>>>>>>>                   <property name="persistent"
> >>>>>>>> value="true"
> >>>>>>>>> />
> >>>>>>>>>>>>>>>>>               </bean>
> >>>>>>>>>>>>>>>>>           </list>
> >>>>>>>>>>>>>>>>>       </property>
> >>>>>>>>>>>>>>>>>   </bean>
> >>>>>>>>>>>>>>>>> </property>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Makes sense?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Vladimir.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy
> >>>> Setrakyan <
> >>>>>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Firstly all, why not call it DataPolicy
> >> instead
> >>>> of
> >>>>>>>>>>> MemoryPolicy?
> >>>>>>>>>>>>>>>>> Secondly,
> >>>>>>>>>>>>>>>>>> why not set data policies directly on
> >>>>>>>> IgniteConfiguration.
> >>>>>>>>>> And
> >>>>>>>>>>>>>> lastly,
> >>>>>>>>>>>>>>>>> how
> >>>>>>>>>>>>>>>>>> about we combine memory and disk properties
> >> in
> >>>> one
> >>>>>> bean
> >>>>>>>>> with
> >>>>>>>>>>>> clear
> >>>>>>>>>>>>>>> naming
> >>>>>>>>>>>>>>>>>> convention?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Here is the example. Note that all properties
> >>>> above
> >>>>>>> must
> >>>>>>>>>> start
> >>>>>>>>>>>> with
> >>>>>>>>>>>>>>> with
> >>>>>>>>>>>>>>>>>> "Memory" or "Disk".
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *IgniteConfiguration cfg = new
> >>>>>> IgniteConfiguration();*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> *cfg.setDataPolicies(    new
> >>>>>> DataPolicyConfiguration()
> >>>>>>>>>>>>>>>>>>> .setName("bla"),
> >>> .setMemoryMaxSize(1024),
> >>>>> //
> >>>>>>> must
> >>>>>>>>> be
> >>>>>>>>>>>>> greater
> >>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>> 0,
> >>>>>>>>>>>>>>>>>>> since memory always needs to be enabled.
> >>>>>>>>>>>>> .setDiskMaxSize(0),
> >>>>>>>>>>>>>> //
> >>>>>>>>>>>>>>>>> if
> >>>>>>>>>>>>>>>>>>> greater than 0, then persistence is enabled.
> >>>>> );*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I think this approach is much more concise
> >> and
> >>>>>> straight
> >>>>>>>>>>> forward.
> >>>>>>>>>>>>> What
> >>>>>>>>>>>>>>> do
> >>>>>>>>>>>>>>>>>> you think?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir
> >>> Ozerov
> >>>> <
> >>>>>>>>>>>>>> vozerov@gridgain.com
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> I prefer the second. Composition over
> >>>> inheritance
> >>>>> -
> >>>>>>> this
> >>>>>>>>> is
> >>>>>>>>>>> how
> >>>>>>>>>>>>> all
> >>>>>>>>>>>>>>> our
> >>>>>>>>>>>>>>>>>>> configuration is crafted.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> E.g. we do not have "CacheConfiguration"
> >> and "
> >>>>>>>>>>>>>>>>>>> StoreEnabledCacheConfiguration".
> >>>>>>>>>>>>>>>>>>> Instead, we have "CacheConfiguration.
> >>>>>>>>> setCacheStoreFactory".
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey
> >>>> Goncharuk
> >>>>> <
> >>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Reiterating this based on some feedback
> >> from
> >>>> PDS
> >>>>>>> users.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> It might be confusing to configure
> >>> persistence
> >>>>> with
> >>>>>>>>>>>>> "MemoryPolicy",
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>> another approach is to deprecate the old
> >>> names
> >>>>> and
> >>>>>>>>>> introduce
> >>>>>>>>>>> a
> >>>>>>>>>>>>> new
> >>>>>>>>>>>>>>>>> name
> >>>>>>>>>>>>>>>>>>>> "DataRegion" because it reflects the actual
> >>>> state
> >>>>>>> when
> >>>>>>>>> data
> >>>>>>>>>>> is
> >>>>>>>>>>>>>> stored
> >>>>>>>>>>>>>>>>>> on
> >>>>>>>>>>>>>>>>>>>> disk and partially in memory. I have two
> >>>> options
> >>>>> in
> >>>>>>>> mind,
> >>>>>>>>>>> each
> >>>>>>>>>>>> of
> >>>>>>>>>>>>>>>>> them
> >>>>>>>>>>>>>>>>>>>> looks acceptable to me, so I would like to
> >>> have
> >>>>>> some
> >>>>>>>>>> feedback
> >>>>>>>>>>>>> from
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> community. Old configuration names will be
> >>>>>> deprecated
> >>>>>>>>> (but
> >>>>>>>>>>>> still
> >>>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>> taken
> >>>>>>>>>>>>>>>>>>>> if used for backward compatibility). Note,
> >>> that
> >>>>> old
> >>>>>>>> names
> >>>>>>>>>>>>>> deprecation
> >>>>>>>>>>>>>>>>>>>> handles default configuration compatibility
> >>>> very
> >>>>>>>> nicely -
> >>>>>>>>>>>> current
> >>>>>>>>>>>>>> PDS
> >>>>>>>>>>>>>>>>>>> users
> >>>>>>>>>>>>>>>>>>>> will not need to change anything to keep
> >>>>> everything
> >>>>>>>>>> working.
> >>>>>>>>>>>> The
> >>>>>>>>>>>>>> two
> >>>>>>>>>>>>>>>>>>>> options I mentioned are below:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> * we have two separate classes for
> >> in-memory
> >>>> and
> >>>>>>>>> persisted
> >>>>>>>>>>> data
> >>>>>>>>>>>>>>>>>> regions,
> >>>>>>>>>>>>>>>>>>>> so the configuration would look like so:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> IgniteConfiguration cfg = new
> >>>>>> IgniteConfiguration();
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> cfg.setDataRegionsConfiguration(new
> >>>>>>>>>>> DataRegionsConfiguration()
> >>>>>>>>>>>>>>>>>>>>   .setDataRegions(
> >>>>>>>>>>>>>>>>>>>>       new MemoryDataRegion()
> >>>>>>>>>>>>>>>>>>>>           .setName("volatileCaches")
> >>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
> >>>>>>>>>>>>>>>>>>>>       new PersistentDataRegion()
> >>>>>>>>>>>>>>>>>>>>           .setName("persistentCaches")
> >>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...)
> >>>>>>>>>>>>>>>>>>>>           .setMaxDiskSize()));
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> cfg.setPersistentStoreConfiguration(new
> >>>>>>>>>>>>>>>>> PersistentStoreConfiguration()
> >>>>>>>>>>>>>>>>>> );
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> * we have one class for data region
> >>>>> configuration,
> >>>>>>> but
> >>>>>>>> it
> >>>>>>>>>>> will
> >>>>>>>>>>>>>> have a
> >>>>>>>>>>>>>>>>>>>> sub-bean for persistence configuration:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> IgniteConfiguration cfg = new
> >>>>>> IgniteConfiguration();
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> cfg.setDataRegionsConfiguration(new
> >>>>>>>>>>> DataRegionsConfiguration()
> >>>>>>>>>>>>>>>>>>>>   .setDataRegions(
> >>>>>>>>>>>>>>>>>>>>       new DataRegion()
> >>>>>>>>>>>>>>>>>>>>           .setName("volatileCaches")
> >>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
> >>>>>>>>>>>>>>>>>>>>       new DataRegion()
> >>>>>>>>>>>>>>>>>>>>           .setName("persistentCaches")
> >>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
> >>>>>>>>>>>>>>>>>>>>           .setPersistenceConfiguration(
> >>>>>>>>>>>>>>>>>>>>               new
> >>>> DataRegionPersistenceConfigura
> >>>>>>> tion()
> >>>>>>>>>>>>>>>>>>>>                   .setMaxDiskSize(...))));
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> cfg.setPersistentStoreConfiguration(new
> >>>>>>>>>>>>>>>>> PersistentStoreConfiguration()
> >>>>>>>>>>>>>>>>>> );
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

Vote for 1.

—
Denis

> On Sep 26, 2017, at 11:23 PM, Vladimir Ozerov <vo...@gridgain.com> wrote:
> 
> Folks,
> 
> Let me summarize current naming ideas one more time:
> 
> 1) [StorageConfiguration - StorageRegionConfiguration]
> 2) [DurableMemoryConfiguration - DataRegionConfiguration]
> 3) [DurableMemoryConfiguration - DurableMemoryRegionConfiguration] - out of
> question, as "durable memory region" is too misleading.
> 
> My vote for p.1. Short, simple and intuitive.
> 
> Vladimir.
> 
> On Tue, Sep 26, 2017 at 10:16 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> 
>> On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov <dp...@gmail.com>
>> wrote:
>> 
>>> Hi Dmitriy, thank you for reply. Do you agree Memory Policy already
>> became
>>> Ignite's term? We call this configuration now
>> MemoryPolicy(Configuration),
>>> can we call new configuration elments by their existings name? We can
>> avoid
>>> introduction of second Ignite's term in that case.
>>> 
>> 
>> The refactoring is about merging memory and persistence configuration under
>> the same umbrella. The term "MemoryPolicy" does not make sense anymore,
>> given that it now also includes persistent configuration as well.
>> 
>> 
>>> 
>>> вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan <ds...@apache.org>:
>>> 
>>>> Dmitriy, we are not renaming classes, we are refactoring them. Prior to
>>>> this design, it was impossible to set persistence configuration on
>>>> per-cache basis. With this new design, users will be able to configure
>>> some
>>>> caches to be in-memory only and others to be on disk.
>>>> 
>>>> Given that we are already refactoring, it only makes sense to pick
>>> better,
>>>> more appropriate names.
>>>> 
>>>> D.
>>>> 
>>>> On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dp...@gmail.com>
>>>> wrote:
>>>> 
>>>>> Vladimir, it is not clear for me, why we need to rename existing
>>>>> configuration classes. Could you explain? And if we can't get
>> consensus
>>>>> now, should we pospond solution?
>>>>> 
>>>>> My idea is that user needs this feature more than elegant names in
>>>>> configuration.
>>>>> 
>>>>> Moreover once MemoryPolicyConfiguration was introduced as Ignite term
>>> it
>>>> is
>>>>> simpler to keep it as is, than create new terms.
>>>>> 
>>>>> Sincerely,
>>>>> Dmitriy Pavlov
>>>>> 
>>>>> вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vozerov@gridgain.com
>>> :
>>>>> 
>>>>>> I do not understand why we should delay with renames. Yes, it will
>>>> cause
>>>>>> questions, so we will have to put additional efforts to docs and
>>>>> JavaDocs.
>>>>>> But the earlier we do that, the better.
>>>>>> 
>>>>>> On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <
>>> dpavlov.spb@gmail.com>
>>>>>> wrote:
>>>>>> 
>>>>>>> Hi Igniters, sorry for late response. I didn't catch idea of
>>>> renaming.
>>>>>>> PersistentStoreConfiguration is intuitive, and
>>>>> MemoryPolicyConfiguration
>>>>>> is
>>>>>>> intuitive also.
>>>>>>> 
>>>>>>> If we rename these classes now, it will bring more questions to
>>> user
>>>>>> list.
>>>>>>> Users may be confused by old and new names and by trying to match
>>> it.
>>>>>> More
>>>>>>> issues can came from XML configs that users already have.
>>>>>>> 
>>>>>>> Can we postpone the renaming? I suggest to finish 'persistence
>> per
>>>>> memory
>>>>>>> policy' task without renaming, and create separate JIRA issue for
>>>>>> creating
>>>>>>> future decision?
>>>>>>> 
>>>>>>> вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com
>>>>>>>> :
>>>>>>> 
>>>>>>>> I do not like DurableMemoryConfiguration, because it's quite
>>>>> confusing
>>>>>> -
>>>>>>> we
>>>>>>>> configure in-memory caches using DurableMemory class, which
>>>>> immediately
>>>>>>>> suggests that everything will be persisted. I am not sure if
>> this
>>>> is
>>>>> a
>>>>>>>> right wording choice for the documentation either. I would go
>>> with
>>>>>>>> DataStoreConfiguration and DataRegionConfiguration.
>>>>>>>> 
>>>>>>>> --AG
>>>>>>>> 
>>>>>>>> 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <
>>> dsetrakyan@apache.org
>>>>> :
>>>>>>>> 
>>>>>>>>> Given that we already have a notion of CacheStore which comes
>>>> from
>>>>>>> JCache
>>>>>>>>> spec, I think having other stores may get confusing. I like
>>>>>>>>> DurableMemoryConfiguration.
>>>>>>>>> 
>>>>>>>>> Other opinions?
>>>>>>>>> 
>>>>>>>>> D.
>>>>>>>>> 
>>>>>>>>> On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
>>>>>>> vozerov@gridgain.com>
>>>>>>>>> wrote:
>>>>>>>>> 
>>>>>>>>>> Dima, let's finalize the design first.
>>>>>>>>>> 
>>>>>>>>>> As I understand, we are happy with idea to merge
>>>>>> MemoryConfiguration
>>>>>>>>>> and PersistentStoreConfiguration
>>>>>>>>>> into something what I called DataConfiguration, and to
>> rename
>>>>>>>>>> MemoryPolicyConfiguration to DataRegionConfiguration.
>>>>>>>>>> 
>>>>>>>>>> The only outstanding qurestion is whether DataConfiguration
>>> is
>>>> a
>>>>>> good
>>>>>>>>> name.
>>>>>>>>>> I am not very happy with it, so let's think of other
>>>>> alternatives.
>>>>>>>> Quick
>>>>>>>>>> ideas:
>>>>>>>>>> 1) StoreConfiguration - looks perfect to me - short and
>>>>>>>> self-describing,
>>>>>>>>>> but clashes a bit with existing CacheStore
>>>>>>>>>> 2) DataStoreConfiguration - same as p.1, but the word
>> "data"
>>> is
>>>>> not
>>>>>>>>>> necessary IMO
>>>>>>>>>> 3) PageStoreConfiguration? GIves a hint to our page-based
>>>>>>> architecture.
>>>>>>>>>> 4) DurableMemoryConfiguration - aligns well with our docs,
>>> but
>>>> I
>>>>> do
>>>>>>> not
>>>>>>>>>> like it - too long and misleading
>>>>>>>>>> 
>>>>>>>>>> Any other ideas?
>>>>>>>>>> 
>>>>>>>>>> I would prefer to have either [StoreConfiguration +
>>>>>>>>>> StoreRegionConfiguration] or [PageStoreConfiguration and
>>>>>>>>>> PageStoreRegionConfiguration]. Looks clean and simple.
>>>>>>>>>> 
>>>>>>>>>> Vladimir.
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>> 
>>>>>>>>>>> Vladimir,
>>>>>>>>>>> 
>>>>>>>>>>> Can you please add the configuration example in the
>> ticket?
>>>>>>>>>>> 
>>>>>>>>>>> D.
>>>>>>>>>>> 
>>>>>>>>>>> On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Guys,
>>>>>>>>>>>> 
>>>>>>>>>>>> I suggest we finalize the configuration changes in the
>>>>> original
>>>>>>>>> ticket
>>>>>>>>>>>> then: https://issues.apache.org/
>> jira/browse/IGNITE-6030
>>>> and
>>>>>>>> proceed
>>>>>>>>>> with
>>>>>>>>>>>> the changes.
>>>>>>>>>>>> 
>>>>>>>>>>>> 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>> :
>>>>>>>>>>>> 
>>>>>>>>>>>>> Can we specify what metrics will look like? I think
>> we
>>>>> should
>>>>>>> not
>>>>>>>>>> just
>>>>>>>>>>>>> blindly merge them.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
>>>>>>>>>>> vozerov@gridgain.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Denis,
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Makes sense. Thanks for catching it!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
>>>>>>>> dmagda@apache.org>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> If we’re taking the consolidation path for Memory
>>> and
>>>>>>>>> Persistence
>>>>>>>>>>>>>>> configurations then it makes sense to merge
>>>>> MemoryMetrics
>>>>>>> [1]
>>>>>>>>> and
>>>>>>>>>>>>>>> PersistenceMetrics [2] plus their JMX beans.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Agree?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> [1]
>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/
>>>>>>>>>>>>>>> apache/ignite/MemoryMetrics.html <
>>>>>>> https://ignite.apache.org/
>>>>>>>>>>>>>>> releases/latest/javadoc/org/
>>>>> apache/ignite/MemoryMetrics.
>>>>>>> html>
>>>>>>>>>>>>>>> [2]
>>>>>> https://ignite.apache.org/releases/latest/javadoc/org/
>>>>>>>>>>>>> apache/ignite/
>>>>>>>>>>>>>>> PersistenceMetrics.html
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Sep 22, 2017, at 10:18 PM, Dmitriy
>> Setrakyan <
>>>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> I like it.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Alexey G, can you please chime in?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On Fri, Sep 22, 2017 at 11:33 AM, Vladimir
>>> Ozerov <
>>>>>>>>>>>>>> vozerov@gridgain.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Here is my proposal:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 1) MemoryPolicyConfiguration is renamed to
>>>>>>>>>>>>> *DataRegionConfiguration*.
>>>>>>>>>>>>>>>>> 2) PersistenceConfiguration is merged with
>>>>>>>>> MemoryConfiguration
>>>>>>>>>>> and
>>>>>>>>>>>>>>> renamed
>>>>>>>>>>>>>>>>> to ... *DataStorageConfiguration*! It has:
>>> common
>>>>>> memory
>>>>>>>>>>> settings
>>>>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>>>>> default data region), persistence settings
>> (e.g.
>>>>> WAL)
>>>>>>> and
>>>>>>>> a
>>>>>>>>>> list
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> DataRegionConfiguration beans.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> What we have in the end:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> <property name="dataConfiguration">
>>>>>>>>>>>>>>>>>   <bean class="o.a.i.DataConfiguration">
>>>>>>>>>>>>>>>>>       <property name="pageSize" value="8192"
>> />
>>>>>>>>>>>>>>>>>       <property name="persistentStorePath"
>>>>>>>> value="/my/path"
>>>>>>>>>> />
>>>>>>>>>>>>>>>>>       <property name="dataRegions">
>>>>>>>>>>>>>>>>>           <list>
>>>>>>>>>>>>>>>>>               <bean
>>>>>>>> class="o.a.i.DataRegionConfiguration">
>>>>>>>>>>>>>>>>>                   <property name="name"
>>>>>>> value="VOLATILE"
>>>>>>>> />
>>>>>>>>>>>>>>>>>                   <property name="maxSize"
>>>>>>>>>>> value="1_000_000_000"
>>>>>>>>>>>> />
>>>>>>>>>>>>>>>>>               </bean>
>>>>>>>>>>>>>>>>>               <bean
>>>>>>>> class="o.a.i.DataRegionConfiguration">
>>>>>>>>>>>>>>>>>                   <property name="name"
>>>>>>>> value="PERSISTENT"
>>>>>>>>> />
>>>>>>>>>>>>>>>>>                   <property name="maxSize"
>>>>>>>>>>> value="1_000_000_000"
>>>>>>>>>>>> />
>>>>>>>>>>>>>>>>>                   <property name="persistent"
>>>>>>>> value="true"
>>>>>>>>> />
>>>>>>>>>>>>>>>>>               </bean>
>>>>>>>>>>>>>>>>>           </list>
>>>>>>>>>>>>>>>>>       </property>
>>>>>>>>>>>>>>>>>   </bean>
>>>>>>>>>>>>>>>>> </property>
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Makes sense?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Vladimir.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy
>>>> Setrakyan <
>>>>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Firstly all, why not call it DataPolicy
>> instead
>>>> of
>>>>>>>>>>> MemoryPolicy?
>>>>>>>>>>>>>>>>> Secondly,
>>>>>>>>>>>>>>>>>> why not set data policies directly on
>>>>>>>> IgniteConfiguration.
>>>>>>>>>> And
>>>>>>>>>>>>>> lastly,
>>>>>>>>>>>>>>>>> how
>>>>>>>>>>>>>>>>>> about we combine memory and disk properties
>> in
>>>> one
>>>>>> bean
>>>>>>>>> with
>>>>>>>>>>>> clear
>>>>>>>>>>>>>>> naming
>>>>>>>>>>>>>>>>>> convention?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Here is the example. Note that all properties
>>>> above
>>>>>>> must
>>>>>>>>>> start
>>>>>>>>>>>> with
>>>>>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> "Memory" or "Disk".
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> *IgniteConfiguration cfg = new
>>>>>> IgniteConfiguration();*
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> *cfg.setDataPolicies(    new
>>>>>> DataPolicyConfiguration()
>>>>>>>>>>>>>>>>>>> .setName("bla"),
>>> .setMemoryMaxSize(1024),
>>>>> //
>>>>>>> must
>>>>>>>>> be
>>>>>>>>>>>>> greater
>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>> 0,
>>>>>>>>>>>>>>>>>>> since memory always needs to be enabled.
>>>>>>>>>>>>> .setDiskMaxSize(0),
>>>>>>>>>>>>>> //
>>>>>>>>>>>>>>>>> if
>>>>>>>>>>>>>>>>>>> greater than 0, then persistence is enabled.
>>>>> );*
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> I think this approach is much more concise
>> and
>>>>>> straight
>>>>>>>>>>> forward.
>>>>>>>>>>>>> What
>>>>>>>>>>>>>>> do
>>>>>>>>>>>>>>>>>> you think?
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir
>>> Ozerov
>>>> <
>>>>>>>>>>>>>> vozerov@gridgain.com
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> I prefer the second. Composition over
>>>> inheritance
>>>>> -
>>>>>>> this
>>>>>>>>> is
>>>>>>>>>>> how
>>>>>>>>>>>>> all
>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>>>> configuration is crafted.
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> E.g. we do not have "CacheConfiguration"
>> and "
>>>>>>>>>>>>>>>>>>> StoreEnabledCacheConfiguration".
>>>>>>>>>>>>>>>>>>> Instead, we have "CacheConfiguration.
>>>>>>>>> setCacheStoreFactory".
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey
>>>> Goncharuk
>>>>> <
>>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> Reiterating this based on some feedback
>> from
>>>> PDS
>>>>>>> users.
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> It might be confusing to configure
>>> persistence
>>>>> with
>>>>>>>>>>>>> "MemoryPolicy",
>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>> another approach is to deprecate the old
>>> names
>>>>> and
>>>>>>>>>> introduce
>>>>>>>>>>> a
>>>>>>>>>>>>> new
>>>>>>>>>>>>>>>>> name
>>>>>>>>>>>>>>>>>>>> "DataRegion" because it reflects the actual
>>>> state
>>>>>>> when
>>>>>>>>> data
>>>>>>>>>>> is
>>>>>>>>>>>>>> stored
>>>>>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>>>>>> disk and partially in memory. I have two
>>>> options
>>>>> in
>>>>>>>> mind,
>>>>>>>>>>> each
>>>>>>>>>>>> of
>>>>>>>>>>>>>>>>> them
>>>>>>>>>>>>>>>>>>>> looks acceptable to me, so I would like to
>>> have
>>>>>> some
>>>>>>>>>> feedback
>>>>>>>>>>>>> from
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> community. Old configuration names will be
>>>>>> deprecated
>>>>>>>>> (but
>>>>>>>>>>>> still
>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>> taken
>>>>>>>>>>>>>>>>>>>> if used for backward compatibility). Note,
>>> that
>>>>> old
>>>>>>>> names
>>>>>>>>>>>>>> deprecation
>>>>>>>>>>>>>>>>>>>> handles default configuration compatibility
>>>> very
>>>>>>>> nicely -
>>>>>>>>>>>> current
>>>>>>>>>>>>>> PDS
>>>>>>>>>>>>>>>>>>> users
>>>>>>>>>>>>>>>>>>>> will not need to change anything to keep
>>>>> everything
>>>>>>>>>> working.
>>>>>>>>>>>> The
>>>>>>>>>>>>>> two
>>>>>>>>>>>>>>>>>>>> options I mentioned are below:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> * we have two separate classes for
>> in-memory
>>>> and
>>>>>>>>> persisted
>>>>>>>>>>> data
>>>>>>>>>>>>>>>>>> regions,
>>>>>>>>>>>>>>>>>>>> so the configuration would look like so:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> IgniteConfiguration cfg = new
>>>>>> IgniteConfiguration();
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> cfg.setDataRegionsConfiguration(new
>>>>>>>>>>> DataRegionsConfiguration()
>>>>>>>>>>>>>>>>>>>>   .setDataRegions(
>>>>>>>>>>>>>>>>>>>>       new MemoryDataRegion()
>>>>>>>>>>>>>>>>>>>>           .setName("volatileCaches")
>>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
>>>>>>>>>>>>>>>>>>>>       new PersistentDataRegion()
>>>>>>>>>>>>>>>>>>>>           .setName("persistentCaches")
>>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...)
>>>>>>>>>>>>>>>>>>>>           .setMaxDiskSize()));
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> cfg.setPersistentStoreConfiguration(new
>>>>>>>>>>>>>>>>> PersistentStoreConfiguration()
>>>>>>>>>>>>>>>>>> );
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> * we have one class for data region
>>>>> configuration,
>>>>>>> but
>>>>>>>> it
>>>>>>>>>>> will
>>>>>>>>>>>>>> have a
>>>>>>>>>>>>>>>>>>>> sub-bean for persistence configuration:
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> IgniteConfiguration cfg = new
>>>>>> IgniteConfiguration();
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> cfg.setDataRegionsConfiguration(new
>>>>>>>>>>> DataRegionsConfiguration()
>>>>>>>>>>>>>>>>>>>>   .setDataRegions(
>>>>>>>>>>>>>>>>>>>>       new DataRegion()
>>>>>>>>>>>>>>>>>>>>           .setName("volatileCaches")
>>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
>>>>>>>>>>>>>>>>>>>>       new DataRegion()
>>>>>>>>>>>>>>>>>>>>           .setName("persistentCaches")
>>>>>>>>>>>>>>>>>>>>           .setMaxMemorySize(...),
>>>>>>>>>>>>>>>>>>>>           .setPersistenceConfiguration(
>>>>>>>>>>>>>>>>>>>>               new
>>>> DataRegionPersistenceConfigura
>>>>>>> tion()
>>>>>>>>>>>>>>>>>>>>                   .setMaxDiskSize(...))));
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>>> cfg.setPersistentStoreConfiguration(new
>>>>>>>>>>>>>>>>> PersistentStoreConfiguration()
>>>>>>>>>>>>>>>>>> );
>>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 
>>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Folks,

Let me summarize current naming ideas one more time:

1) [StorageConfiguration - StorageRegionConfiguration]
2) [DurableMemoryConfiguration - DataRegionConfiguration]
3) [DurableMemoryConfiguration - DurableMemoryRegionConfiguration] - out of
question, as "durable memory region" is too misleading.

My vote for p.1. Short, simple and intuitive.

Vladimir.

On Tue, Sep 26, 2017 at 10:16 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov <dp...@gmail.com>
> wrote:
>
> > Hi Dmitriy, thank you for reply. Do you agree Memory Policy already
> became
> > Ignite's term? We call this configuration now
> MemoryPolicy(Configuration),
> > can we call new configuration elments by their existings name? We can
> avoid
> > introduction of second Ignite's term in that case.
> >
>
> The refactoring is about merging memory and persistence configuration under
> the same umbrella. The term "MemoryPolicy" does not make sense anymore,
> given that it now also includes persistent configuration as well.
>
>
> >
> > вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan <ds...@apache.org>:
> >
> > > Dmitriy, we are not renaming classes, we are refactoring them. Prior to
> > > this design, it was impossible to set persistence configuration on
> > > per-cache basis. With this new design, users will be able to configure
> > some
> > > caches to be in-memory only and others to be on disk.
> > >
> > > Given that we are already refactoring, it only makes sense to pick
> > better,
> > > more appropriate names.
> > >
> > > D.
> > >
> > > On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dp...@gmail.com>
> > > wrote:
> > >
> > > > Vladimir, it is not clear for me, why we need to rename existing
> > > > configuration classes. Could you explain? And if we can't get
> consensus
> > > > now, should we pospond solution?
> > > >
> > > > My idea is that user needs this feature more than elegant names in
> > > > configuration.
> > > >
> > > > Moreover once MemoryPolicyConfiguration was introduced as Ignite term
> > it
> > > is
> > > > simpler to keep it as is, than create new terms.
> > > >
> > > > Sincerely,
> > > > Dmitriy Pavlov
> > > >
> > > > вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vozerov@gridgain.com
> >:
> > > >
> > > > > I do not understand why we should delay with renames. Yes, it will
> > > cause
> > > > > questions, so we will have to put additional efforts to docs and
> > > > JavaDocs.
> > > > > But the earlier we do that, the better.
> > > > >
> > > > > On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <
> > dpavlov.spb@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hi Igniters, sorry for late response. I didn't catch idea of
> > > renaming.
> > > > > > PersistentStoreConfiguration is intuitive, and
> > > > MemoryPolicyConfiguration
> > > > > is
> > > > > > intuitive also.
> > > > > >
> > > > > > If we rename these classes now, it will bring more questions to
> > user
> > > > > list.
> > > > > > Users may be confused by old and new names and by trying to match
> > it.
> > > > > More
> > > > > > issues can came from XML configs that users already have.
> > > > > >
> > > > > > Can we postpone the renaming? I suggest to finish 'persistence
> per
> > > > memory
> > > > > > policy' task without renaming, and create separate JIRA issue for
> > > > > creating
> > > > > > future decision?
> > > > > >
> > > > > > вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com
> > > > > > >:
> > > > > >
> > > > > > > I do not like DurableMemoryConfiguration, because it's quite
> > > > confusing
> > > > > -
> > > > > > we
> > > > > > > configure in-memory caches using DurableMemory class, which
> > > > immediately
> > > > > > > suggests that everything will be persisted. I am not sure if
> this
> > > is
> > > > a
> > > > > > > right wording choice for the documentation either. I would go
> > with
> > > > > > > DataStoreConfiguration and DataRegionConfiguration.
> > > > > > >
> > > > > > > --AG
> > > > > > >
> > > > > > > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <
> > dsetrakyan@apache.org
> > > >:
> > > > > > >
> > > > > > > > Given that we already have a notion of CacheStore which comes
> > > from
> > > > > > JCache
> > > > > > > > spec, I think having other stores may get confusing. I like
> > > > > > > > DurableMemoryConfiguration.
> > > > > > > >
> > > > > > > > Other opinions?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> > > > > > vozerov@gridgain.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Dima, let's finalize the design first.
> > > > > > > > >
> > > > > > > > > As I understand, we are happy with idea to merge
> > > > > MemoryConfiguration
> > > > > > > > > and PersistentStoreConfiguration
> > > > > > > > > into something what I called DataConfiguration, and to
> rename
> > > > > > > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > > > > > > >
> > > > > > > > > The only outstanding qurestion is whether DataConfiguration
> > is
> > > a
> > > > > good
> > > > > > > > name.
> > > > > > > > > I am not very happy with it, so let's think of other
> > > > alternatives.
> > > > > > > Quick
> > > > > > > > > ideas:
> > > > > > > > > 1) StoreConfiguration - looks perfect to me - short and
> > > > > > > self-describing,
> > > > > > > > > but clashes a bit with existing CacheStore
> > > > > > > > > 2) DataStoreConfiguration - same as p.1, but the word
> "data"
> > is
> > > > not
> > > > > > > > > necessary IMO
> > > > > > > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> > > > > > architecture.
> > > > > > > > > 4) DurableMemoryConfiguration - aligns well with our docs,
> > but
> > > I
> > > > do
> > > > > > not
> > > > > > > > > like it - too long and misleading
> > > > > > > > >
> > > > > > > > > Any other ideas?
> > > > > > > > >
> > > > > > > > > I would prefer to have either [StoreConfiguration +
> > > > > > > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > > > > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > > > > > > >
> > > > > > > > > Vladimir.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Vladimir,
> > > > > > > > > >
> > > > > > > > > > Can you please add the configuration example in the
> ticket?
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > >
> > > > > > > > > > > Guys,
> > > > > > > > > > >
> > > > > > > > > > > I suggest we finalize the configuration changes in the
> > > > original
> > > > > > > > ticket
> > > > > > > > > > > then: https://issues.apache.org/
> jira/browse/IGNITE-6030
> > > and
> > > > > > > proceed
> > > > > > > > > with
> > > > > > > > > > > the changes.
> > > > > > > > > > >
> > > > > > > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org
> > > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > Can we specify what metrics will look like? I think
> we
> > > > should
> > > > > > not
> > > > > > > > > just
> > > > > > > > > > > > blindly merge them.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Denis,
> > > > > > > > > > > > >
> > > > > > > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > > > > > > dmagda@apache.org>
> > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > If we’re taking the consolidation path for Memory
> > and
> > > > > > > > Persistence
> > > > > > > > > > > > > > configurations then it makes sense to merge
> > > > MemoryMetrics
> > > > > > [1]
> > > > > > > > and
> > > > > > > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Agree?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > [1]
> > > > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > > > > apache/ignite/MemoryMetrics.html <
> > > > > > https://ignite.apache.org/
> > > > > > > > > > > > > > releases/latest/javadoc/org/
> > > > apache/ignite/MemoryMetrics.
> > > > > > html>
> > > > > > > > > > > > > > [2]
> > > > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > > apache/ignite/
> > > > > > > > > > > > > > PersistenceMetrics.html
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > —
> > > > > > > > > > > > > > Denis
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy
> Setrakyan <
> > > > > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I like it.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > D.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir
> > Ozerov <
> > > > > > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >> Guys,
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Here is my proposal:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > > > > > > *DataRegionConfiguration*.
> > > > > > > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > > > > > > MemoryConfiguration
> > > > > > > > > > and
> > > > > > > > > > > > > > renamed
> > > > > > > > > > > > > > >> to ... *DataStorageConfiguration*! It has:
> > common
> > > > > memory
> > > > > > > > > > settings
> > > > > > > > > > > > > (e.g.
> > > > > > > > > > > > > > >> default data region), persistence settings
> (e.g.
> > > > WAL)
> > > > > > and
> > > > > > > a
> > > > > > > > > list
> > > > > > > > > > > of
> > > > > > > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> What we have in the end:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > > > > > > >>        <property name="pageSize" value="8192"
> />
> > > > > > > > > > > > > > >>        <property name="persistentStorePath"
> > > > > > > value="/my/path"
> > > > > > > > > />
> > > > > > > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > > > > > > >>            <list>
> > > > > > > > > > > > > > >>                <bean
> > > > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > > > >>                    <property name="name"
> > > > > > value="VOLATILE"
> > > > > > > />
> > > > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > > > value="1_000_000_000"
> > > > > > > > > > > />
> > > > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > > > >>                <bean
> > > > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > > > >>                    <property name="name"
> > > > > > > value="PERSISTENT"
> > > > > > > > />
> > > > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > > > value="1_000_000_000"
> > > > > > > > > > > />
> > > > > > > > > > > > > > >>                    <property name="persistent"
> > > > > > > value="true"
> > > > > > > > />
> > > > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > > > >>            </list>
> > > > > > > > > > > > > > >>        </property>
> > > > > > > > > > > > > > >>    </bean>
> > > > > > > > > > > > > > >> </property>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Makes sense?
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> Vladimir.
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy
> > > Setrakyan <
> > > > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > >>> Firstly all, why not call it DataPolicy
> instead
> > > of
> > > > > > > > > > MemoryPolicy?
> > > > > > > > > > > > > > >> Secondly,
> > > > > > > > > > > > > > >>> why not set data policies directly on
> > > > > > > IgniteConfiguration.
> > > > > > > > > And
> > > > > > > > > > > > > lastly,
> > > > > > > > > > > > > > >> how
> > > > > > > > > > > > > > >>> about we combine memory and disk properties
> in
> > > one
> > > > > bean
> > > > > > > > with
> > > > > > > > > > > clear
> > > > > > > > > > > > > > naming
> > > > > > > > > > > > > > >>> convention?
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> Here is the example. Note that all properties
> > > above
> > > > > > must
> > > > > > > > > start
> > > > > > > > > > > with
> > > > > > > > > > > > > > with
> > > > > > > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> *IgniteConfiguration cfg = new
> > > > > IgniteConfiguration();*
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> *cfg.setDataPolicies(    new
> > > > > DataPolicyConfiguration()
> > > > > > > > > > > > > > >>>> .setName("bla"),
> > .setMemoryMaxSize(1024),
> > > > //
> > > > > > must
> > > > > > > > be
> > > > > > > > > > > > greater
> > > > > > > > > > > > > > >> than
> > > > > > > > > > > > > > >>> 0,
> > > > > > > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > > > > > > .setDiskMaxSize(0),
> > > > > > > > > > > > > //
> > > > > > > > > > > > > > >> if
> > > > > > > > > > > > > > >>>> greater than 0, then persistence is enabled.
> > > > );*
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> I think this approach is much more concise
> and
> > > > > straight
> > > > > > > > > > forward.
> > > > > > > > > > > > What
> > > > > > > > > > > > > > do
> > > > > > > > > > > > > > >>> you think?
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> D.
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir
> > Ozerov
> > > <
> > > > > > > > > > > > > vozerov@gridgain.com
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>>> I prefer the second. Composition over
> > > inheritance
> > > > -
> > > > > > this
> > > > > > > > is
> > > > > > > > > > how
> > > > > > > > > > > > all
> > > > > > > > > > > > > > our
> > > > > > > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> E.g. we do not have "CacheConfiguration"
> and "
> > > > > > > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > > > > > > setCacheStoreFactory".
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey
> > > Goncharuk
> > > > <
> > > > > > > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>>> Reiterating this based on some feedback
> from
> > > PDS
> > > > > > users.
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> It might be confusing to configure
> > persistence
> > > > with
> > > > > > > > > > > > "MemoryPolicy",
> > > > > > > > > > > > > > >> so
> > > > > > > > > > > > > > >>>>> another approach is to deprecate the old
> > names
> > > > and
> > > > > > > > > introduce
> > > > > > > > > > a
> > > > > > > > > > > > new
> > > > > > > > > > > > > > >> name
> > > > > > > > > > > > > > >>>>> "DataRegion" because it reflects the actual
> > > state
> > > > > > when
> > > > > > > > data
> > > > > > > > > > is
> > > > > > > > > > > > > stored
> > > > > > > > > > > > > > >>> on
> > > > > > > > > > > > > > >>>>> disk and partially in memory. I have two
> > > options
> > > > in
> > > > > > > mind,
> > > > > > > > > > each
> > > > > > > > > > > of
> > > > > > > > > > > > > > >> them
> > > > > > > > > > > > > > >>>>> looks acceptable to me, so I would like to
> > have
> > > > > some
> > > > > > > > > feedback
> > > > > > > > > > > > from
> > > > > > > > > > > > > > >> the
> > > > > > > > > > > > > > >>>>> community. Old configuration names will be
> > > > > deprecated
> > > > > > > > (but
> > > > > > > > > > > still
> > > > > > > > > > > > be
> > > > > > > > > > > > > > >>> taken
> > > > > > > > > > > > > > >>>>> if used for backward compatibility). Note,
> > that
> > > > old
> > > > > > > names
> > > > > > > > > > > > > deprecation
> > > > > > > > > > > > > > >>>>> handles default configuration compatibility
> > > very
> > > > > > > nicely -
> > > > > > > > > > > current
> > > > > > > > > > > > > PDS
> > > > > > > > > > > > > > >>>> users
> > > > > > > > > > > > > > >>>>> will not need to change anything to keep
> > > > everything
> > > > > > > > > working.
> > > > > > > > > > > The
> > > > > > > > > > > > > two
> > > > > > > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> * we have two separate classes for
> in-memory
> > > and
> > > > > > > > persisted
> > > > > > > > > > data
> > > > > > > > > > > > > > >>> regions,
> > > > > > > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > > > IgniteConfiguration();
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > > > >>> );
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> * we have one class for data region
> > > > configuration,
> > > > > > but
> > > > > > > it
> > > > > > > > > > will
> > > > > > > > > > > > > have a
> > > > > > > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > > > IgniteConfiguration();
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > > > > > > >>>>>                new
> > > DataRegionPersistenceConfigura
> > > > > > tion()
> > > > > > > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > > > >>> );
> > > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > >>
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

On Tue, Sep 26, 2017 at 7:33 AM, Dmitry Pavlov <dp...@gmail.com>
wrote:

> Hi Dmitriy, thank you for reply. Do you agree Memory Policy already became
> Ignite's term? We call this configuration now MemoryPolicy(Configuration),
> can we call new configuration elments by their existings name? We can avoid
> introduction of second Ignite's term in that case.
>

The refactoring is about merging memory and persistence configuration under
the same umbrella. The term "MemoryPolicy" does not make sense anymore,
given that it now also includes persistent configuration as well.


>
> вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan <ds...@apache.org>:
>
> > Dmitriy, we are not renaming classes, we are refactoring them. Prior to
> > this design, it was impossible to set persistence configuration on
> > per-cache basis. With this new design, users will be able to configure
> some
> > caches to be in-memory only and others to be on disk.
> >
> > Given that we are already refactoring, it only makes sense to pick
> better,
> > more appropriate names.
> >
> > D.
> >
> > On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dp...@gmail.com>
> > wrote:
> >
> > > Vladimir, it is not clear for me, why we need to rename existing
> > > configuration classes. Could you explain? And if we can't get consensus
> > > now, should we pospond solution?
> > >
> > > My idea is that user needs this feature more than elegant names in
> > > configuration.
> > >
> > > Moreover once MemoryPolicyConfiguration was introduced as Ignite term
> it
> > is
> > > simpler to keep it as is, than create new terms.
> > >
> > > Sincerely,
> > > Dmitriy Pavlov
> > >
> > > вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vo...@gridgain.com>:
> > >
> > > > I do not understand why we should delay with renames. Yes, it will
> > cause
> > > > questions, so we will have to put additional efforts to docs and
> > > JavaDocs.
> > > > But the earlier we do that, the better.
> > > >
> > > > On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <
> dpavlov.spb@gmail.com>
> > > > wrote:
> > > >
> > > > > Hi Igniters, sorry for late response. I didn't catch idea of
> > renaming.
> > > > > PersistentStoreConfiguration is intuitive, and
> > > MemoryPolicyConfiguration
> > > > is
> > > > > intuitive also.
> > > > >
> > > > > If we rename these classes now, it will bring more questions to
> user
> > > > list.
> > > > > Users may be confused by old and new names and by trying to match
> it.
> > > > More
> > > > > issues can came from XML configs that users already have.
> > > > >
> > > > > Can we postpone the renaming? I suggest to finish 'persistence per
> > > memory
> > > > > policy' task without renaming, and create separate JIRA issue for
> > > > creating
> > > > > future decision?
> > > > >
> > > > > вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com
> > > > > >:
> > > > >
> > > > > > I do not like DurableMemoryConfiguration, because it's quite
> > > confusing
> > > > -
> > > > > we
> > > > > > configure in-memory caches using DurableMemory class, which
> > > immediately
> > > > > > suggests that everything will be persisted. I am not sure if this
> > is
> > > a
> > > > > > right wording choice for the documentation either. I would go
> with
> > > > > > DataStoreConfiguration and DataRegionConfiguration.
> > > > > >
> > > > > > --AG
> > > > > >
> > > > > > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >:
> > > > > >
> > > > > > > Given that we already have a notion of CacheStore which comes
> > from
> > > > > JCache
> > > > > > > spec, I think having other stores may get confusing. I like
> > > > > > > DurableMemoryConfiguration.
> > > > > > >
> > > > > > > Other opinions?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> > > > > vozerov@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Dima, let's finalize the design first.
> > > > > > > >
> > > > > > > > As I understand, we are happy with idea to merge
> > > > MemoryConfiguration
> > > > > > > > and PersistentStoreConfiguration
> > > > > > > > into something what I called DataConfiguration, and to rename
> > > > > > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > > > > > >
> > > > > > > > The only outstanding qurestion is whether DataConfiguration
> is
> > a
> > > > good
> > > > > > > name.
> > > > > > > > I am not very happy with it, so let's think of other
> > > alternatives.
> > > > > > Quick
> > > > > > > > ideas:
> > > > > > > > 1) StoreConfiguration - looks perfect to me - short and
> > > > > > self-describing,
> > > > > > > > but clashes a bit with existing CacheStore
> > > > > > > > 2) DataStoreConfiguration - same as p.1, but the word "data"
> is
> > > not
> > > > > > > > necessary IMO
> > > > > > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> > > > > architecture.
> > > > > > > > 4) DurableMemoryConfiguration - aligns well with our docs,
> but
> > I
> > > do
> > > > > not
> > > > > > > > like it - too long and misleading
> > > > > > > >
> > > > > > > > Any other ideas?
> > > > > > > >
> > > > > > > > I would prefer to have either [StoreConfiguration +
> > > > > > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > > > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Vladimir,
> > > > > > > > >
> > > > > > > > > Can you please add the configuration example in the ticket?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > > Guys,
> > > > > > > > > >
> > > > > > > > > > I suggest we finalize the configuration changes in the
> > > original
> > > > > > > ticket
> > > > > > > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030
> > and
> > > > > > proceed
> > > > > > > > with
> > > > > > > > > > the changes.
> > > > > > > > > >
> > > > > > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org
> > > > > > > >:
> > > > > > > > > >
> > > > > > > > > > > Can we specify what metrics will look like? I think we
> > > should
> > > > > not
> > > > > > > > just
> > > > > > > > > > > blindly merge them.
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Denis,
> > > > > > > > > > > >
> > > > > > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > > > > > >
> > > > > > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > > > > > dmagda@apache.org>
> > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > If we’re taking the consolidation path for Memory
> and
> > > > > > > Persistence
> > > > > > > > > > > > > configurations then it makes sense to merge
> > > MemoryMetrics
> > > > > [1]
> > > > > > > and
> > > > > > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Agree?
> > > > > > > > > > > > >
> > > > > > > > > > > > > [1]
> > > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > > > apache/ignite/MemoryMetrics.html <
> > > > > https://ignite.apache.org/
> > > > > > > > > > > > > releases/latest/javadoc/org/
> > > apache/ignite/MemoryMetrics.
> > > > > html>
> > > > > > > > > > > > > [2]
> > > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > apache/ignite/
> > > > > > > > > > > > > PersistenceMetrics.html
> > > > > > > > > > > > >
> > > > > > > > > > > > > —
> > > > > > > > > > > > > Denis
> > > > > > > > > > > > >
> > > > > > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > > > >
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > I like it.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > D.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir
> Ozerov <
> > > > > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >> Guys,
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Here is my proposal:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > > > > > *DataRegionConfiguration*.
> > > > > > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > > > > > MemoryConfiguration
> > > > > > > > > and
> > > > > > > > > > > > > renamed
> > > > > > > > > > > > > >> to ... *DataStorageConfiguration*! It has:
> common
> > > > memory
> > > > > > > > > settings
> > > > > > > > > > > > (e.g.
> > > > > > > > > > > > > >> default data region), persistence settings (e.g.
> > > WAL)
> > > > > and
> > > > > > a
> > > > > > > > list
> > > > > > > > > > of
> > > > > > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> What we have in the end:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > > > > > > >>        <property name="persistentStorePath"
> > > > > > value="/my/path"
> > > > > > > > />
> > > > > > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > > > > > >>            <list>
> > > > > > > > > > > > > >>                <bean
> > > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > > >>                    <property name="name"
> > > > > value="VOLATILE"
> > > > > > />
> > > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > > value="1_000_000_000"
> > > > > > > > > > />
> > > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > > >>                <bean
> > > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > > >>                    <property name="name"
> > > > > > value="PERSISTENT"
> > > > > > > />
> > > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > > value="1_000_000_000"
> > > > > > > > > > />
> > > > > > > > > > > > > >>                    <property name="persistent"
> > > > > > value="true"
> > > > > > > />
> > > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > > >>            </list>
> > > > > > > > > > > > > >>        </property>
> > > > > > > > > > > > > >>    </bean>
> > > > > > > > > > > > > >> </property>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Makes sense?
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> Vladimir.
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy
> > Setrakyan <
> > > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > > >>
> > > > > > > > > > > > > >>> Firstly all, why not call it DataPolicy instead
> > of
> > > > > > > > > MemoryPolicy?
> > > > > > > > > > > > > >> Secondly,
> > > > > > > > > > > > > >>> why not set data policies directly on
> > > > > > IgniteConfiguration.
> > > > > > > > And
> > > > > > > > > > > > lastly,
> > > > > > > > > > > > > >> how
> > > > > > > > > > > > > >>> about we combine memory and disk properties in
> > one
> > > > bean
> > > > > > > with
> > > > > > > > > > clear
> > > > > > > > > > > > > naming
> > > > > > > > > > > > > >>> convention?
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> Here is the example. Note that all properties
> > above
> > > > > must
> > > > > > > > start
> > > > > > > > > > with
> > > > > > > > > > > > > with
> > > > > > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> *IgniteConfiguration cfg = new
> > > > IgniteConfiguration();*
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> *cfg.setDataPolicies(    new
> > > > DataPolicyConfiguration()
> > > > > > > > > > > > > >>>> .setName("bla"),
> .setMemoryMaxSize(1024),
> > > //
> > > > > must
> > > > > > > be
> > > > > > > > > > > greater
> > > > > > > > > > > > > >> than
> > > > > > > > > > > > > >>> 0,
> > > > > > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > > > > > .setDiskMaxSize(0),
> > > > > > > > > > > > //
> > > > > > > > > > > > > >> if
> > > > > > > > > > > > > >>>> greater than 0, then persistence is enabled.
> > > );*
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> I think this approach is much more concise and
> > > > straight
> > > > > > > > > forward.
> > > > > > > > > > > What
> > > > > > > > > > > > > do
> > > > > > > > > > > > > >>> you think?
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> D.
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir
> Ozerov
> > <
> > > > > > > > > > > > vozerov@gridgain.com
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>>> I prefer the second. Composition over
> > inheritance
> > > -
> > > > > this
> > > > > > > is
> > > > > > > > > how
> > > > > > > > > > > all
> > > > > > > > > > > > > our
> > > > > > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > > > > > setCacheStoreFactory".
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey
> > Goncharuk
> > > <
> > > > > > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>>> Reiterating this based on some feedback from
> > PDS
> > > > > users.
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> It might be confusing to configure
> persistence
> > > with
> > > > > > > > > > > "MemoryPolicy",
> > > > > > > > > > > > > >> so
> > > > > > > > > > > > > >>>>> another approach is to deprecate the old
> names
> > > and
> > > > > > > > introduce
> > > > > > > > > a
> > > > > > > > > > > new
> > > > > > > > > > > > > >> name
> > > > > > > > > > > > > >>>>> "DataRegion" because it reflects the actual
> > state
> > > > > when
> > > > > > > data
> > > > > > > > > is
> > > > > > > > > > > > stored
> > > > > > > > > > > > > >>> on
> > > > > > > > > > > > > >>>>> disk and partially in memory. I have two
> > options
> > > in
> > > > > > mind,
> > > > > > > > > each
> > > > > > > > > > of
> > > > > > > > > > > > > >> them
> > > > > > > > > > > > > >>>>> looks acceptable to me, so I would like to
> have
> > > > some
> > > > > > > > feedback
> > > > > > > > > > > from
> > > > > > > > > > > > > >> the
> > > > > > > > > > > > > >>>>> community. Old configuration names will be
> > > > deprecated
> > > > > > > (but
> > > > > > > > > > still
> > > > > > > > > > > be
> > > > > > > > > > > > > >>> taken
> > > > > > > > > > > > > >>>>> if used for backward compatibility). Note,
> that
> > > old
> > > > > > names
> > > > > > > > > > > > deprecation
> > > > > > > > > > > > > >>>>> handles default configuration compatibility
> > very
> > > > > > nicely -
> > > > > > > > > > current
> > > > > > > > > > > > PDS
> > > > > > > > > > > > > >>>> users
> > > > > > > > > > > > > >>>>> will not need to change anything to keep
> > > everything
> > > > > > > > working.
> > > > > > > > > > The
> > > > > > > > > > > > two
> > > > > > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> * we have two separate classes for in-memory
> > and
> > > > > > > persisted
> > > > > > > > > data
> > > > > > > > > > > > > >>> regions,
> > > > > > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > > IgniteConfiguration();
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > > >>> );
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> * we have one class for data region
> > > configuration,
> > > > > but
> > > > > > it
> > > > > > > > > will
> > > > > > > > > > > > have a
> > > > > > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > > IgniteConfiguration();
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > > > > > >>>>>                new
> > DataRegionPersistenceConfigura
> > > > > tion()
> > > > > > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > > >>> );
> > > > > > > > > > > > > >>>>>
> > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > >>>
> > > > > > > > > > > > > >>
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitry Pavlov <dp...@gmail.com>]

Hi Dmitriy, thank you for reply. Do you agree Memory Policy already became
Ignite's term? We call this configuration now MemoryPolicy(Configuration),
can we call new configuration elments by their existings name? We can avoid
introduction of second Ignite's term in that case.

вт, 26 сент. 2017 г. в 17:27, Dmitriy Setrakyan <ds...@apache.org>:

> Dmitriy, we are not renaming classes, we are refactoring them. Prior to
> this design, it was impossible to set persistence configuration on
> per-cache basis. With this new design, users will be able to configure some
> caches to be in-memory only and others to be on disk.
>
> Given that we are already refactoring, it only makes sense to pick better,
> more appropriate names.
>
> D.
>
> On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dp...@gmail.com>
> wrote:
>
> > Vladimir, it is not clear for me, why we need to rename existing
> > configuration classes. Could you explain? And if we can't get consensus
> > now, should we pospond solution?
> >
> > My idea is that user needs this feature more than elegant names in
> > configuration.
> >
> > Moreover once MemoryPolicyConfiguration was introduced as Ignite term it
> is
> > simpler to keep it as is, than create new terms.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vo...@gridgain.com>:
> >
> > > I do not understand why we should delay with renames. Yes, it will
> cause
> > > questions, so we will have to put additional efforts to docs and
> > JavaDocs.
> > > But the earlier we do that, the better.
> > >
> > > On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <dp...@gmail.com>
> > > wrote:
> > >
> > > > Hi Igniters, sorry for late response. I didn't catch idea of
> renaming.
> > > > PersistentStoreConfiguration is intuitive, and
> > MemoryPolicyConfiguration
> > > is
> > > > intuitive also.
> > > >
> > > > If we rename these classes now, it will bring more questions to user
> > > list.
> > > > Users may be confused by old and new names and by trying to match it.
> > > More
> > > > issues can came from XML configs that users already have.
> > > >
> > > > Can we postpone the renaming? I suggest to finish 'persistence per
> > memory
> > > > policy' task without renaming, and create separate JIRA issue for
> > > creating
> > > > future decision?
> > > >
> > > > вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >:
> > > >
> > > > > I do not like DurableMemoryConfiguration, because it's quite
> > confusing
> > > -
> > > > we
> > > > > configure in-memory caches using DurableMemory class, which
> > immediately
> > > > > suggests that everything will be persisted. I am not sure if this
> is
> > a
> > > > > right wording choice for the documentation either. I would go with
> > > > > DataStoreConfiguration and DataRegionConfiguration.
> > > > >
> > > > > --AG
> > > > >
> > > > > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> > > > >
> > > > > > Given that we already have a notion of CacheStore which comes
> from
> > > > JCache
> > > > > > spec, I think having other stores may get confusing. I like
> > > > > > DurableMemoryConfiguration.
> > > > > >
> > > > > > Other opinions?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> > > > vozerov@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Dima, let's finalize the design first.
> > > > > > >
> > > > > > > As I understand, we are happy with idea to merge
> > > MemoryConfiguration
> > > > > > > and PersistentStoreConfiguration
> > > > > > > into something what I called DataConfiguration, and to rename
> > > > > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > > > > >
> > > > > > > The only outstanding qurestion is whether DataConfiguration is
> a
> > > good
> > > > > > name.
> > > > > > > I am not very happy with it, so let's think of other
> > alternatives.
> > > > > Quick
> > > > > > > ideas:
> > > > > > > 1) StoreConfiguration - looks perfect to me - short and
> > > > > self-describing,
> > > > > > > but clashes a bit with existing CacheStore
> > > > > > > 2) DataStoreConfiguration - same as p.1, but the word "data" is
> > not
> > > > > > > necessary IMO
> > > > > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> > > > architecture.
> > > > > > > 4) DurableMemoryConfiguration - aligns well with our docs, but
> I
> > do
> > > > not
> > > > > > > like it - too long and misleading
> > > > > > >
> > > > > > > Any other ideas?
> > > > > > >
> > > > > > > I would prefer to have either [StoreConfiguration +
> > > > > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > > > > >
> > > > > > > Vladimir.
> > > > > > >
> > > > > > >
> > > > > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Vladimir,
> > > > > > > >
> > > > > > > > Can you please add the configuration example in the ticket?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > > >
> > > > > > > > > Guys,
> > > > > > > > >
> > > > > > > > > I suggest we finalize the configuration changes in the
> > original
> > > > > > ticket
> > > > > > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030
> and
> > > > > proceed
> > > > > > > with
> > > > > > > > > the changes.
> > > > > > > > >
> > > > > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org
> > > > > > >:
> > > > > > > > >
> > > > > > > > > > Can we specify what metrics will look like? I think we
> > should
> > > > not
> > > > > > > just
> > > > > > > > > > blindly merge them.
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Denis,
> > > > > > > > > > >
> > > > > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > > > > >
> > > > > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > > > > dmagda@apache.org>
> > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > If we’re taking the consolidation path for Memory and
> > > > > > Persistence
> > > > > > > > > > > > configurations then it makes sense to merge
> > MemoryMetrics
> > > > [1]
> > > > > > and
> > > > > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > > > > >
> > > > > > > > > > > > Agree?
> > > > > > > > > > > >
> > > > > > > > > > > > [1]
> > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > > apache/ignite/MemoryMetrics.html <
> > > > https://ignite.apache.org/
> > > > > > > > > > > > releases/latest/javadoc/org/
> > apache/ignite/MemoryMetrics.
> > > > html>
> > > > > > > > > > > > [2]
> > > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > apache/ignite/
> > > > > > > > > > > > PersistenceMetrics.html
> > > > > > > > > > > >
> > > > > > > > > > > > —
> > > > > > > > > > > > Denis
> > > > > > > > > > > >
> > > > > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > > >
> > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > > I like it.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > > > > >
> > > > > > > > > > > > > D.
> > > > > > > > > > > > >
> > > > > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > > > wrote:
> > > > > > > > > > > > >
> > > > > > > > > > > > >> Guys,
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Here is my proposal:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > > > > *DataRegionConfiguration*.
> > > > > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > > > > MemoryConfiguration
> > > > > > > > and
> > > > > > > > > > > > renamed
> > > > > > > > > > > > >> to ... *DataStorageConfiguration*! It has: common
> > > memory
> > > > > > > > settings
> > > > > > > > > > > (e.g.
> > > > > > > > > > > > >> default data region), persistence settings (e.g.
> > WAL)
> > > > and
> > > > > a
> > > > > > > list
> > > > > > > > > of
> > > > > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> What we have in the end:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > > > > > >>        <property name="persistentStorePath"
> > > > > value="/my/path"
> > > > > > > />
> > > > > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > > > > >>            <list>
> > > > > > > > > > > > >>                <bean
> > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > >>                    <property name="name"
> > > > value="VOLATILE"
> > > > > />
> > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > value="1_000_000_000"
> > > > > > > > > />
> > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > >>                <bean
> > > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > > >>                    <property name="name"
> > > > > value="PERSISTENT"
> > > > > > />
> > > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > > value="1_000_000_000"
> > > > > > > > > />
> > > > > > > > > > > > >>                    <property name="persistent"
> > > > > value="true"
> > > > > > />
> > > > > > > > > > > > >>                </bean>
> > > > > > > > > > > > >>            </list>
> > > > > > > > > > > > >>        </property>
> > > > > > > > > > > > >>    </bean>
> > > > > > > > > > > > >> </property>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Makes sense?
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> Vladimir.
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>
> > > > > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy
> Setrakyan <
> > > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > > >> wrote:
> > > > > > > > > > > > >>
> > > > > > > > > > > > >>> Firstly all, why not call it DataPolicy instead
> of
> > > > > > > > MemoryPolicy?
> > > > > > > > > > > > >> Secondly,
> > > > > > > > > > > > >>> why not set data policies directly on
> > > > > IgniteConfiguration.
> > > > > > > And
> > > > > > > > > > > lastly,
> > > > > > > > > > > > >> how
> > > > > > > > > > > > >>> about we combine memory and disk properties in
> one
> > > bean
> > > > > > with
> > > > > > > > > clear
> > > > > > > > > > > > naming
> > > > > > > > > > > > >>> convention?
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> Here is the example. Note that all properties
> above
> > > > must
> > > > > > > start
> > > > > > > > > with
> > > > > > > > > > > > with
> > > > > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> *IgniteConfiguration cfg = new
> > > IgniteConfiguration();*
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> *cfg.setDataPolicies(    new
> > > DataPolicyConfiguration()
> > > > > > > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024),
> > //
> > > > must
> > > > > > be
> > > > > > > > > > greater
> > > > > > > > > > > > >> than
> > > > > > > > > > > > >>> 0,
> > > > > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > > > > .setDiskMaxSize(0),
> > > > > > > > > > > //
> > > > > > > > > > > > >> if
> > > > > > > > > > > > >>>> greater than 0, then persistence is enabled.
> > );*
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> I think this approach is much more concise and
> > > straight
> > > > > > > > forward.
> > > > > > > > > > What
> > > > > > > > > > > > do
> > > > > > > > > > > > >>> you think?
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> D.
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov
> <
> > > > > > > > > > > vozerov@gridgain.com
> > > > > > > > > > > > >
> > > > > > > > > > > > >>> wrote:
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>>> I prefer the second. Composition over
> inheritance
> > -
> > > > this
> > > > > > is
> > > > > > > > how
> > > > > > > > > > all
> > > > > > > > > > > > our
> > > > > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > > > > setCacheStoreFactory".
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey
> Goncharuk
> > <
> > > > > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>>> Reiterating this based on some feedback from
> PDS
> > > > users.
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> It might be confusing to configure persistence
> > with
> > > > > > > > > > "MemoryPolicy",
> > > > > > > > > > > > >> so
> > > > > > > > > > > > >>>>> another approach is to deprecate the old names
> > and
> > > > > > > introduce
> > > > > > > > a
> > > > > > > > > > new
> > > > > > > > > > > > >> name
> > > > > > > > > > > > >>>>> "DataRegion" because it reflects the actual
> state
> > > > when
> > > > > > data
> > > > > > > > is
> > > > > > > > > > > stored
> > > > > > > > > > > > >>> on
> > > > > > > > > > > > >>>>> disk and partially in memory. I have two
> options
> > in
> > > > > mind,
> > > > > > > > each
> > > > > > > > > of
> > > > > > > > > > > > >> them
> > > > > > > > > > > > >>>>> looks acceptable to me, so I would like to have
> > > some
> > > > > > > feedback
> > > > > > > > > > from
> > > > > > > > > > > > >> the
> > > > > > > > > > > > >>>>> community. Old configuration names will be
> > > deprecated
> > > > > > (but
> > > > > > > > > still
> > > > > > > > > > be
> > > > > > > > > > > > >>> taken
> > > > > > > > > > > > >>>>> if used for backward compatibility). Note, that
> > old
> > > > > names
> > > > > > > > > > > deprecation
> > > > > > > > > > > > >>>>> handles default configuration compatibility
> very
> > > > > nicely -
> > > > > > > > > current
> > > > > > > > > > > PDS
> > > > > > > > > > > > >>>> users
> > > > > > > > > > > > >>>>> will not need to change anything to keep
> > everything
> > > > > > > working.
> > > > > > > > > The
> > > > > > > > > > > two
> > > > > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> * we have two separate classes for in-memory
> and
> > > > > > persisted
> > > > > > > > data
> > > > > > > > > > > > >>> regions,
> > > > > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > IgniteConfiguration();
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > >>> );
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> * we have one class for data region
> > configuration,
> > > > but
> > > > > it
> > > > > > > > will
> > > > > > > > > > > have a
> > > > > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > > IgniteConfiguration();
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > > > > >>>>>                new
> DataRegionPersistenceConfigura
> > > > tion()
> > > > > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > > >>> );
> > > > > > > > > > > > >>>>>
> > > > > > > > > > > > >>>>
> > > > > > > > > > > > >>>
> > > > > > > > > > > > >>
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Dmitriy, we are not renaming classes, we are refactoring them. Prior to
this design, it was impossible to set persistence configuration on
per-cache basis. With this new design, users will be able to configure some
caches to be in-memory only and others to be on disk.

Given that we are already refactoring, it only makes sense to pick better,
more appropriate names.

D.

On Tue, Sep 26, 2017 at 7:20 AM, Dmitry Pavlov <dp...@gmail.com>
wrote:

> Vladimir, it is not clear for me, why we need to rename existing
> configuration classes. Could you explain? And if we can't get consensus
> now, should we pospond solution?
>
> My idea is that user needs this feature more than elegant names in
> configuration.
>
> Moreover once MemoryPolicyConfiguration was introduced as Ignite term it is
> simpler to keep it as is, than create new terms.
>
> Sincerely,
> Dmitriy Pavlov
>
> вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vo...@gridgain.com>:
>
> > I do not understand why we should delay with renames. Yes, it will cause
> > questions, so we will have to put additional efforts to docs and
> JavaDocs.
> > But the earlier we do that, the better.
> >
> > On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <dp...@gmail.com>
> > wrote:
> >
> > > Hi Igniters, sorry for late response. I didn't catch idea of renaming.
> > > PersistentStoreConfiguration is intuitive, and
> MemoryPolicyConfiguration
> > is
> > > intuitive also.
> > >
> > > If we rename these classes now, it will bring more questions to user
> > list.
> > > Users may be confused by old and new names and by trying to match it.
> > More
> > > issues can came from XML configs that users already have.
> > >
> > > Can we postpone the renaming? I suggest to finish 'persistence per
> memory
> > > policy' task without renaming, and create separate JIRA issue for
> > creating
> > > future decision?
> > >
> > > вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >:
> > >
> > > > I do not like DurableMemoryConfiguration, because it's quite
> confusing
> > -
> > > we
> > > > configure in-memory caches using DurableMemory class, which
> immediately
> > > > suggests that everything will be persisted. I am not sure if this is
> a
> > > > right wording choice for the documentation either. I would go with
> > > > DataStoreConfiguration and DataRegionConfiguration.
> > > >
> > > > --AG
> > > >
> > > > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> > > >
> > > > > Given that we already have a notion of CacheStore which comes from
> > > JCache
> > > > > spec, I think having other stores may get confusing. I like
> > > > > DurableMemoryConfiguration.
> > > > >
> > > > > Other opinions?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> > > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Dima, let's finalize the design first.
> > > > > >
> > > > > > As I understand, we are happy with idea to merge
> > MemoryConfiguration
> > > > > > and PersistentStoreConfiguration
> > > > > > into something what I called DataConfiguration, and to rename
> > > > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > > > >
> > > > > > The only outstanding qurestion is whether DataConfiguration is a
> > good
> > > > > name.
> > > > > > I am not very happy with it, so let's think of other
> alternatives.
> > > > Quick
> > > > > > ideas:
> > > > > > 1) StoreConfiguration - looks perfect to me - short and
> > > > self-describing,
> > > > > > but clashes a bit with existing CacheStore
> > > > > > 2) DataStoreConfiguration - same as p.1, but the word "data" is
> not
> > > > > > necessary IMO
> > > > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> > > architecture.
> > > > > > 4) DurableMemoryConfiguration - aligns well with our docs, but I
> do
> > > not
> > > > > > like it - too long and misleading
> > > > > >
> > > > > > Any other ideas?
> > > > > >
> > > > > > I would prefer to have either [StoreConfiguration +
> > > > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > > > >
> > > > > > Vladimir.
> > > > > >
> > > > > >
> > > > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > Can you please add the configuration example in the ticket?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > > >
> > > > > > > > Guys,
> > > > > > > >
> > > > > > > > I suggest we finalize the configuration changes in the
> original
> > > > > ticket
> > > > > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and
> > > > proceed
> > > > > > with
> > > > > > > > the changes.
> > > > > > > >
> > > > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >:
> > > > > > > >
> > > > > > > > > Can we specify what metrics will look like? I think we
> should
> > > not
> > > > > > just
> > > > > > > > > blindly merge them.
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > > > vozerov@gridgain.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Denis,
> > > > > > > > > >
> > > > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > > > >
> > > > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > > > dmagda@apache.org>
> > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > If we’re taking the consolidation path for Memory and
> > > > > Persistence
> > > > > > > > > > > configurations then it makes sense to merge
> MemoryMetrics
> > > [1]
> > > > > and
> > > > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > > > >
> > > > > > > > > > > Agree?
> > > > > > > > > > >
> > > > > > > > > > > [1]
> > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > > apache/ignite/MemoryMetrics.html <
> > > https://ignite.apache.org/
> > > > > > > > > > > releases/latest/javadoc/org/
> apache/ignite/MemoryMetrics.
> > > html>
> > > > > > > > > > > [2]
> > https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > apache/ignite/
> > > > > > > > > > > PersistenceMetrics.html
> > > > > > > > > > >
> > > > > > > > > > > —
> > > > > > > > > > > Denis
> > > > > > > > > > >
> > > > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > >
> > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > I like it.
> > > > > > > > > > > >
> > > > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > > > >
> > > > > > > > > > > > D.
> > > > > > > > > > > >
> > > > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > > wrote:
> > > > > > > > > > > >
> > > > > > > > > > > >> Guys,
> > > > > > > > > > > >>
> > > > > > > > > > > >> Here is my proposal:
> > > > > > > > > > > >>
> > > > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > > > *DataRegionConfiguration*.
> > > > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > > > MemoryConfiguration
> > > > > > > and
> > > > > > > > > > > renamed
> > > > > > > > > > > >> to ... *DataStorageConfiguration*! It has: common
> > memory
> > > > > > > settings
> > > > > > > > > > (e.g.
> > > > > > > > > > > >> default data region), persistence settings (e.g.
> WAL)
> > > and
> > > > a
> > > > > > list
> > > > > > > > of
> > > > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > > > >>
> > > > > > > > > > > >> What we have in the end:
> > > > > > > > > > > >>
> > > > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > > > > >>        <property name="persistentStorePath"
> > > > value="/my/path"
> > > > > > />
> > > > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > > > >>            <list>
> > > > > > > > > > > >>                <bean
> > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > >>                    <property name="name"
> > > value="VOLATILE"
> > > > />
> > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > value="1_000_000_000"
> > > > > > > > />
> > > > > > > > > > > >>                </bean>
> > > > > > > > > > > >>                <bean
> > > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > > >>                    <property name="name"
> > > > value="PERSISTENT"
> > > > > />
> > > > > > > > > > > >>                    <property name="maxSize"
> > > > > > > value="1_000_000_000"
> > > > > > > > />
> > > > > > > > > > > >>                    <property name="persistent"
> > > > value="true"
> > > > > />
> > > > > > > > > > > >>                </bean>
> > > > > > > > > > > >>            </list>
> > > > > > > > > > > >>        </property>
> > > > > > > > > > > >>    </bean>
> > > > > > > > > > > >> </property>
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> Makes sense?
> > > > > > > > > > > >>
> > > > > > > > > > > >> Vladimir.
> > > > > > > > > > > >>
> > > > > > > > > > > >>
> > > > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > > >> wrote:
> > > > > > > > > > > >>
> > > > > > > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > > > > > > MemoryPolicy?
> > > > > > > > > > > >> Secondly,
> > > > > > > > > > > >>> why not set data policies directly on
> > > > IgniteConfiguration.
> > > > > > And
> > > > > > > > > > lastly,
> > > > > > > > > > > >> how
> > > > > > > > > > > >>> about we combine memory and disk properties in one
> > bean
> > > > > with
> > > > > > > > clear
> > > > > > > > > > > naming
> > > > > > > > > > > >>> convention?
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> Here is the example. Note that all properties above
> > > must
> > > > > > start
> > > > > > > > with
> > > > > > > > > > > with
> > > > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> *IgniteConfiguration cfg = new
> > IgniteConfiguration();*
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> *cfg.setDataPolicies(    new
> > DataPolicyConfiguration()
> > > > > > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024),
> //
> > > must
> > > > > be
> > > > > > > > > greater
> > > > > > > > > > > >> than
> > > > > > > > > > > >>> 0,
> > > > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > > > .setDiskMaxSize(0),
> > > > > > > > > > //
> > > > > > > > > > > >> if
> > > > > > > > > > > >>>> greater than 0, then persistence is enabled.
> );*
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> I think this approach is much more concise and
> > straight
> > > > > > > forward.
> > > > > > > > > What
> > > > > > > > > > > do
> > > > > > > > > > > >>> you think?
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> D.
> > > > > > > > > > > >>>
> > > > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > > > > > > vozerov@gridgain.com
> > > > > > > > > > > >
> > > > > > > > > > > >>> wrote:
> > > > > > > > > > > >>>
> > > > > > > > > > > >>>> I prefer the second. Composition over inheritance
> -
> > > this
> > > > > is
> > > > > > > how
> > > > > > > > > all
> > > > > > > > > > > our
> > > > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > > > setCacheStoreFactory".
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk
> <
> > > > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>>> Reiterating this based on some feedback from PDS
> > > users.
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> It might be confusing to configure persistence
> with
> > > > > > > > > "MemoryPolicy",
> > > > > > > > > > > >> so
> > > > > > > > > > > >>>>> another approach is to deprecate the old names
> and
> > > > > > introduce
> > > > > > > a
> > > > > > > > > new
> > > > > > > > > > > >> name
> > > > > > > > > > > >>>>> "DataRegion" because it reflects the actual state
> > > when
> > > > > data
> > > > > > > is
> > > > > > > > > > stored
> > > > > > > > > > > >>> on
> > > > > > > > > > > >>>>> disk and partially in memory. I have two options
> in
> > > > mind,
> > > > > > > each
> > > > > > > > of
> > > > > > > > > > > >> them
> > > > > > > > > > > >>>>> looks acceptable to me, so I would like to have
> > some
> > > > > > feedback
> > > > > > > > > from
> > > > > > > > > > > >> the
> > > > > > > > > > > >>>>> community. Old configuration names will be
> > deprecated
> > > > > (but
> > > > > > > > still
> > > > > > > > > be
> > > > > > > > > > > >>> taken
> > > > > > > > > > > >>>>> if used for backward compatibility). Note, that
> old
> > > > names
> > > > > > > > > > deprecation
> > > > > > > > > > > >>>>> handles default configuration compatibility very
> > > > nicely -
> > > > > > > > current
> > > > > > > > > > PDS
> > > > > > > > > > > >>>> users
> > > > > > > > > > > >>>>> will not need to change anything to keep
> everything
> > > > > > working.
> > > > > > > > The
> > > > > > > > > > two
> > > > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> * we have two separate classes for in-memory and
> > > > > persisted
> > > > > > > data
> > > > > > > > > > > >>> regions,
> > > > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > IgniteConfiguration();
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > >>> );
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> * we have one class for data region
> configuration,
> > > but
> > > > it
> > > > > > > will
> > > > > > > > > > have a
> > > > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> > IgniteConfiguration();
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > > DataRegionsConfiguration()
> > > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > > > >>>>>                new DataRegionPersistenceConfigura
> > > tion()
> > > > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > > >>> );
> > > > > > > > > > > >>>>>
> > > > > > > > > > > >>>>
> > > > > > > > > > > >>>
> > > > > > > > > > > >>
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitry Pavlov <dp...@gmail.com>]

Vladimir, it is not clear for me, why we need to rename existing
configuration classes. Could you explain? And if we can't get consensus
now, should we pospond solution?

My idea is that user needs this feature more than elegant names in
configuration.

Moreover once MemoryPolicyConfiguration was introduced as Ignite term it is
simpler to keep it as is, than create new terms.

Sincerely,
Dmitriy Pavlov

вт, 26 сент. 2017 г. в 16:59, Vladimir Ozerov <vo...@gridgain.com>:

> I do not understand why we should delay with renames. Yes, it will cause
> questions, so we will have to put additional efforts to docs and JavaDocs.
> But the earlier we do that, the better.
>
> On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <dp...@gmail.com>
> wrote:
>
> > Hi Igniters, sorry for late response. I didn't catch idea of renaming.
> > PersistentStoreConfiguration is intuitive, and MemoryPolicyConfiguration
> is
> > intuitive also.
> >
> > If we rename these classes now, it will bring more questions to user
> list.
> > Users may be confused by old and new names and by trying to match it.
> More
> > issues can came from XML configs that users already have.
> >
> > Can we postpone the renaming? I suggest to finish 'persistence per memory
> > policy' task without renaming, and create separate JIRA issue for
> creating
> > future decision?
> >
> > вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >:
> >
> > > I do not like DurableMemoryConfiguration, because it's quite confusing
> -
> > we
> > > configure in-memory caches using DurableMemory class, which immediately
> > > suggests that everything will be persisted. I am not sure if this is a
> > > right wording choice for the documentation either. I would go with
> > > DataStoreConfiguration and DataRegionConfiguration.
> > >
> > > --AG
> > >
> > > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> > >
> > > > Given that we already have a notion of CacheStore which comes from
> > JCache
> > > > spec, I think having other stores may get confusing. I like
> > > > DurableMemoryConfiguration.
> > > >
> > > > Other opinions?
> > > >
> > > > D.
> > > >
> > > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > Dima, let's finalize the design first.
> > > > >
> > > > > As I understand, we are happy with idea to merge
> MemoryConfiguration
> > > > > and PersistentStoreConfiguration
> > > > > into something what I called DataConfiguration, and to rename
> > > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > > >
> > > > > The only outstanding qurestion is whether DataConfiguration is a
> good
> > > > name.
> > > > > I am not very happy with it, so let's think of other alternatives.
> > > Quick
> > > > > ideas:
> > > > > 1) StoreConfiguration - looks perfect to me - short and
> > > self-describing,
> > > > > but clashes a bit with existing CacheStore
> > > > > 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> > > > > necessary IMO
> > > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> > architecture.
> > > > > 4) DurableMemoryConfiguration - aligns well with our docs, but I do
> > not
> > > > > like it - too long and misleading
> > > > >
> > > > > Any other ideas?
> > > > >
> > > > > I would prefer to have either [StoreConfiguration +
> > > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > > >
> > > > > Vladimir.
> > > > >
> > > > >
> > > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org>
> > > > > wrote:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > Can you please add the configuration example in the ticket?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > >
> > > > > > > Guys,
> > > > > > >
> > > > > > > I suggest we finalize the configuration changes in the original
> > > > ticket
> > > > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and
> > > proceed
> > > > > with
> > > > > > > the changes.
> > > > > > >
> > > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > > dsetrakyan@apache.org
> > > > >:
> > > > > > >
> > > > > > > > Can we specify what metrics will look like? I think we should
> > not
> > > > > just
> > > > > > > > blindly merge them.
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > > vozerov@gridgain.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Denis,
> > > > > > > > >
> > > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > > >
> > > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > > dmagda@apache.org>
> > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > If we’re taking the consolidation path for Memory and
> > > > Persistence
> > > > > > > > > > configurations then it makes sense to merge MemoryMetrics
> > [1]
> > > > and
> > > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > > >
> > > > > > > > > > Agree?
> > > > > > > > > >
> > > > > > > > > > [1]
> https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > > apache/ignite/MemoryMetrics.html <
> > https://ignite.apache.org/
> > > > > > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.
> > html>
> > > > > > > > > > [2]
> https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > apache/ignite/
> > > > > > > > > > PersistenceMetrics.html
> > > > > > > > > >
> > > > > > > > > > —
> > > > > > > > > > Denis
> > > > > > > > > >
> > > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > > > dsetrakyan@apache.org
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > I like it.
> > > > > > > > > > >
> > > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > > >
> > > > > > > > > > > D.
> > > > > > > > > > >
> > > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > >> Guys,
> > > > > > > > > > >>
> > > > > > > > > > >> Here is my proposal:
> > > > > > > > > > >>
> > > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > > *DataRegionConfiguration*.
> > > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > > MemoryConfiguration
> > > > > > and
> > > > > > > > > > renamed
> > > > > > > > > > >> to ... *DataStorageConfiguration*! It has: common
> memory
> > > > > > settings
> > > > > > > > > (e.g.
> > > > > > > > > > >> default data region), persistence settings (e.g. WAL)
> > and
> > > a
> > > > > list
> > > > > > > of
> > > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > > >>
> > > > > > > > > > >> What we have in the end:
> > > > > > > > > > >>
> > > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > > > >>        <property name="persistentStorePath"
> > > value="/my/path"
> > > > > />
> > > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > > >>            <list>
> > > > > > > > > > >>                <bean
> > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > >>                    <property name="name"
> > value="VOLATILE"
> > > />
> > > > > > > > > > >>                    <property name="maxSize"
> > > > > > value="1_000_000_000"
> > > > > > > />
> > > > > > > > > > >>                </bean>
> > > > > > > > > > >>                <bean
> > > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > > >>                    <property name="name"
> > > value="PERSISTENT"
> > > > />
> > > > > > > > > > >>                    <property name="maxSize"
> > > > > > value="1_000_000_000"
> > > > > > > />
> > > > > > > > > > >>                    <property name="persistent"
> > > value="true"
> > > > />
> > > > > > > > > > >>                </bean>
> > > > > > > > > > >>            </list>
> > > > > > > > > > >>        </property>
> > > > > > > > > > >>    </bean>
> > > > > > > > > > >> </property>
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> Makes sense?
> > > > > > > > > > >>
> > > > > > > > > > >> Vladimir.
> > > > > > > > > > >>
> > > > > > > > > > >>
> > > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > > >> wrote:
> > > > > > > > > > >>
> > > > > > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > > > > > MemoryPolicy?
> > > > > > > > > > >> Secondly,
> > > > > > > > > > >>> why not set data policies directly on
> > > IgniteConfiguration.
> > > > > And
> > > > > > > > > lastly,
> > > > > > > > > > >> how
> > > > > > > > > > >>> about we combine memory and disk properties in one
> bean
> > > > with
> > > > > > > clear
> > > > > > > > > > naming
> > > > > > > > > > >>> convention?
> > > > > > > > > > >>>
> > > > > > > > > > >>> Here is the example. Note that all properties above
> > must
> > > > > start
> > > > > > > with
> > > > > > > > > > with
> > > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > > >>>
> > > > > > > > > > >>> *IgniteConfiguration cfg = new
> IgniteConfiguration();*
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> *cfg.setDataPolicies(    new
> DataPolicyConfiguration()
> > > > > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), //
> > must
> > > > be
> > > > > > > > greater
> > > > > > > > > > >> than
> > > > > > > > > > >>> 0,
> > > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > > .setDiskMaxSize(0),
> > > > > > > > > //
> > > > > > > > > > >> if
> > > > > > > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>>
> > > > > > > > > > >>> I think this approach is much more concise and
> straight
> > > > > > forward.
> > > > > > > > What
> > > > > > > > > > do
> > > > > > > > > > >>> you think?
> > > > > > > > > > >>>
> > > > > > > > > > >>> D.
> > > > > > > > > > >>>
> > > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > > > > > vozerov@gridgain.com
> > > > > > > > > > >
> > > > > > > > > > >>> wrote:
> > > > > > > > > > >>>
> > > > > > > > > > >>>> I prefer the second. Composition over inheritance -
> > this
> > > > is
> > > > > > how
> > > > > > > > all
> > > > > > > > > > our
> > > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > > setCacheStoreFactory".
> > > > > > > > > > >>>>
> > > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > > >>>>
> > > > > > > > > > >>>>> Reiterating this based on some feedback from PDS
> > users.
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> It might be confusing to configure persistence with
> > > > > > > > "MemoryPolicy",
> > > > > > > > > > >> so
> > > > > > > > > > >>>>> another approach is to deprecate the old names and
> > > > > introduce
> > > > > > a
> > > > > > > > new
> > > > > > > > > > >> name
> > > > > > > > > > >>>>> "DataRegion" because it reflects the actual state
> > when
> > > > data
> > > > > > is
> > > > > > > > > stored
> > > > > > > > > > >>> on
> > > > > > > > > > >>>>> disk and partially in memory. I have two options in
> > > mind,
> > > > > > each
> > > > > > > of
> > > > > > > > > > >> them
> > > > > > > > > > >>>>> looks acceptable to me, so I would like to have
> some
> > > > > feedback
> > > > > > > > from
> > > > > > > > > > >> the
> > > > > > > > > > >>>>> community. Old configuration names will be
> deprecated
> > > > (but
> > > > > > > still
> > > > > > > > be
> > > > > > > > > > >>> taken
> > > > > > > > > > >>>>> if used for backward compatibility). Note, that old
> > > names
> > > > > > > > > deprecation
> > > > > > > > > > >>>>> handles default configuration compatibility very
> > > nicely -
> > > > > > > current
> > > > > > > > > PDS
> > > > > > > > > > >>>> users
> > > > > > > > > > >>>>> will not need to change anything to keep everything
> > > > > working.
> > > > > > > The
> > > > > > > > > two
> > > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> * we have two separate classes for in-memory and
> > > > persisted
> > > > > > data
> > > > > > > > > > >>> regions,
> > > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> IgniteConfiguration();
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > DataRegionsConfiguration()
> > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > >>> );
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> * we have one class for data region configuration,
> > but
> > > it
> > > > > > will
> > > > > > > > > have a
> > > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> IgniteConfiguration cfg = new
> IgniteConfiguration();
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > > DataRegionsConfiguration()
> > > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > > >>>>>                new DataRegionPersistenceConfigura
> > tion()
> > > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > > >>> );
> > > > > > > > > > >>>>>
> > > > > > > > > > >>>>
> > > > > > > > > > >>>
> > > > > > > > > > >>
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

I do not understand why we should delay with renames. Yes, it will cause
questions, so we will have to put additional efforts to docs and JavaDocs.
But the earlier we do that, the better.

On Tue, Sep 26, 2017 at 4:50 PM, Dmitry Pavlov <dp...@gmail.com>
wrote:

> Hi Igniters, sorry for late response. I didn't catch idea of renaming.
> PersistentStoreConfiguration is intuitive, and MemoryPolicyConfiguration is
> intuitive also.
>
> If we rename these classes now, it will bring more questions to user list.
> Users may be confused by old and new names and by trying to match it. More
> issues can came from XML configs that users already have.
>
> Can we postpone the renaming? I suggest to finish 'persistence per memory
> policy' task without renaming, and create separate JIRA issue for creating
> future decision?
>
> вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > I do not like DurableMemoryConfiguration, because it's quite confusing -
> we
> > configure in-memory caches using DurableMemory class, which immediately
> > suggests that everything will be persisted. I am not sure if this is a
> > right wording choice for the documentation either. I would go with
> > DataStoreConfiguration and DataRegionConfiguration.
> >
> > --AG
> >
> > 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> >
> > > Given that we already have a notion of CacheStore which comes from
> JCache
> > > spec, I think having other stores may get confusing. I like
> > > DurableMemoryConfiguration.
> > >
> > > Other opinions?
> > >
> > > D.
> > >
> > > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Dima, let's finalize the design first.
> > > >
> > > > As I understand, we are happy with idea to merge MemoryConfiguration
> > > > and PersistentStoreConfiguration
> > > > into something what I called DataConfiguration, and to rename
> > > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > > >
> > > > The only outstanding qurestion is whether DataConfiguration is a good
> > > name.
> > > > I am not very happy with it, so let's think of other alternatives.
> > Quick
> > > > ideas:
> > > > 1) StoreConfiguration - looks perfect to me - short and
> > self-describing,
> > > > but clashes a bit with existing CacheStore
> > > > 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> > > > necessary IMO
> > > > 3) PageStoreConfiguration? GIves a hint to our page-based
> architecture.
> > > > 4) DurableMemoryConfiguration - aligns well with our docs, but I do
> not
> > > > like it - too long and misleading
> > > >
> > > > Any other ideas?
> > > >
> > > > I would prefer to have either [StoreConfiguration +
> > > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > > PageStoreRegionConfiguration]. Looks clean and simple.
> > > >
> > > > Vladimir.
> > > >
> > > >
> > > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > Can you please add the configuration example in the ticket?
> > > > >
> > > > > D.
> > > > >
> > > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com> wrote:
> > > > >
> > > > > > Guys,
> > > > > >
> > > > > > I suggest we finalize the configuration changes in the original
> > > ticket
> > > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and
> > proceed
> > > > with
> > > > > > the changes.
> > > > > >
> > > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> > dsetrakyan@apache.org
> > > >:
> > > > > >
> > > > > > > Can we specify what metrics will look like? I think we should
> not
> > > > just
> > > > > > > blindly merge them.
> > > > > > >
> > > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > > vozerov@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Denis,
> > > > > > > >
> > > > > > > > Makes sense. Thanks for catching it!
> > > > > > > >
> > > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> > dmagda@apache.org>
> > > > > > wrote:
> > > > > > > >
> > > > > > > > > If we’re taking the consolidation path for Memory and
> > > Persistence
> > > > > > > > > configurations then it makes sense to merge MemoryMetrics
> [1]
> > > and
> > > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > > >
> > > > > > > > > Agree?
> > > > > > > > >
> > > > > > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > > apache/ignite/MemoryMetrics.html <
> https://ignite.apache.org/
> > > > > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.
> html>
> > > > > > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > apache/ignite/
> > > > > > > > > PersistenceMetrics.html
> > > > > > > > >
> > > > > > > > > —
> > > > > > > > > Denis
> > > > > > > > >
> > > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > I like it.
> > > > > > > > > >
> > > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > > >
> > > > > > > > > > D.
> > > > > > > > > >
> > > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > > > vozerov@gridgain.com>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > >> Guys,
> > > > > > > > > >>
> > > > > > > > > >> Here is my proposal:
> > > > > > > > > >>
> > > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > > *DataRegionConfiguration*.
> > > > > > > > > >> 2) PersistenceConfiguration is merged with
> > > MemoryConfiguration
> > > > > and
> > > > > > > > > renamed
> > > > > > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> > > > > settings
> > > > > > > > (e.g.
> > > > > > > > > >> default data region), persistence settings (e.g. WAL)
> and
> > a
> > > > list
> > > > > > of
> > > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > > >>
> > > > > > > > > >> What we have in the end:
> > > > > > > > > >>
> > > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > > >>        <property name="persistentStorePath"
> > value="/my/path"
> > > > />
> > > > > > > > > >>        <property name="dataRegions">
> > > > > > > > > >>            <list>
> > > > > > > > > >>                <bean
> > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > >>                    <property name="name"
> value="VOLATILE"
> > />
> > > > > > > > > >>                    <property name="maxSize"
> > > > > value="1_000_000_000"
> > > > > > />
> > > > > > > > > >>                </bean>
> > > > > > > > > >>                <bean
> > class="o.a.i.DataRegionConfiguration">
> > > > > > > > > >>                    <property name="name"
> > value="PERSISTENT"
> > > />
> > > > > > > > > >>                    <property name="maxSize"
> > > > > value="1_000_000_000"
> > > > > > />
> > > > > > > > > >>                    <property name="persistent"
> > value="true"
> > > />
> > > > > > > > > >>                </bean>
> > > > > > > > > >>            </list>
> > > > > > > > > >>        </property>
> > > > > > > > > >>    </bean>
> > > > > > > > > >> </property>
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> Makes sense?
> > > > > > > > > >>
> > > > > > > > > >> Vladimir.
> > > > > > > > > >>
> > > > > > > > > >>
> > > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > > > > dsetrakyan@apache.org>
> > > > > > > > > >> wrote:
> > > > > > > > > >>
> > > > > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > > > > MemoryPolicy?
> > > > > > > > > >> Secondly,
> > > > > > > > > >>> why not set data policies directly on
> > IgniteConfiguration.
> > > > And
> > > > > > > > lastly,
> > > > > > > > > >> how
> > > > > > > > > >>> about we combine memory and disk properties in one bean
> > > with
> > > > > > clear
> > > > > > > > > naming
> > > > > > > > > >>> convention?
> > > > > > > > > >>>
> > > > > > > > > >>> Here is the example. Note that all properties above
> must
> > > > start
> > > > > > with
> > > > > > > > > with
> > > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > > >>>
> > > > > > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), //
> must
> > > be
> > > > > > > greater
> > > > > > > > > >> than
> > > > > > > > > >>> 0,
> > > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > > .setDiskMaxSize(0),
> > > > > > > > //
> > > > > > > > > >> if
> > > > > > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>>
> > > > > > > > > >>> I think this approach is much more concise and straight
> > > > > forward.
> > > > > > > What
> > > > > > > > > do
> > > > > > > > > >>> you think?
> > > > > > > > > >>>
> > > > > > > > > >>> D.
> > > > > > > > > >>>
> > > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > > > > vozerov@gridgain.com
> > > > > > > > > >
> > > > > > > > > >>> wrote:
> > > > > > > > > >>>
> > > > > > > > > >>>> I prefer the second. Composition over inheritance -
> this
> > > is
> > > > > how
> > > > > > > all
> > > > > > > > > our
> > > > > > > > > >>>> configuration is crafted.
> > > > > > > > > >>>>
> > > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > > setCacheStoreFactory".
> > > > > > > > > >>>>
> > > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > > >>>>
> > > > > > > > > >>>>> Reiterating this based on some feedback from PDS
> users.
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> It might be confusing to configure persistence with
> > > > > > > "MemoryPolicy",
> > > > > > > > > >> so
> > > > > > > > > >>>>> another approach is to deprecate the old names and
> > > > introduce
> > > > > a
> > > > > > > new
> > > > > > > > > >> name
> > > > > > > > > >>>>> "DataRegion" because it reflects the actual state
> when
> > > data
> > > > > is
> > > > > > > > stored
> > > > > > > > > >>> on
> > > > > > > > > >>>>> disk and partially in memory. I have two options in
> > mind,
> > > > > each
> > > > > > of
> > > > > > > > > >> them
> > > > > > > > > >>>>> looks acceptable to me, so I would like to have some
> > > > feedback
> > > > > > > from
> > > > > > > > > >> the
> > > > > > > > > >>>>> community. Old configuration names will be deprecated
> > > (but
> > > > > > still
> > > > > > > be
> > > > > > > > > >>> taken
> > > > > > > > > >>>>> if used for backward compatibility). Note, that old
> > names
> > > > > > > > deprecation
> > > > > > > > > >>>>> handles default configuration compatibility very
> > nicely -
> > > > > > current
> > > > > > > > PDS
> > > > > > > > > >>>> users
> > > > > > > > > >>>>> will not need to change anything to keep everything
> > > > working.
> > > > > > The
> > > > > > > > two
> > > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> * we have two separate classes for in-memory and
> > > persisted
> > > > > data
> > > > > > > > > >>> regions,
> > > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > DataRegionsConfiguration()
> > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > >>> );
> > > > > > > > > >>>>>
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> * we have one class for data region configuration,
> but
> > it
> > > > > will
> > > > > > > > have a
> > > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > > DataRegionsConfiguration()
> > > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > >>>>>        new DataRegion()
> > > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > > >>>>>                new DataRegionPersistenceConfigura
> tion()
> > > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > > >>>>>
> > > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > > >>> );
> > > > > > > > > >>>>>
> > > > > > > > > >>>>
> > > > > > > > > >>>
> > > > > > > > > >>
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitry Pavlov <dp...@gmail.com>]

Hi Igniters, sorry for late response. I didn't catch idea of renaming.
PersistentStoreConfiguration is intuitive, and MemoryPolicyConfiguration is
intuitive also.

If we rename these classes now, it will bring more questions to user list.
Users may be confused by old and new names and by trying to match it. More
issues can came from XML configs that users already have.

Can we postpone the renaming? I suggest to finish 'persistence per memory
policy' task without renaming, and create separate JIRA issue for creating
future decision?

вт, 26 сент. 2017 г. в 15:25, Alexey Goncharuk <al...@gmail.com>:

> I do not like DurableMemoryConfiguration, because it's quite confusing - we
> configure in-memory caches using DurableMemory class, which immediately
> suggests that everything will be persisted. I am not sure if this is a
> right wording choice for the documentation either. I would go with
> DataStoreConfiguration and DataRegionConfiguration.
>
> --AG
>
> 2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>
> > Given that we already have a notion of CacheStore which comes from JCache
> > spec, I think having other stores may get confusing. I like
> > DurableMemoryConfiguration.
> >
> > Other opinions?
> >
> > D.
> >
> > On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Dima, let's finalize the design first.
> > >
> > > As I understand, we are happy with idea to merge MemoryConfiguration
> > > and PersistentStoreConfiguration
> > > into something what I called DataConfiguration, and to rename
> > > MemoryPolicyConfiguration to DataRegionConfiguration.
> > >
> > > The only outstanding qurestion is whether DataConfiguration is a good
> > name.
> > > I am not very happy with it, so let's think of other alternatives.
> Quick
> > > ideas:
> > > 1) StoreConfiguration - looks perfect to me - short and
> self-describing,
> > > but clashes a bit with existing CacheStore
> > > 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> > > necessary IMO
> > > 3) PageStoreConfiguration? GIves a hint to our page-based architecture.
> > > 4) DurableMemoryConfiguration - aligns well with our docs, but I do not
> > > like it - too long and misleading
> > >
> > > Any other ideas?
> > >
> > > I would prefer to have either [StoreConfiguration +
> > > StoreRegionConfiguration] or [PageStoreConfiguration and
> > > PageStoreRegionConfiguration]. Looks clean and simple.
> > >
> > > Vladimir.
> > >
> > >
> > > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > Can you please add the configuration example in the ticket?
> > > >
> > > > D.
> > > >
> > > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com> wrote:
> > > >
> > > > > Guys,
> > > > >
> > > > > I suggest we finalize the configuration changes in the original
> > ticket
> > > > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and
> proceed
> > > with
> > > > > the changes.
> > > > >
> > > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >:
> > > > >
> > > > > > Can we specify what metrics will look like? I think we should not
> > > just
> > > > > > blindly merge them.
> > > > > >
> > > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > > vozerov@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Denis,
> > > > > > >
> > > > > > > Makes sense. Thanks for catching it!
> > > > > > >
> > > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <
> dmagda@apache.org>
> > > > > wrote:
> > > > > > >
> > > > > > > > If we’re taking the consolidation path for Memory and
> > Persistence
> > > > > > > > configurations then it makes sense to merge MemoryMetrics [1]
> > and
> > > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > > >
> > > > > > > > Agree?
> > > > > > > >
> > > > > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > apache/ignite/
> > > > > > > > PersistenceMetrics.html
> > > > > > > >
> > > > > > > > —
> > > > > > > > Denis
> > > > > > > >
> > > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > I like it.
> > > > > > > > >
> > > > > > > > > Alexey G, can you please chime in?
> > > > > > > > >
> > > > > > > > > D.
> > > > > > > > >
> > > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > > vozerov@gridgain.com>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > >> Guys,
> > > > > > > > >>
> > > > > > > > >> Here is my proposal:
> > > > > > > > >>
> > > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > > *DataRegionConfiguration*.
> > > > > > > > >> 2) PersistenceConfiguration is merged with
> > MemoryConfiguration
> > > > and
> > > > > > > > renamed
> > > > > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> > > > settings
> > > > > > > (e.g.
> > > > > > > > >> default data region), persistence settings (e.g. WAL) and
> a
> > > list
> > > > > of
> > > > > > > > >> DataRegionConfiguration beans.
> > > > > > > > >>
> > > > > > > > >> What we have in the end:
> > > > > > > > >>
> > > > > > > > >> <property name="dataConfiguration">
> > > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > > >>        <property name="persistentStorePath"
> value="/my/path"
> > > />
> > > > > > > > >>        <property name="dataRegions">
> > > > > > > > >>            <list>
> > > > > > > > >>                <bean
> class="o.a.i.DataRegionConfiguration">
> > > > > > > > >>                    <property name="name" value="VOLATILE"
> />
> > > > > > > > >>                    <property name="maxSize"
> > > > value="1_000_000_000"
> > > > > />
> > > > > > > > >>                </bean>
> > > > > > > > >>                <bean
> class="o.a.i.DataRegionConfiguration">
> > > > > > > > >>                    <property name="name"
> value="PERSISTENT"
> > />
> > > > > > > > >>                    <property name="maxSize"
> > > > value="1_000_000_000"
> > > > > />
> > > > > > > > >>                    <property name="persistent"
> value="true"
> > />
> > > > > > > > >>                </bean>
> > > > > > > > >>            </list>
> > > > > > > > >>        </property>
> > > > > > > > >>    </bean>
> > > > > > > > >> </property>
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> Makes sense?
> > > > > > > > >>
> > > > > > > > >> Vladimir.
> > > > > > > > >>
> > > > > > > > >>
> > > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > > > dsetrakyan@apache.org>
> > > > > > > > >> wrote:
> > > > > > > > >>
> > > > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > > > MemoryPolicy?
> > > > > > > > >> Secondly,
> > > > > > > > >>> why not set data policies directly on
> IgniteConfiguration.
> > > And
> > > > > > > lastly,
> > > > > > > > >> how
> > > > > > > > >>> about we combine memory and disk properties in one bean
> > with
> > > > > clear
> > > > > > > > naming
> > > > > > > > >>> convention?
> > > > > > > > >>>
> > > > > > > > >>> Here is the example. Note that all properties above must
> > > start
> > > > > with
> > > > > > > > with
> > > > > > > > >>> "Memory" or "Disk".
> > > > > > > > >>>
> > > > > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>>
> > > > > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must
> > be
> > > > > > greater
> > > > > > > > >> than
> > > > > > > > >>> 0,
> > > > > > > > >>>> since memory always needs to be enabled.
> > > > > > .setDiskMaxSize(0),
> > > > > > > //
> > > > > > > > >> if
> > > > > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>>
> > > > > > > > >>> I think this approach is much more concise and straight
> > > > forward.
> > > > > > What
> > > > > > > > do
> > > > > > > > >>> you think?
> > > > > > > > >>>
> > > > > > > > >>> D.
> > > > > > > > >>>
> > > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > > > vozerov@gridgain.com
> > > > > > > > >
> > > > > > > > >>> wrote:
> > > > > > > > >>>
> > > > > > > > >>>> I prefer the second. Composition over inheritance - this
> > is
> > > > how
> > > > > > all
> > > > > > > > our
> > > > > > > > >>>> configuration is crafted.
> > > > > > > > >>>>
> > > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > > >>>> Instead, we have "CacheConfiguration.
> > setCacheStoreFactory".
> > > > > > > > >>>>
> > > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > > >>>>
> > > > > > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > > > > > >>>>>
> > > > > > > > >>>>> It might be confusing to configure persistence with
> > > > > > "MemoryPolicy",
> > > > > > > > >> so
> > > > > > > > >>>>> another approach is to deprecate the old names and
> > > introduce
> > > > a
> > > > > > new
> > > > > > > > >> name
> > > > > > > > >>>>> "DataRegion" because it reflects the actual state when
> > data
> > > > is
> > > > > > > stored
> > > > > > > > >>> on
> > > > > > > > >>>>> disk and partially in memory. I have two options in
> mind,
> > > > each
> > > > > of
> > > > > > > > >> them
> > > > > > > > >>>>> looks acceptable to me, so I would like to have some
> > > feedback
> > > > > > from
> > > > > > > > >> the
> > > > > > > > >>>>> community. Old configuration names will be deprecated
> > (but
> > > > > still
> > > > > > be
> > > > > > > > >>> taken
> > > > > > > > >>>>> if used for backward compatibility). Note, that old
> names
> > > > > > > deprecation
> > > > > > > > >>>>> handles default configuration compatibility very
> nicely -
> > > > > current
> > > > > > > PDS
> > > > > > > > >>>> users
> > > > > > > > >>>>> will not need to change anything to keep everything
> > > working.
> > > > > The
> > > > > > > two
> > > > > > > > >>>>> options I mentioned are below:
> > > > > > > > >>>>>
> > > > > > > > >>>>> * we have two separate classes for in-memory and
> > persisted
> > > > data
> > > > > > > > >>> regions,
> > > > > > > > >>>>> so the configuration would look like so:
> > > > > > > > >>>>>
> > > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > > >>>>>
> > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > DataRegionsConfiguration()
> > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > > >>>>>
> > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > >>> );
> > > > > > > > >>>>>
> > > > > > > > >>>>>
> > > > > > > > >>>>> * we have one class for data region configuration, but
> it
> > > > will
> > > > > > > have a
> > > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > > >>>>>
> > > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > > >>>>>
> > > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > > DataRegionsConfiguration()
> > > > > > > > >>>>>    .setDataRegions(
> > > > > > > > >>>>>        new DataRegion()
> > > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > >>>>>        new DataRegion()
> > > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > > >>>>>
> > > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > > >> PersistentStoreConfiguration()
> > > > > > > > >>> );
> > > > > > > > >>>>>
> > > > > > > > >>>>
> > > > > > > > >>>
> > > > > > > > >>
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

I do not like DurableMemoryConfiguration, because it's quite confusing - we
configure in-memory caches using DurableMemory class, which immediately
suggests that everything will be persisted. I am not sure if this is a
right wording choice for the documentation either. I would go with
DataStoreConfiguration and DataRegionConfiguration.

--AG

2017-09-26 2:22 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:

> Given that we already have a notion of CacheStore which comes from JCache
> spec, I think having other stores may get confusing. I like
> DurableMemoryConfiguration.
>
> Other opinions?
>
> D.
>
> On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Dima, let's finalize the design first.
> >
> > As I understand, we are happy with idea to merge MemoryConfiguration
> > and PersistentStoreConfiguration
> > into something what I called DataConfiguration, and to rename
> > MemoryPolicyConfiguration to DataRegionConfiguration.
> >
> > The only outstanding qurestion is whether DataConfiguration is a good
> name.
> > I am not very happy with it, so let's think of other alternatives. Quick
> > ideas:
> > 1) StoreConfiguration - looks perfect to me - short and self-describing,
> > but clashes a bit with existing CacheStore
> > 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> > necessary IMO
> > 3) PageStoreConfiguration? GIves a hint to our page-based architecture.
> > 4) DurableMemoryConfiguration - aligns well with our docs, but I do not
> > like it - too long and misleading
> >
> > Any other ideas?
> >
> > I would prefer to have either [StoreConfiguration +
> > StoreRegionConfiguration] or [PageStoreConfiguration and
> > PageStoreRegionConfiguration]. Looks clean and simple.
> >
> > Vladimir.
> >
> >
> > On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> > wrote:
> >
> > > Vladimir,
> > >
> > > Can you please add the configuration example in the ticket?
> > >
> > > D.
> > >
> > > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > Guys,
> > > >
> > > > I suggest we finalize the configuration changes in the original
> ticket
> > > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed
> > with
> > > > the changes.
> > > >
> > > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org
> >:
> > > >
> > > > > Can we specify what metrics will look like? I think we should not
> > just
> > > > > blindly merge them.
> > > > >
> > > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Denis,
> > > > > >
> > > > > > Makes sense. Thanks for catching it!
> > > > > >
> > > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org>
> > > > wrote:
> > > > > >
> > > > > > > If we’re taking the consolidation path for Memory and
> Persistence
> > > > > > > configurations then it makes sense to merge MemoryMetrics [1]
> and
> > > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > > >
> > > > > > > Agree?
> > > > > > >
> > > > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > apache/ignite/
> > > > > > > PersistenceMetrics.html
> > > > > > >
> > > > > > > —
> > > > > > > Denis
> > > > > > >
> > > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org
> > > > > > >
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > I like it.
> > > > > > > >
> > > > > > > > Alexey G, can you please chime in?
> > > > > > > >
> > > > > > > > D.
> > > > > > > >
> > > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > > vozerov@gridgain.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > >> Guys,
> > > > > > > >>
> > > > > > > >> Here is my proposal:
> > > > > > > >>
> > > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > > *DataRegionConfiguration*.
> > > > > > > >> 2) PersistenceConfiguration is merged with
> MemoryConfiguration
> > > and
> > > > > > > renamed
> > > > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> > > settings
> > > > > > (e.g.
> > > > > > > >> default data region), persistence settings (e.g. WAL) and a
> > list
> > > > of
> > > > > > > >> DataRegionConfiguration beans.
> > > > > > > >>
> > > > > > > >> What we have in the end:
> > > > > > > >>
> > > > > > > >> <property name="dataConfiguration">
> > > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > > >>        <property name="persistentStorePath" value="/my/path"
> > />
> > > > > > > >>        <property name="dataRegions">
> > > > > > > >>            <list>
> > > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > > >>                    <property name="name" value="VOLATILE" />
> > > > > > > >>                    <property name="maxSize"
> > > value="1_000_000_000"
> > > > />
> > > > > > > >>                </bean>
> > > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > > >>                    <property name="name" value="PERSISTENT"
> />
> > > > > > > >>                    <property name="maxSize"
> > > value="1_000_000_000"
> > > > />
> > > > > > > >>                    <property name="persistent" value="true"
> />
> > > > > > > >>                </bean>
> > > > > > > >>            </list>
> > > > > > > >>        </property>
> > > > > > > >>    </bean>
> > > > > > > >> </property>
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> Makes sense?
> > > > > > > >>
> > > > > > > >> Vladimir.
> > > > > > > >>
> > > > > > > >>
> > > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > > dsetrakyan@apache.org>
> > > > > > > >> wrote:
> > > > > > > >>
> > > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > > MemoryPolicy?
> > > > > > > >> Secondly,
> > > > > > > >>> why not set data policies directly on IgniteConfiguration.
> > And
> > > > > > lastly,
> > > > > > > >> how
> > > > > > > >>> about we combine memory and disk properties in one bean
> with
> > > > clear
> > > > > > > naming
> > > > > > > >>> convention?
> > > > > > > >>>
> > > > > > > >>> Here is the example. Note that all properties above must
> > start
> > > > with
> > > > > > > with
> > > > > > > >>> "Memory" or "Disk".
> > > > > > > >>>
> > > > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>>
> > > > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must
> be
> > > > > greater
> > > > > > > >> than
> > > > > > > >>> 0,
> > > > > > > >>>> since memory always needs to be enabled.
> > > > > .setDiskMaxSize(0),
> > > > > > //
> > > > > > > >> if
> > > > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>>
> > > > > > > >>> I think this approach is much more concise and straight
> > > forward.
> > > > > What
> > > > > > > do
> > > > > > > >>> you think?
> > > > > > > >>>
> > > > > > > >>> D.
> > > > > > > >>>
> > > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > > vozerov@gridgain.com
> > > > > > > >
> > > > > > > >>> wrote:
> > > > > > > >>>
> > > > > > > >>>> I prefer the second. Composition over inheritance - this
> is
> > > how
> > > > > all
> > > > > > > our
> > > > > > > >>>> configuration is crafted.
> > > > > > > >>>>
> > > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > > >>>> Instead, we have "CacheConfiguration.
> setCacheStoreFactory".
> > > > > > > >>>>
> > > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > > >>>>
> > > > > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > > > > >>>>>
> > > > > > > >>>>> It might be confusing to configure persistence with
> > > > > "MemoryPolicy",
> > > > > > > >> so
> > > > > > > >>>>> another approach is to deprecate the old names and
> > introduce
> > > a
> > > > > new
> > > > > > > >> name
> > > > > > > >>>>> "DataRegion" because it reflects the actual state when
> data
> > > is
> > > > > > stored
> > > > > > > >>> on
> > > > > > > >>>>> disk and partially in memory. I have two options in mind,
> > > each
> > > > of
> > > > > > > >> them
> > > > > > > >>>>> looks acceptable to me, so I would like to have some
> > feedback
> > > > > from
> > > > > > > >> the
> > > > > > > >>>>> community. Old configuration names will be deprecated
> (but
> > > > still
> > > > > be
> > > > > > > >>> taken
> > > > > > > >>>>> if used for backward compatibility). Note, that old names
> > > > > > deprecation
> > > > > > > >>>>> handles default configuration compatibility very nicely -
> > > > current
> > > > > > PDS
> > > > > > > >>>> users
> > > > > > > >>>>> will not need to change anything to keep everything
> > working.
> > > > The
> > > > > > two
> > > > > > > >>>>> options I mentioned are below:
> > > > > > > >>>>>
> > > > > > > >>>>> * we have two separate classes for in-memory and
> persisted
> > > data
> > > > > > > >>> regions,
> > > > > > > >>>>> so the configuration would look like so:
> > > > > > > >>>>>
> > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > >>>>>
> > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > DataRegionsConfiguration()
> > > > > > > >>>>>    .setDataRegions(
> > > > > > > >>>>>        new MemoryDataRegion()
> > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > >>>>>        new PersistentDataRegion()
> > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > > >>>>>
> > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > >> PersistentStoreConfiguration()
> > > > > > > >>> );
> > > > > > > >>>>>
> > > > > > > >>>>>
> > > > > > > >>>>> * we have one class for data region configuration, but it
> > > will
> > > > > > have a
> > > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > > >>>>>
> > > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > > >>>>>
> > > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > > DataRegionsConfiguration()
> > > > > > > >>>>>    .setDataRegions(
> > > > > > > >>>>>        new DataRegion()
> > > > > > > >>>>>            .setName("volatileCaches")
> > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > >>>>>        new DataRegion()
> > > > > > > >>>>>            .setName("persistentCaches")
> > > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > > >>>>>
> > > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > > >> PersistentStoreConfiguration()
> > > > > > > >>> );
> > > > > > > >>>>>
> > > > > > > >>>>
> > > > > > > >>>
> > > > > > > >>
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Given that we already have a notion of CacheStore which comes from JCache
spec, I think having other stores may get confusing. I like
DurableMemoryConfiguration.

Other opinions?

D.

On Mon, Sep 25, 2017 at 12:24 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> Dima, let's finalize the design first.
>
> As I understand, we are happy with idea to merge MemoryConfiguration
> and PersistentStoreConfiguration
> into something what I called DataConfiguration, and to rename
> MemoryPolicyConfiguration to DataRegionConfiguration.
>
> The only outstanding qurestion is whether DataConfiguration is a good name.
> I am not very happy with it, so let's think of other alternatives. Quick
> ideas:
> 1) StoreConfiguration - looks perfect to me - short and self-describing,
> but clashes a bit with existing CacheStore
> 2) DataStoreConfiguration - same as p.1, but the word "data" is not
> necessary IMO
> 3) PageStoreConfiguration? GIves a hint to our page-based architecture.
> 4) DurableMemoryConfiguration - aligns well with our docs, but I do not
> like it - too long and misleading
>
> Any other ideas?
>
> I would prefer to have either [StoreConfiguration +
> StoreRegionConfiguration] or [PageStoreConfiguration and
> PageStoreRegionConfiguration]. Looks clean and simple.
>
> Vladimir.
>
>
> On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Vladimir,
> >
> > Can you please add the configuration example in the ticket?
> >
> > D.
> >
> > On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Guys,
> > >
> > > I suggest we finalize the configuration changes in the original ticket
> > > then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed
> with
> > > the changes.
> > >
> > > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> > >
> > > > Can we specify what metrics will look like? I think we should not
> just
> > > > blindly merge them.
> > > >
> > > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > Denis,
> > > > >
> > > > > Makes sense. Thanks for catching it!
> > > > >
> > > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org>
> > > wrote:
> > > > >
> > > > > > If we’re taking the consolidation path for Memory and Persistence
> > > > > > configurations then it makes sense to merge MemoryMetrics [1] and
> > > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > > >
> > > > > > Agree?
> > > > > >
> > > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > apache/ignite/
> > > > > > PersistenceMetrics.html
> > > > > >
> > > > > > —
> > > > > > Denis
> > > > > >
> > > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org
> > > > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > I like it.
> > > > > > >
> > > > > > > Alexey G, can you please chime in?
> > > > > > >
> > > > > > > D.
> > > > > > >
> > > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > > vozerov@gridgain.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Guys,
> > > > > > >>
> > > > > > >> Here is my proposal:
> > > > > > >>
> > > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > > *DataRegionConfiguration*.
> > > > > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration
> > and
> > > > > > renamed
> > > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> > settings
> > > > > (e.g.
> > > > > > >> default data region), persistence settings (e.g. WAL) and a
> list
> > > of
> > > > > > >> DataRegionConfiguration beans.
> > > > > > >>
> > > > > > >> What we have in the end:
> > > > > > >>
> > > > > > >> <property name="dataConfiguration">
> > > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > > >>        <property name="pageSize" value="8192" />
> > > > > > >>        <property name="persistentStorePath" value="/my/path"
> />
> > > > > > >>        <property name="dataRegions">
> > > > > > >>            <list>
> > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > >>                    <property name="name" value="VOLATILE" />
> > > > > > >>                    <property name="maxSize"
> > value="1_000_000_000"
> > > />
> > > > > > >>                </bean>
> > > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > > >>                    <property name="name" value="PERSISTENT" />
> > > > > > >>                    <property name="maxSize"
> > value="1_000_000_000"
> > > />
> > > > > > >>                    <property name="persistent" value="true" />
> > > > > > >>                </bean>
> > > > > > >>            </list>
> > > > > > >>        </property>
> > > > > > >>    </bean>
> > > > > > >> </property>
> > > > > > >>
> > > > > > >>
> > > > > > >> Makes sense?
> > > > > > >>
> > > > > > >> Vladimir.
> > > > > > >>
> > > > > > >>
> > > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org>
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >>> Firstly all, why not call it DataPolicy instead of
> > MemoryPolicy?
> > > > > > >> Secondly,
> > > > > > >>> why not set data policies directly on IgniteConfiguration.
> And
> > > > > lastly,
> > > > > > >> how
> > > > > > >>> about we combine memory and disk properties in one bean with
> > > clear
> > > > > > naming
> > > > > > >>> convention?
> > > > > > >>>
> > > > > > >>> Here is the example. Note that all properties above must
> start
> > > with
> > > > > > with
> > > > > > >>> "Memory" or "Disk".
> > > > > > >>>
> > > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be
> > > > greater
> > > > > > >> than
> > > > > > >>> 0,
> > > > > > >>>> since memory always needs to be enabled.
> > > > .setDiskMaxSize(0),
> > > > > //
> > > > > > >> if
> > > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> I think this approach is much more concise and straight
> > forward.
> > > > What
> > > > > > do
> > > > > > >>> you think?
> > > > > > >>>
> > > > > > >>> D.
> > > > > > >>>
> > > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > > vozerov@gridgain.com
> > > > > > >
> > > > > > >>> wrote:
> > > > > > >>>
> > > > > > >>>> I prefer the second. Composition over inheritance - this is
> > how
> > > > all
> > > > > > our
> > > > > > >>>> configuration is crafted.
> > > > > > >>>>
> > > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > > > > >>>>
> > > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > > >>>>
> > > > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > > > >>>>>
> > > > > > >>>>> It might be confusing to configure persistence with
> > > > "MemoryPolicy",
> > > > > > >> so
> > > > > > >>>>> another approach is to deprecate the old names and
> introduce
> > a
> > > > new
> > > > > > >> name
> > > > > > >>>>> "DataRegion" because it reflects the actual state when data
> > is
> > > > > stored
> > > > > > >>> on
> > > > > > >>>>> disk and partially in memory. I have two options in mind,
> > each
> > > of
> > > > > > >> them
> > > > > > >>>>> looks acceptable to me, so I would like to have some
> feedback
> > > > from
> > > > > > >> the
> > > > > > >>>>> community. Old configuration names will be deprecated (but
> > > still
> > > > be
> > > > > > >>> taken
> > > > > > >>>>> if used for backward compatibility). Note, that old names
> > > > > deprecation
> > > > > > >>>>> handles default configuration compatibility very nicely -
> > > current
> > > > > PDS
> > > > > > >>>> users
> > > > > > >>>>> will not need to change anything to keep everything
> working.
> > > The
> > > > > two
> > > > > > >>>>> options I mentioned are below:
> > > > > > >>>>>
> > > > > > >>>>> * we have two separate classes for in-memory and persisted
> > data
> > > > > > >>> regions,
> > > > > > >>>>> so the configuration would look like so:
> > > > > > >>>>>
> > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > >>>>>
> > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > DataRegionsConfiguration()
> > > > > > >>>>>    .setDataRegions(
> > > > > > >>>>>        new MemoryDataRegion()
> > > > > > >>>>>            .setName("volatileCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>        new PersistentDataRegion()
> > > > > > >>>>>            .setName("persistentCaches")
> > > > > > >>>>>            .setMaxMemorySize(...)
> > > > > > >>>>>            .setMaxDiskSize()));
> > > > > > >>>>>
> > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > >> PersistentStoreConfiguration()
> > > > > > >>> );
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> * we have one class for data region configuration, but it
> > will
> > > > > have a
> > > > > > >>>>> sub-bean for persistence configuration:
> > > > > > >>>>>
> > > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > > >>>>>
> > > > > > >>>>> cfg.setDataRegionsConfiguration(new
> > DataRegionsConfiguration()
> > > > > > >>>>>    .setDataRegions(
> > > > > > >>>>>        new DataRegion()
> > > > > > >>>>>            .setName("volatileCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>        new DataRegion()
> > > > > > >>>>>            .setName("persistentCaches")
> > > > > > >>>>>            .setMaxMemorySize(...),
> > > > > > >>>>>            .setPersistenceConfiguration(
> > > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > > >>>>>
> > > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > > >> PersistentStoreConfiguration()
> > > > > > >>> );
> > > > > > >>>>>
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Dima, let's finalize the design first.

As I understand, we are happy with idea to merge MemoryConfiguration
and PersistentStoreConfiguration
into something what I called DataConfiguration, and to rename
MemoryPolicyConfiguration to DataRegionConfiguration.

The only outstanding qurestion is whether DataConfiguration is a good name.
I am not very happy with it, so let's think of other alternatives. Quick
ideas:
1) StoreConfiguration - looks perfect to me - short and self-describing,
but clashes a bit with existing CacheStore
2) DataStoreConfiguration - same as p.1, but the word "data" is not
necessary IMO
3) PageStoreConfiguration? GIves a hint to our page-based architecture.
4) DurableMemoryConfiguration - aligns well with our docs, but I do not
like it - too long and misleading

Any other ideas?

I would prefer to have either [StoreConfiguration +
StoreRegionConfiguration] or [PageStoreConfiguration and
PageStoreRegionConfiguration]. Looks clean and simple.

Vladimir.


On Mon, Sep 25, 2017 at 3:49 PM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> Vladimir,
>
> Can you please add the configuration example in the ticket?
>
> D.
>
> On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Guys,
> >
> > I suggest we finalize the configuration changes in the original ticket
> > then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed with
> > the changes.
> >
> > 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
> >
> > > Can we specify what metrics will look like? I think we should not just
> > > blindly merge them.
> > >
> > > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > > > Denis,
> > > >
> > > > Makes sense. Thanks for catching it!
> > > >
> > > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org>
> > wrote:
> > > >
> > > > > If we’re taking the consolidation path for Memory and Persistence
> > > > > configurations then it makes sense to merge MemoryMetrics [1] and
> > > > > PersistenceMetrics [2] plus their JMX beans.
> > > > >
> > > > > Agree?
> > > > >
> > > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > > apache/ignite/
> > > > > PersistenceMetrics.html
> > > > >
> > > > > —
> > > > > Denis
> > > > >
> > > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org
> > > > >
> > > > > wrote:
> > > > > >
> > > > > > I like it.
> > > > > >
> > > > > > Alexey G, can you please chime in?
> > > > > >
> > > > > > D.
> > > > > >
> > > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com>
> > > > > > wrote:
> > > > > >
> > > > > >> Guys,
> > > > > >>
> > > > > >> Here is my proposal:
> > > > > >>
> > > > > >> 1) MemoryPolicyConfiguration is renamed to
> > > *DataRegionConfiguration*.
> > > > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration
> and
> > > > > renamed
> > > > > >> to ... *DataStorageConfiguration*! It has: common memory
> settings
> > > > (e.g.
> > > > > >> default data region), persistence settings (e.g. WAL) and a list
> > of
> > > > > >> DataRegionConfiguration beans.
> > > > > >>
> > > > > >> What we have in the end:
> > > > > >>
> > > > > >> <property name="dataConfiguration">
> > > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > > >>        <property name="pageSize" value="8192" />
> > > > > >>        <property name="persistentStorePath" value="/my/path" />
> > > > > >>        <property name="dataRegions">
> > > > > >>            <list>
> > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > >>                    <property name="name" value="VOLATILE" />
> > > > > >>                    <property name="maxSize"
> value="1_000_000_000"
> > />
> > > > > >>                </bean>
> > > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > > >>                    <property name="name" value="PERSISTENT" />
> > > > > >>                    <property name="maxSize"
> value="1_000_000_000"
> > />
> > > > > >>                    <property name="persistent" value="true" />
> > > > > >>                </bean>
> > > > > >>            </list>
> > > > > >>        </property>
> > > > > >>    </bean>
> > > > > >> </property>
> > > > > >>
> > > > > >>
> > > > > >> Makes sense?
> > > > > >>
> > > > > >> Vladimir.
> > > > > >>
> > > > > >>
> > > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Firstly all, why not call it DataPolicy instead of
> MemoryPolicy?
> > > > > >> Secondly,
> > > > > >>> why not set data policies directly on IgniteConfiguration. And
> > > > lastly,
> > > > > >> how
> > > > > >>> about we combine memory and disk properties in one bean with
> > clear
> > > > > naming
> > > > > >>> convention?
> > > > > >>>
> > > > > >>> Here is the example. Note that all properties above must start
> > with
> > > > > with
> > > > > >>> "Memory" or "Disk".
> > > > > >>>
> > > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be
> > > greater
> > > > > >> than
> > > > > >>> 0,
> > > > > >>>> since memory always needs to be enabled.
> > > .setDiskMaxSize(0),
> > > > //
> > > > > >> if
> > > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>> I think this approach is much more concise and straight
> forward.
> > > What
> > > > > do
> > > > > >>> you think?
> > > > > >>>
> > > > > >>> D.
> > > > > >>>
> > > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > > vozerov@gridgain.com
> > > > > >
> > > > > >>> wrote:
> > > > > >>>
> > > > > >>>> I prefer the second. Composition over inheritance - this is
> how
> > > all
> > > > > our
> > > > > >>>> configuration is crafted.
> > > > > >>>>
> > > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > > >>>> StoreEnabledCacheConfiguration".
> > > > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > > > >>>>
> > > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > > >>>>
> > > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > > >>>>>
> > > > > >>>>> It might be confusing to configure persistence with
> > > "MemoryPolicy",
> > > > > >> so
> > > > > >>>>> another approach is to deprecate the old names and introduce
> a
> > > new
> > > > > >> name
> > > > > >>>>> "DataRegion" because it reflects the actual state when data
> is
> > > > stored
> > > > > >>> on
> > > > > >>>>> disk and partially in memory. I have two options in mind,
> each
> > of
> > > > > >> them
> > > > > >>>>> looks acceptable to me, so I would like to have some feedback
> > > from
> > > > > >> the
> > > > > >>>>> community. Old configuration names will be deprecated (but
> > still
> > > be
> > > > > >>> taken
> > > > > >>>>> if used for backward compatibility). Note, that old names
> > > > deprecation
> > > > > >>>>> handles default configuration compatibility very nicely -
> > current
> > > > PDS
> > > > > >>>> users
> > > > > >>>>> will not need to change anything to keep everything working.
> > The
> > > > two
> > > > > >>>>> options I mentioned are below:
> > > > > >>>>>
> > > > > >>>>> * we have two separate classes for in-memory and persisted
> data
> > > > > >>> regions,
> > > > > >>>>> so the configuration would look like so:
> > > > > >>>>>
> > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > >>>>>
> > > > > >>>>> cfg.setDataRegionsConfiguration(new
> DataRegionsConfiguration()
> > > > > >>>>>    .setDataRegions(
> > > > > >>>>>        new MemoryDataRegion()
> > > > > >>>>>            .setName("volatileCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>        new PersistentDataRegion()
> > > > > >>>>>            .setName("persistentCaches")
> > > > > >>>>>            .setMaxMemorySize(...)
> > > > > >>>>>            .setMaxDiskSize()));
> > > > > >>>>>
> > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > >> PersistentStoreConfiguration()
> > > > > >>> );
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> * we have one class for data region configuration, but it
> will
> > > > have a
> > > > > >>>>> sub-bean for persistence configuration:
> > > > > >>>>>
> > > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > > >>>>>
> > > > > >>>>> cfg.setDataRegionsConfiguration(new
> DataRegionsConfiguration()
> > > > > >>>>>    .setDataRegions(
> > > > > >>>>>        new DataRegion()
> > > > > >>>>>            .setName("volatileCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>        new DataRegion()
> > > > > >>>>>            .setName("persistentCaches")
> > > > > >>>>>            .setMaxMemorySize(...),
> > > > > >>>>>            .setPersistenceConfiguration(
> > > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > > >>>>>                    .setMaxDiskSize(...))));
> > > > > >>>>>
> > > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > > >> PersistentStoreConfiguration()
> > > > > >>> );
> > > > > >>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>
> > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Vladimir,

Can you please add the configuration example in the ticket?

D.

On Mon, Sep 25, 2017 at 12:20 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Guys,
>
> I suggest we finalize the configuration changes in the original ticket
> then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed with
> the changes.
>
> 2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:
>
> > Can we specify what metrics will look like? I think we should not just
> > blindly merge them.
> >
> > On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > Denis,
> > >
> > > Makes sense. Thanks for catching it!
> > >
> > > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org>
> wrote:
> > >
> > > > If we’re taking the consolidation path for Memory and Persistence
> > > > configurations then it makes sense to merge MemoryMetrics [1] and
> > > > PersistenceMetrics [2] plus their JMX beans.
> > > >
> > > > Agree?
> > > >
> > > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> > apache/ignite/
> > > > PersistenceMetrics.html
> > > >
> > > > —
> > > > Denis
> > > >
> > > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org
> > > >
> > > > wrote:
> > > > >
> > > > > I like it.
> > > > >
> > > > > Alexey G, can you please chime in?
> > > > >
> > > > > D.
> > > > >
> > > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > >> Guys,
> > > > >>
> > > > >> Here is my proposal:
> > > > >>
> > > > >> 1) MemoryPolicyConfiguration is renamed to
> > *DataRegionConfiguration*.
> > > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration and
> > > > renamed
> > > > >> to ... *DataStorageConfiguration*! It has: common memory settings
> > > (e.g.
> > > > >> default data region), persistence settings (e.g. WAL) and a list
> of
> > > > >> DataRegionConfiguration beans.
> > > > >>
> > > > >> What we have in the end:
> > > > >>
> > > > >> <property name="dataConfiguration">
> > > > >>    <bean class="o.a.i.DataConfiguration">
> > > > >>        <property name="pageSize" value="8192" />
> > > > >>        <property name="persistentStorePath" value="/my/path" />
> > > > >>        <property name="dataRegions">
> > > > >>            <list>
> > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > >>                    <property name="name" value="VOLATILE" />
> > > > >>                    <property name="maxSize" value="1_000_000_000"
> />
> > > > >>                </bean>
> > > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > > >>                    <property name="name" value="PERSISTENT" />
> > > > >>                    <property name="maxSize" value="1_000_000_000"
> />
> > > > >>                    <property name="persistent" value="true" />
> > > > >>                </bean>
> > > > >>            </list>
> > > > >>        </property>
> > > > >>    </bean>
> > > > >> </property>
> > > > >>
> > > > >>
> > > > >> Makes sense?
> > > > >>
> > > > >> Vladimir.
> > > > >>
> > > > >>
> > > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org>
> > > > >> wrote:
> > > > >>
> > > > >>> Firstly all, why not call it DataPolicy instead of MemoryPolicy?
> > > > >> Secondly,
> > > > >>> why not set data policies directly on IgniteConfiguration. And
> > > lastly,
> > > > >> how
> > > > >>> about we combine memory and disk properties in one bean with
> clear
> > > > naming
> > > > >>> convention?
> > > > >>>
> > > > >>> Here is the example. Note that all properties above must start
> with
> > > > with
> > > > >>> "Memory" or "Disk".
> > > > >>>
> > > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be
> > greater
> > > > >> than
> > > > >>> 0,
> > > > >>>> since memory always needs to be enabled.
> > .setDiskMaxSize(0),
> > > //
> > > > >> if
> > > > >>>> greater than 0, then persistence is enabled.    );*
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> I think this approach is much more concise and straight forward.
> > What
> > > > do
> > > > >>> you think?
> > > > >>>
> > > > >>> D.
> > > > >>>
> > > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > > vozerov@gridgain.com
> > > > >
> > > > >>> wrote:
> > > > >>>
> > > > >>>> I prefer the second. Composition over inheritance - this is how
> > all
> > > > our
> > > > >>>> configuration is crafted.
> > > > >>>>
> > > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > > >>>> StoreEnabledCacheConfiguration".
> > > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > > >>>>
> > > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > > >>>>
> > > > >>>>> Reiterating this based on some feedback from PDS users.
> > > > >>>>>
> > > > >>>>> It might be confusing to configure persistence with
> > "MemoryPolicy",
> > > > >> so
> > > > >>>>> another approach is to deprecate the old names and introduce a
> > new
> > > > >> name
> > > > >>>>> "DataRegion" because it reflects the actual state when data is
> > > stored
> > > > >>> on
> > > > >>>>> disk and partially in memory. I have two options in mind, each
> of
> > > > >> them
> > > > >>>>> looks acceptable to me, so I would like to have some feedback
> > from
> > > > >> the
> > > > >>>>> community. Old configuration names will be deprecated (but
> still
> > be
> > > > >>> taken
> > > > >>>>> if used for backward compatibility). Note, that old names
> > > deprecation
> > > > >>>>> handles default configuration compatibility very nicely -
> current
> > > PDS
> > > > >>>> users
> > > > >>>>> will not need to change anything to keep everything working.
> The
> > > two
> > > > >>>>> options I mentioned are below:
> > > > >>>>>
> > > > >>>>> * we have two separate classes for in-memory and persisted data
> > > > >>> regions,
> > > > >>>>> so the configuration would look like so:
> > > > >>>>>
> > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > >>>>>
> > > > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > > >>>>>    .setDataRegions(
> > > > >>>>>        new MemoryDataRegion()
> > > > >>>>>            .setName("volatileCaches")
> > > > >>>>>            .setMaxMemorySize(...),
> > > > >>>>>        new PersistentDataRegion()
> > > > >>>>>            .setName("persistentCaches")
> > > > >>>>>            .setMaxMemorySize(...)
> > > > >>>>>            .setMaxDiskSize()));
> > > > >>>>>
> > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > >> PersistentStoreConfiguration()
> > > > >>> );
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> * we have one class for data region configuration, but it will
> > > have a
> > > > >>>>> sub-bean for persistence configuration:
> > > > >>>>>
> > > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > > >>>>>
> > > > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > > >>>>>    .setDataRegions(
> > > > >>>>>        new DataRegion()
> > > > >>>>>            .setName("volatileCaches")
> > > > >>>>>            .setMaxMemorySize(...),
> > > > >>>>>        new DataRegion()
> > > > >>>>>            .setName("persistentCaches")
> > > > >>>>>            .setMaxMemorySize(...),
> > > > >>>>>            .setPersistenceConfiguration(
> > > > >>>>>                new DataRegionPersistenceConfiguration()
> > > > >>>>>                    .setMaxDiskSize(...))));
> > > > >>>>>
> > > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > > >> PersistentStoreConfiguration()
> > > > >>> );
> > > > >>>>>
> > > > >>>>
> > > > >>>
> > > > >>
> > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Guys,

I suggest we finalize the configuration changes in the original ticket
then: https://issues.apache.org/jira/browse/IGNITE-6030 and proceed with
the changes.

2017-09-23 17:08 GMT+03:00 Dmitriy Setrakyan <ds...@apache.org>:

> Can we specify what metrics will look like? I think we should not just
> blindly merge them.
>
> On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > Denis,
> >
> > Makes sense. Thanks for catching it!
> >
> > On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org> wrote:
> >
> > > If we’re taking the consolidation path for Memory and Persistence
> > > configurations then it makes sense to merge MemoryMetrics [1] and
> > > PersistenceMetrics [2] plus their JMX beans.
> > >
> > > Agree?
> > >
> > > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > > [2] https://ignite.apache.org/releases/latest/javadoc/org/
> apache/ignite/
> > > PersistenceMetrics.html
> > >
> > > —
> > > Denis
> > >
> > > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > >
> > > wrote:
> > > >
> > > > I like it.
> > > >
> > > > Alexey G, can you please chime in?
> > > >
> > > > D.
> > > >
> > > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > >> Guys,
> > > >>
> > > >> Here is my proposal:
> > > >>
> > > >> 1) MemoryPolicyConfiguration is renamed to
> *DataRegionConfiguration*.
> > > >> 2) PersistenceConfiguration is merged with MemoryConfiguration and
> > > renamed
> > > >> to ... *DataStorageConfiguration*! It has: common memory settings
> > (e.g.
> > > >> default data region), persistence settings (e.g. WAL) and a list of
> > > >> DataRegionConfiguration beans.
> > > >>
> > > >> What we have in the end:
> > > >>
> > > >> <property name="dataConfiguration">
> > > >>    <bean class="o.a.i.DataConfiguration">
> > > >>        <property name="pageSize" value="8192" />
> > > >>        <property name="persistentStorePath" value="/my/path" />
> > > >>        <property name="dataRegions">
> > > >>            <list>
> > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > >>                    <property name="name" value="VOLATILE" />
> > > >>                    <property name="maxSize" value="1_000_000_000" />
> > > >>                </bean>
> > > >>                <bean class="o.a.i.DataRegionConfiguration">
> > > >>                    <property name="name" value="PERSISTENT" />
> > > >>                    <property name="maxSize" value="1_000_000_000" />
> > > >>                    <property name="persistent" value="true" />
> > > >>                </bean>
> > > >>            </list>
> > > >>        </property>
> > > >>    </bean>
> > > >> </property>
> > > >>
> > > >>
> > > >> Makes sense?
> > > >>
> > > >> Vladimir.
> > > >>
> > > >>
> > > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > >> wrote:
> > > >>
> > > >>> Firstly all, why not call it DataPolicy instead of MemoryPolicy?
> > > >> Secondly,
> > > >>> why not set data policies directly on IgniteConfiguration. And
> > lastly,
> > > >> how
> > > >>> about we combine memory and disk properties in one bean with clear
> > > naming
> > > >>> convention?
> > > >>>
> > > >>> Here is the example. Note that all properties above must start with
> > > with
> > > >>> "Memory" or "Disk".
> > > >>>
> > > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be
> greater
> > > >> than
> > > >>> 0,
> > > >>>> since memory always needs to be enabled.
> .setDiskMaxSize(0),
> > //
> > > >> if
> > > >>>> greater than 0, then persistence is enabled.    );*
> > > >>>
> > > >>>
> > > >>>
> > > >>> I think this approach is much more concise and straight forward.
> What
> > > do
> > > >>> you think?
> > > >>>
> > > >>> D.
> > > >>>
> > > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> > vozerov@gridgain.com
> > > >
> > > >>> wrote:
> > > >>>
> > > >>>> I prefer the second. Composition over inheritance - this is how
> all
> > > our
> > > >>>> configuration is crafted.
> > > >>>>
> > > >>>> E.g. we do not have "CacheConfiguration" and "
> > > >>>> StoreEnabledCacheConfiguration".
> > > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > > >>>>
> > > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > >>>> alexey.goncharuk@gmail.com> wrote:
> > > >>>>
> > > >>>>> Reiterating this based on some feedback from PDS users.
> > > >>>>>
> > > >>>>> It might be confusing to configure persistence with
> "MemoryPolicy",
> > > >> so
> > > >>>>> another approach is to deprecate the old names and introduce a
> new
> > > >> name
> > > >>>>> "DataRegion" because it reflects the actual state when data is
> > stored
> > > >>> on
> > > >>>>> disk and partially in memory. I have two options in mind, each of
> > > >> them
> > > >>>>> looks acceptable to me, so I would like to have some feedback
> from
> > > >> the
> > > >>>>> community. Old configuration names will be deprecated (but still
> be
> > > >>> taken
> > > >>>>> if used for backward compatibility). Note, that old names
> > deprecation
> > > >>>>> handles default configuration compatibility very nicely - current
> > PDS
> > > >>>> users
> > > >>>>> will not need to change anything to keep everything working. The
> > two
> > > >>>>> options I mentioned are below:
> > > >>>>>
> > > >>>>> * we have two separate classes for in-memory and persisted data
> > > >>> regions,
> > > >>>>> so the configuration would look like so:
> > > >>>>>
> > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > >>>>>
> > > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > >>>>>    .setDataRegions(
> > > >>>>>        new MemoryDataRegion()
> > > >>>>>            .setName("volatileCaches")
> > > >>>>>            .setMaxMemorySize(...),
> > > >>>>>        new PersistentDataRegion()
> > > >>>>>            .setName("persistentCaches")
> > > >>>>>            .setMaxMemorySize(...)
> > > >>>>>            .setMaxDiskSize()));
> > > >>>>>
> > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > >> PersistentStoreConfiguration()
> > > >>> );
> > > >>>>>
> > > >>>>>
> > > >>>>> * we have one class for data region configuration, but it will
> > have a
> > > >>>>> sub-bean for persistence configuration:
> > > >>>>>
> > > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > > >>>>>
> > > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > >>>>>    .setDataRegions(
> > > >>>>>        new DataRegion()
> > > >>>>>            .setName("volatileCaches")
> > > >>>>>            .setMaxMemorySize(...),
> > > >>>>>        new DataRegion()
> > > >>>>>            .setName("persistentCaches")
> > > >>>>>            .setMaxMemorySize(...),
> > > >>>>>            .setPersistenceConfiguration(
> > > >>>>>                new DataRegionPersistenceConfiguration()
> > > >>>>>                    .setMaxDiskSize(...))));
> > > >>>>>
> > > >>>>> cfg.setPersistentStoreConfiguration(new
> > > >> PersistentStoreConfiguration()
> > > >>> );
> > > >>>>>
> > > >>>>
> > > >>>
> > > >>
> > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Can we specify what metrics will look like? I think we should not just
blindly merge them.

On Fri, Sep 22, 2017 at 11:01 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> Denis,
>
> Makes sense. Thanks for catching it!
>
> On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org> wrote:
>
> > If we’re taking the consolidation path for Memory and Persistence
> > configurations then it makes sense to merge MemoryMetrics [1] and
> > PersistenceMetrics [2] plus their JMX beans.
> >
> > Agree?
> >
> > [1] https://ignite.apache.org/releases/latest/javadoc/org/
> > apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> > releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> > [2] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/
> > PersistenceMetrics.html
> >
> > —
> > Denis
> >
> > > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> > wrote:
> > >
> > > I like it.
> > >
> > > Alexey G, can you please chime in?
> > >
> > > D.
> > >
> > > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > >> Guys,
> > >>
> > >> Here is my proposal:
> > >>
> > >> 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*.
> > >> 2) PersistenceConfiguration is merged with MemoryConfiguration and
> > renamed
> > >> to ... *DataStorageConfiguration*! It has: common memory settings
> (e.g.
> > >> default data region), persistence settings (e.g. WAL) and a list of
> > >> DataRegionConfiguration beans.
> > >>
> > >> What we have in the end:
> > >>
> > >> <property name="dataConfiguration">
> > >>    <bean class="o.a.i.DataConfiguration">
> > >>        <property name="pageSize" value="8192" />
> > >>        <property name="persistentStorePath" value="/my/path" />
> > >>        <property name="dataRegions">
> > >>            <list>
> > >>                <bean class="o.a.i.DataRegionConfiguration">
> > >>                    <property name="name" value="VOLATILE" />
> > >>                    <property name="maxSize" value="1_000_000_000" />
> > >>                </bean>
> > >>                <bean class="o.a.i.DataRegionConfiguration">
> > >>                    <property name="name" value="PERSISTENT" />
> > >>                    <property name="maxSize" value="1_000_000_000" />
> > >>                    <property name="persistent" value="true" />
> > >>                </bean>
> > >>            </list>
> > >>        </property>
> > >>    </bean>
> > >> </property>
> > >>
> > >>
> > >> Makes sense?
> > >>
> > >> Vladimir.
> > >>
> > >>
> > >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >> wrote:
> > >>
> > >>> Firstly all, why not call it DataPolicy instead of MemoryPolicy?
> > >> Secondly,
> > >>> why not set data policies directly on IgniteConfiguration. And
> lastly,
> > >> how
> > >>> about we combine memory and disk properties in one bean with clear
> > naming
> > >>> convention?
> > >>>
> > >>> Here is the example. Note that all properties above must start with
> > with
> > >>> "Memory" or "Disk".
> > >>>
> > >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be greater
> > >> than
> > >>> 0,
> > >>>> since memory always needs to be enabled.        .setDiskMaxSize(0),
> //
> > >> if
> > >>>> greater than 0, then persistence is enabled.    );*
> > >>>
> > >>>
> > >>>
> > >>> I think this approach is much more concise and straight forward. What
> > do
> > >>> you think?
> > >>>
> > >>> D.
> > >>>
> > >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <
> vozerov@gridgain.com
> > >
> > >>> wrote:
> > >>>
> > >>>> I prefer the second. Composition over inheritance - this is how all
> > our
> > >>>> configuration is crafted.
> > >>>>
> > >>>> E.g. we do not have "CacheConfiguration" and "
> > >>>> StoreEnabledCacheConfiguration".
> > >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > >>>>
> > >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > >>>> alexey.goncharuk@gmail.com> wrote:
> > >>>>
> > >>>>> Reiterating this based on some feedback from PDS users.
> > >>>>>
> > >>>>> It might be confusing to configure persistence with "MemoryPolicy",
> > >> so
> > >>>>> another approach is to deprecate the old names and introduce a new
> > >> name
> > >>>>> "DataRegion" because it reflects the actual state when data is
> stored
> > >>> on
> > >>>>> disk and partially in memory. I have two options in mind, each of
> > >> them
> > >>>>> looks acceptable to me, so I would like to have some feedback from
> > >> the
> > >>>>> community. Old configuration names will be deprecated (but still be
> > >>> taken
> > >>>>> if used for backward compatibility). Note, that old names
> deprecation
> > >>>>> handles default configuration compatibility very nicely - current
> PDS
> > >>>> users
> > >>>>> will not need to change anything to keep everything working. The
> two
> > >>>>> options I mentioned are below:
> > >>>>>
> > >>>>> * we have two separate classes for in-memory and persisted data
> > >>> regions,
> > >>>>> so the configuration would look like so:
> > >>>>>
> > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > >>>>>
> > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > >>>>>    .setDataRegions(
> > >>>>>        new MemoryDataRegion()
> > >>>>>            .setName("volatileCaches")
> > >>>>>            .setMaxMemorySize(...),
> > >>>>>        new PersistentDataRegion()
> > >>>>>            .setName("persistentCaches")
> > >>>>>            .setMaxMemorySize(...)
> > >>>>>            .setMaxDiskSize()));
> > >>>>>
> > >>>>> cfg.setPersistentStoreConfiguration(new
> > >> PersistentStoreConfiguration()
> > >>> );
> > >>>>>
> > >>>>>
> > >>>>> * we have one class for data region configuration, but it will
> have a
> > >>>>> sub-bean for persistence configuration:
> > >>>>>
> > >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> > >>>>>
> > >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > >>>>>    .setDataRegions(
> > >>>>>        new DataRegion()
> > >>>>>            .setName("volatileCaches")
> > >>>>>            .setMaxMemorySize(...),
> > >>>>>        new DataRegion()
> > >>>>>            .setName("persistentCaches")
> > >>>>>            .setMaxMemorySize(...),
> > >>>>>            .setPersistenceConfiguration(
> > >>>>>                new DataRegionPersistenceConfiguration()
> > >>>>>                    .setMaxDiskSize(...))));
> > >>>>>
> > >>>>> cfg.setPersistentStoreConfiguration(new
> > >> PersistentStoreConfiguration()
> > >>> );
> > >>>>>
> > >>>>
> > >>>
> > >>
> >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Denis,

Makes sense. Thanks for catching it!

On Sat, Sep 23, 2017 at 8:45 AM, Denis Magda <dm...@apache.org> wrote:

> If we’re taking the consolidation path for Memory and Persistence
> configurations then it makes sense to merge MemoryMetrics [1] and
> PersistenceMetrics [2] plus their JMX beans.
>
> Agree?
>
> [1] https://ignite.apache.org/releases/latest/javadoc/org/
> apache/ignite/MemoryMetrics.html <https://ignite.apache.org/
> releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
> [2] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/
> PersistenceMetrics.html
>
> —
> Denis
>
> > On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > I like it.
> >
> > Alexey G, can you please chime in?
> >
> > D.
> >
> > On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> >> Guys,
> >>
> >> Here is my proposal:
> >>
> >> 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*.
> >> 2) PersistenceConfiguration is merged with MemoryConfiguration and
> renamed
> >> to ... *DataStorageConfiguration*! It has: common memory settings (e.g.
> >> default data region), persistence settings (e.g. WAL) and a list of
> >> DataRegionConfiguration beans.
> >>
> >> What we have in the end:
> >>
> >> <property name="dataConfiguration">
> >>    <bean class="o.a.i.DataConfiguration">
> >>        <property name="pageSize" value="8192" />
> >>        <property name="persistentStorePath" value="/my/path" />
> >>        <property name="dataRegions">
> >>            <list>
> >>                <bean class="o.a.i.DataRegionConfiguration">
> >>                    <property name="name" value="VOLATILE" />
> >>                    <property name="maxSize" value="1_000_000_000" />
> >>                </bean>
> >>                <bean class="o.a.i.DataRegionConfiguration">
> >>                    <property name="name" value="PERSISTENT" />
> >>                    <property name="maxSize" value="1_000_000_000" />
> >>                    <property name="persistent" value="true" />
> >>                </bean>
> >>            </list>
> >>        </property>
> >>    </bean>
> >> </property>
> >>
> >>
> >> Makes sense?
> >>
> >> Vladimir.
> >>
> >>
> >> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >> wrote:
> >>
> >>> Firstly all, why not call it DataPolicy instead of MemoryPolicy?
> >> Secondly,
> >>> why not set data policies directly on IgniteConfiguration. And lastly,
> >> how
> >>> about we combine memory and disk properties in one bean with clear
> naming
> >>> convention?
> >>>
> >>> Here is the example. Note that all properties above must start with
> with
> >>> "Memory" or "Disk".
> >>>
> >>> *IgniteConfiguration cfg = new IgniteConfiguration();*
> >>>
> >>>
> >>>
> >>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> >>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be greater
> >> than
> >>> 0,
> >>>> since memory always needs to be enabled.        .setDiskMaxSize(0), //
> >> if
> >>>> greater than 0, then persistence is enabled.    );*
> >>>
> >>>
> >>>
> >>> I think this approach is much more concise and straight forward. What
> do
> >>> you think?
> >>>
> >>> D.
> >>>
> >>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <vozerov@gridgain.com
> >
> >>> wrote:
> >>>
> >>>> I prefer the second. Composition over inheritance - this is how all
> our
> >>>> configuration is crafted.
> >>>>
> >>>> E.g. we do not have "CacheConfiguration" and "
> >>>> StoreEnabledCacheConfiguration".
> >>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
> >>>>
> >>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> >>>> alexey.goncharuk@gmail.com> wrote:
> >>>>
> >>>>> Reiterating this based on some feedback from PDS users.
> >>>>>
> >>>>> It might be confusing to configure persistence with "MemoryPolicy",
> >> so
> >>>>> another approach is to deprecate the old names and introduce a new
> >> name
> >>>>> "DataRegion" because it reflects the actual state when data is stored
> >>> on
> >>>>> disk and partially in memory. I have two options in mind, each of
> >> them
> >>>>> looks acceptable to me, so I would like to have some feedback from
> >> the
> >>>>> community. Old configuration names will be deprecated (but still be
> >>> taken
> >>>>> if used for backward compatibility). Note, that old names deprecation
> >>>>> handles default configuration compatibility very nicely - current PDS
> >>>> users
> >>>>> will not need to change anything to keep everything working. The two
> >>>>> options I mentioned are below:
> >>>>>
> >>>>> * we have two separate classes for in-memory and persisted data
> >>> regions,
> >>>>> so the configuration would look like so:
> >>>>>
> >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> >>>>>
> >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> >>>>>    .setDataRegions(
> >>>>>        new MemoryDataRegion()
> >>>>>            .setName("volatileCaches")
> >>>>>            .setMaxMemorySize(...),
> >>>>>        new PersistentDataRegion()
> >>>>>            .setName("persistentCaches")
> >>>>>            .setMaxMemorySize(...)
> >>>>>            .setMaxDiskSize()));
> >>>>>
> >>>>> cfg.setPersistentStoreConfiguration(new
> >> PersistentStoreConfiguration()
> >>> );
> >>>>>
> >>>>>
> >>>>> * we have one class for data region configuration, but it will have a
> >>>>> sub-bean for persistence configuration:
> >>>>>
> >>>>> IgniteConfiguration cfg = new IgniteConfiguration();
> >>>>>
> >>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> >>>>>    .setDataRegions(
> >>>>>        new DataRegion()
> >>>>>            .setName("volatileCaches")
> >>>>>            .setMaxMemorySize(...),
> >>>>>        new DataRegion()
> >>>>>            .setName("persistentCaches")
> >>>>>            .setMaxMemorySize(...),
> >>>>>            .setPersistenceConfiguration(
> >>>>>                new DataRegionPersistenceConfiguration()
> >>>>>                    .setMaxDiskSize(...))));
> >>>>>
> >>>>> cfg.setPersistentStoreConfiguration(new
> >> PersistentStoreConfiguration()
> >>> );
> >>>>>
> >>>>
> >>>
> >>
>
>

Re: Persistence per memory policy configuration

[Denis Magda <dm...@apache.org>]

If we’re taking the consolidation path for Memory and Persistence configurations then it makes sense to merge MemoryMetrics [1] and PersistenceMetrics [2] plus their JMX beans.

Agree?

[1] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html <https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/MemoryMetrics.html>
[2] https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/PersistenceMetrics.html

—
Denis

> On Sep 22, 2017, at 10:18 PM, Dmitriy Setrakyan <ds...@apache.org> wrote:
> 
> I like it.
> 
> Alexey G, can you please chime in?
> 
> D.
> 
> On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
> 
>> Guys,
>> 
>> Here is my proposal:
>> 
>> 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*.
>> 2) PersistenceConfiguration is merged with MemoryConfiguration and renamed
>> to ... *DataStorageConfiguration*! It has: common memory settings (e.g.
>> default data region), persistence settings (e.g. WAL) and a list of
>> DataRegionConfiguration beans.
>> 
>> What we have in the end:
>> 
>> <property name="dataConfiguration">
>>    <bean class="o.a.i.DataConfiguration">
>>        <property name="pageSize" value="8192" />
>>        <property name="persistentStorePath" value="/my/path" />
>>        <property name="dataRegions">
>>            <list>
>>                <bean class="o.a.i.DataRegionConfiguration">
>>                    <property name="name" value="VOLATILE" />
>>                    <property name="maxSize" value="1_000_000_000" />
>>                </bean>
>>                <bean class="o.a.i.DataRegionConfiguration">
>>                    <property name="name" value="PERSISTENT" />
>>                    <property name="maxSize" value="1_000_000_000" />
>>                    <property name="persistent" value="true" />
>>                </bean>
>>            </list>
>>        </property>
>>    </bean>
>> </property>
>> 
>> 
>> Makes sense?
>> 
>> Vladimir.
>> 
>> 
>> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>> 
>>> Firstly all, why not call it DataPolicy instead of MemoryPolicy?
>> Secondly,
>>> why not set data policies directly on IgniteConfiguration. And lastly,
>> how
>>> about we combine memory and disk properties in one bean with clear naming
>>> convention?
>>> 
>>> Here is the example. Note that all properties above must start with with
>>> "Memory" or "Disk".
>>> 
>>> *IgniteConfiguration cfg = new IgniteConfiguration();*
>>> 
>>> 
>>> 
>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> *cfg.setDataPolicies(    new DataPolicyConfiguration()
>>>> .setName("bla"),        .setMemoryMaxSize(1024), // must be greater
>> than
>>> 0,
>>>> since memory always needs to be enabled.        .setDiskMaxSize(0), //
>> if
>>>> greater than 0, then persistence is enabled.    );*
>>> 
>>> 
>>> 
>>> I think this approach is much more concise and straight forward. What do
>>> you think?
>>> 
>>> D.
>>> 
>>> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <vo...@gridgain.com>
>>> wrote:
>>> 
>>>> I prefer the second. Composition over inheritance - this is how all our
>>>> configuration is crafted.
>>>> 
>>>> E.g. we do not have "CacheConfiguration" and "
>>>> StoreEnabledCacheConfiguration".
>>>> Instead, we have "CacheConfiguration.setCacheStoreFactory".
>>>> 
>>>> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
>>>> alexey.goncharuk@gmail.com> wrote:
>>>> 
>>>>> Reiterating this based on some feedback from PDS users.
>>>>> 
>>>>> It might be confusing to configure persistence with "MemoryPolicy",
>> so
>>>>> another approach is to deprecate the old names and introduce a new
>> name
>>>>> "DataRegion" because it reflects the actual state when data is stored
>>> on
>>>>> disk and partially in memory. I have two options in mind, each of
>> them
>>>>> looks acceptable to me, so I would like to have some feedback from
>> the
>>>>> community. Old configuration names will be deprecated (but still be
>>> taken
>>>>> if used for backward compatibility). Note, that old names deprecation
>>>>> handles default configuration compatibility very nicely - current PDS
>>>> users
>>>>> will not need to change anything to keep everything working. The two
>>>>> options I mentioned are below:
>>>>> 
>>>>> * we have two separate classes for in-memory and persisted data
>>> regions,
>>>>> so the configuration would look like so:
>>>>> 
>>>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>>>> 
>>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
>>>>>    .setDataRegions(
>>>>>        new MemoryDataRegion()
>>>>>            .setName("volatileCaches")
>>>>>            .setMaxMemorySize(...),
>>>>>        new PersistentDataRegion()
>>>>>            .setName("persistentCaches")
>>>>>            .setMaxMemorySize(...)
>>>>>            .setMaxDiskSize()));
>>>>> 
>>>>> cfg.setPersistentStoreConfiguration(new
>> PersistentStoreConfiguration()
>>> );
>>>>> 
>>>>> 
>>>>> * we have one class for data region configuration, but it will have a
>>>>> sub-bean for persistence configuration:
>>>>> 
>>>>> IgniteConfiguration cfg = new IgniteConfiguration();
>>>>> 
>>>>> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
>>>>>    .setDataRegions(
>>>>>        new DataRegion()
>>>>>            .setName("volatileCaches")
>>>>>            .setMaxMemorySize(...),
>>>>>        new DataRegion()
>>>>>            .setName("persistentCaches")
>>>>>            .setMaxMemorySize(...),
>>>>>            .setPersistenceConfiguration(
>>>>>                new DataRegionPersistenceConfiguration()
>>>>>                    .setMaxDiskSize(...))));
>>>>> 
>>>>> cfg.setPersistentStoreConfiguration(new
>> PersistentStoreConfiguration()
>>> );
>>>>> 
>>>> 
>>> 
>> 

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

I like it.

Alexey G, can you please chime in?

D.

On Fri, Sep 22, 2017 at 11:33 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> Guys,
>
> Here is my proposal:
>
> 1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*.
> 2) PersistenceConfiguration is merged with MemoryConfiguration and renamed
> to ... *DataStorageConfiguration*! It has: common memory settings (e.g.
> default data region), persistence settings (e.g. WAL) and a list of
> DataRegionConfiguration beans.
>
> What we have in the end:
>
> <property name="dataConfiguration">
>     <bean class="o.a.i.DataConfiguration">
>         <property name="pageSize" value="8192" />
>         <property name="persistentStorePath" value="/my/path" />
>         <property name="dataRegions">
>             <list>
>                 <bean class="o.a.i.DataRegionConfiguration">
>                     <property name="name" value="VOLATILE" />
>                     <property name="maxSize" value="1_000_000_000" />
>                 </bean>
>                 <bean class="o.a.i.DataRegionConfiguration">
>                     <property name="name" value="PERSISTENT" />
>                     <property name="maxSize" value="1_000_000_000" />
>                     <property name="persistent" value="true" />
>                 </bean>
>             </list>
>         </property>
>     </bean>
> </property>
>
>
> Makes sense?
>
> Vladimir.
>
>
> On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > Firstly all, why not call it DataPolicy instead of MemoryPolicy?
> Secondly,
> > why not set data policies directly on IgniteConfiguration. And lastly,
> how
> > about we combine memory and disk properties in one bean with clear naming
> > convention?
> >
> > Here is the example. Note that all properties above must start with with
> > "Memory" or "Disk".
> >
> > *IgniteConfiguration cfg = new IgniteConfiguration();*
> >
> >
> >
> >
> > >
> > >
> > >
> > >
> > > *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > > .setName("bla"),        .setMemoryMaxSize(1024), // must be greater
> than
> > 0,
> > > since memory always needs to be enabled.        .setDiskMaxSize(0), //
> if
> > > greater than 0, then persistence is enabled.    );*
> >
> >
> >
> > I think this approach is much more concise and straight forward. What do
> > you think?
> >
> > D.
> >
> > On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > I prefer the second. Composition over inheritance - this is how all our
> > > configuration is crafted.
> > >
> > > E.g. we do not have "CacheConfiguration" and "
> > > StoreEnabledCacheConfiguration".
> > > Instead, we have "CacheConfiguration.setCacheStoreFactory".
> > >
> > > On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > Reiterating this based on some feedback from PDS users.
> > > >
> > > > It might be confusing to configure persistence with "MemoryPolicy",
> so
> > > > another approach is to deprecate the old names and introduce a new
> name
> > > > "DataRegion" because it reflects the actual state when data is stored
> > on
> > > > disk and partially in memory. I have two options in mind, each of
> them
> > > > looks acceptable to me, so I would like to have some feedback from
> the
> > > > community. Old configuration names will be deprecated (but still be
> > taken
> > > > if used for backward compatibility). Note, that old names deprecation
> > > > handles default configuration compatibility very nicely - current PDS
> > > users
> > > > will not need to change anything to keep everything working. The two
> > > > options I mentioned are below:
> > > >
> > > >  * we have two separate classes for in-memory and persisted data
> > regions,
> > > > so the configuration would look like so:
> > > >
> > > > IgniteConfiguration cfg = new IgniteConfiguration();
> > > >
> > > > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > >     .setDataRegions(
> > > >         new MemoryDataRegion()
> > > >             .setName("volatileCaches")
> > > >             .setMaxMemorySize(...),
> > > >         new PersistentDataRegion()
> > > >             .setName("persistentCaches")
> > > >             .setMaxMemorySize(...)
> > > >             .setMaxDiskSize()));
> > > >
> > > > cfg.setPersistentStoreConfiguration(new
> PersistentStoreConfiguration()
> > );
> > > >
> > > >
> > > > * we have one class for data region configuration, but it will have a
> > > > sub-bean for persistence configuration:
> > > >
> > > > IgniteConfiguration cfg = new IgniteConfiguration();
> > > >
> > > > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > > >     .setDataRegions(
> > > >         new DataRegion()
> > > >             .setName("volatileCaches")
> > > >             .setMaxMemorySize(...),
> > > >         new DataRegion()
> > > >             .setName("persistentCaches")
> > > >             .setMaxMemorySize(...),
> > > >             .setPersistenceConfiguration(
> > > >                 new DataRegionPersistenceConfiguration()
> > > >                     .setMaxDiskSize(...))));
> > > >
> > > > cfg.setPersistentStoreConfiguration(new
> PersistentStoreConfiguration()
> > );
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Guys,

Here is my proposal:

1) MemoryPolicyConfiguration is renamed to *DataRegionConfiguration*.
2) PersistenceConfiguration is merged with MemoryConfiguration and renamed
to ... *DataStorageConfiguration*! It has: common memory settings (e.g.
default data region), persistence settings (e.g. WAL) and a list of
DataRegionConfiguration beans.

What we have in the end:

<property name="dataConfiguration">
    <bean class="o.a.i.DataConfiguration">
        <property name="pageSize" value="8192" />
        <property name="persistentStorePath" value="/my/path" />
        <property name="dataRegions">
            <list>
                <bean class="o.a.i.DataRegionConfiguration">
                    <property name="name" value="VOLATILE" />
                    <property name="maxSize" value="1_000_000_000" />
                </bean>
                <bean class="o.a.i.DataRegionConfiguration">
                    <property name="name" value="PERSISTENT" />
                    <property name="maxSize" value="1_000_000_000" />
                    <property name="persistent" value="true" />
                </bean>
            </list>
        </property>
    </bean>
</property>


Makes sense?

Vladimir.


On Thu, Sep 21, 2017 at 7:04 AM, Dmitriy Setrakyan <ds...@apache.org>
wrote:

> Firstly all, why not call it DataPolicy instead of MemoryPolicy? Secondly,
> why not set data policies directly on IgniteConfiguration. And lastly, how
> about we combine memory and disk properties in one bean with clear naming
> convention?
>
> Here is the example. Note that all properties above must start with with
> "Memory" or "Disk".
>
> *IgniteConfiguration cfg = new IgniteConfiguration();*
>
>
>
>
> >
> >
> >
> >
> > *cfg.setDataPolicies(    new DataPolicyConfiguration()
> > .setName("bla"),        .setMemoryMaxSize(1024), // must be greater than
> 0,
> > since memory always needs to be enabled.        .setDiskMaxSize(0), // if
> > greater than 0, then persistence is enabled.    );*
>
>
>
> I think this approach is much more concise and straight forward. What do
> you think?
>
> D.
>
> On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > I prefer the second. Composition over inheritance - this is how all our
> > configuration is crafted.
> >
> > E.g. we do not have "CacheConfiguration" and "
> > StoreEnabledCacheConfiguration".
> > Instead, we have "CacheConfiguration.setCacheStoreFactory".
> >
> > On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Reiterating this based on some feedback from PDS users.
> > >
> > > It might be confusing to configure persistence with "MemoryPolicy", so
> > > another approach is to deprecate the old names and introduce a new name
> > > "DataRegion" because it reflects the actual state when data is stored
> on
> > > disk and partially in memory. I have two options in mind, each of them
> > > looks acceptable to me, so I would like to have some feedback from the
> > > community. Old configuration names will be deprecated (but still be
> taken
> > > if used for backward compatibility). Note, that old names deprecation
> > > handles default configuration compatibility very nicely - current PDS
> > users
> > > will not need to change anything to keep everything working. The two
> > > options I mentioned are below:
> > >
> > >  * we have two separate classes for in-memory and persisted data
> regions,
> > > so the configuration would look like so:
> > >
> > > IgniteConfiguration cfg = new IgniteConfiguration();
> > >
> > > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > >     .setDataRegions(
> > >         new MemoryDataRegion()
> > >             .setName("volatileCaches")
> > >             .setMaxMemorySize(...),
> > >         new PersistentDataRegion()
> > >             .setName("persistentCaches")
> > >             .setMaxMemorySize(...)
> > >             .setMaxDiskSize()));
> > >
> > > cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration()
> );
> > >
> > >
> > > * we have one class for data region configuration, but it will have a
> > > sub-bean for persistence configuration:
> > >
> > > IgniteConfiguration cfg = new IgniteConfiguration();
> > >
> > > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> > >     .setDataRegions(
> > >         new DataRegion()
> > >             .setName("volatileCaches")
> > >             .setMaxMemorySize(...),
> > >         new DataRegion()
> > >             .setName("persistentCaches")
> > >             .setMaxMemorySize(...),
> > >             .setPersistenceConfiguration(
> > >                 new DataRegionPersistenceConfiguration()
> > >                     .setMaxDiskSize(...))));
> > >
> > > cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration()
> );
> > >
> >
>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

Firstly all, why not call it DataPolicy instead of MemoryPolicy? Secondly,
why not set data policies directly on IgniteConfiguration. And lastly, how
about we combine memory and disk properties in one bean with clear naming
convention?

Here is the example. Note that all properties above must start with with
"Memory" or "Disk".

*IgniteConfiguration cfg = new IgniteConfiguration();*




>
>
>
>
> *cfg.setDataPolicies(    new DataPolicyConfiguration()
> .setName("bla"),        .setMemoryMaxSize(1024), // must be greater than 0,
> since memory always needs to be enabled.        .setDiskMaxSize(0), // if
> greater than 0, then persistence is enabled.    );*



I think this approach is much more concise and straight forward. What do
you think?

D.

On Wed, Sep 20, 2017 at 4:55 AM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> I prefer the second. Composition over inheritance - this is how all our
> configuration is crafted.
>
> E.g. we do not have "CacheConfiguration" and "
> StoreEnabledCacheConfiguration".
> Instead, we have "CacheConfiguration.setCacheStoreFactory".
>
> On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Reiterating this based on some feedback from PDS users.
> >
> > It might be confusing to configure persistence with "MemoryPolicy", so
> > another approach is to deprecate the old names and introduce a new name
> > "DataRegion" because it reflects the actual state when data is stored on
> > disk and partially in memory. I have two options in mind, each of them
> > looks acceptable to me, so I would like to have some feedback from the
> > community. Old configuration names will be deprecated (but still be taken
> > if used for backward compatibility). Note, that old names deprecation
> > handles default configuration compatibility very nicely - current PDS
> users
> > will not need to change anything to keep everything working. The two
> > options I mentioned are below:
> >
> >  * we have two separate classes for in-memory and persisted data regions,
> > so the configuration would look like so:
> >
> > IgniteConfiguration cfg = new IgniteConfiguration();
> >
> > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> >     .setDataRegions(
> >         new MemoryDataRegion()
> >             .setName("volatileCaches")
> >             .setMaxMemorySize(...),
> >         new PersistentDataRegion()
> >             .setName("persistentCaches")
> >             .setMaxMemorySize(...)
> >             .setMaxDiskSize()));
> >
> > cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());
> >
> >
> > * we have one class for data region configuration, but it will have a
> > sub-bean for persistence configuration:
> >
> > IgniteConfiguration cfg = new IgniteConfiguration();
> >
> > cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
> >     .setDataRegions(
> >         new DataRegion()
> >             .setName("volatileCaches")
> >             .setMaxMemorySize(...),
> >         new DataRegion()
> >             .setName("persistentCaches")
> >             .setMaxMemorySize(...),
> >             .setPersistenceConfiguration(
> >                 new DataRegionPersistenceConfiguration()
> >                     .setMaxDiskSize(...))));
> >
> > cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

I prefer the second. Composition over inheritance - this is how all our
configuration is crafted.

E.g. we do not have "CacheConfiguration" and "StoreEnabledCacheConfiguration".
Instead, we have "CacheConfiguration.setCacheStoreFactory".

On Wed, Sep 20, 2017 at 2:46 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Reiterating this based on some feedback from PDS users.
>
> It might be confusing to configure persistence with "MemoryPolicy", so
> another approach is to deprecate the old names and introduce a new name
> "DataRegion" because it reflects the actual state when data is stored on
> disk and partially in memory. I have two options in mind, each of them
> looks acceptable to me, so I would like to have some feedback from the
> community. Old configuration names will be deprecated (but still be taken
> if used for backward compatibility). Note, that old names deprecation
> handles default configuration compatibility very nicely - current PDS users
> will not need to change anything to keep everything working. The two
> options I mentioned are below:
>
>  * we have two separate classes for in-memory and persisted data regions,
> so the configuration would look like so:
>
> IgniteConfiguration cfg = new IgniteConfiguration();
>
> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
>     .setDataRegions(
>         new MemoryDataRegion()
>             .setName("volatileCaches")
>             .setMaxMemorySize(...),
>         new PersistentDataRegion()
>             .setName("persistentCaches")
>             .setMaxMemorySize(...)
>             .setMaxDiskSize()));
>
> cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());
>
>
> * we have one class for data region configuration, but it will have a
> sub-bean for persistence configuration:
>
> IgniteConfiguration cfg = new IgniteConfiguration();
>
> cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
>     .setDataRegions(
>         new DataRegion()
>             .setName("volatileCaches")
>             .setMaxMemorySize(...),
>         new DataRegion()
>             .setName("persistentCaches")
>             .setMaxMemorySize(...),
>             .setPersistenceConfiguration(
>                 new DataRegionPersistenceConfiguration()
>                     .setMaxDiskSize(...))));
>
> cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Reiterating this based on some feedback from PDS users.

It might be confusing to configure persistence with "MemoryPolicy", so
another approach is to deprecate the old names and introduce a new name
"DataRegion" because it reflects the actual state when data is stored on
disk and partially in memory. I have two options in mind, each of them
looks acceptable to me, so I would like to have some feedback from the
community. Old configuration names will be deprecated (but still be taken
if used for backward compatibility). Note, that old names deprecation
handles default configuration compatibility very nicely - current PDS users
will not need to change anything to keep everything working. The two
options I mentioned are below:

 * we have two separate classes for in-memory and persisted data regions,
so the configuration would look like so:

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
    .setDataRegions(
        new MemoryDataRegion()
            .setName("volatileCaches")
            .setMaxMemorySize(...),
        new PersistentDataRegion()
            .setName("persistentCaches")
            .setMaxMemorySize(...)
            .setMaxDiskSize()));

cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());


* we have one class for data region configuration, but it will have a
sub-bean for persistence configuration:

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setDataRegionsConfiguration(new DataRegionsConfiguration()
    .setDataRegions(
        new DataRegion()
            .setName("volatileCaches")
            .setMaxMemorySize(...),
        new DataRegion()
            .setName("persistentCaches")
            .setMaxMemorySize(...),
            .setPersistenceConfiguration(
                new DataRegionPersistenceConfiguration()
                    .setMaxDiskSize(...))));

cfg.setPersistentStoreConfiguration(new PersistentStoreConfiguration());

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Vladimir,

The configuration would look like so:

IgniteConfiguration cfg = new IgniteConfiguration();

cfg.setMemoryConfiguration(new MemoryConfiguration()
    .setMemoryPolicies(
        new MemoryPolicyConfiguration().setName("InMemory").setMaxSize(...),
        new PersistentMemoryPolicyConfiguration().setName("Persisted").setMaxSize(...));

cfg.setCacheConfiguration(
    new CacheConfiguration("tmpCache").setMemoryPolicyName("InMemory"),
    new CacheConfiguration("dbCache").setMemoryPolicyName("Persisted"));



2017-09-14 12:20 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:

> >>Can we have separate MemoryPolycy for every CacheGroup? In that case we
> could auto generate MemoryPolycy based on CacheGroup settings and have a
> "cache level" persistence settings.
> >>We can use some kind of MemoryPolicyTemplate to add default values and
> even re-use existing MemoryPolicy if all settings are the same.
>
> Konstantin, can you please elaborate?
>
> --Yakov
>

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

>>Can we have separate MemoryPolycy for every CacheGroup? In that case we
could auto generate MemoryPolycy based on CacheGroup settings and have a
"cache level" persistence settings.
>>We can use some kind of MemoryPolicyTemplate to add default values and
even re-use existing MemoryPolicy if all settings are the same.

Konstantin, can you please elaborate?

--Yakov

Re: Persistence per memory policy configuration

[Konstantin Dudkov <kd...@ya.ru>]

Can we have separate MemoryPolycy for every CacheGroup? In that case we 
could auto generate MemoryPolycy based on CacheGroup settings and have a 
"cache level" persistence settings.

We can use some kind of MemoryPolicyTemplate to add default values and 
even re-use existing MemoryPolicy if all settings are the same.

> To extend on the idea of 2 different policies for memory and persistence,
> can we have 2 completely different configuration classes?
>
>   - MemoryPolicy
>   - PersistentMemoryPolicy (extends MemoryPolicy?)
>
> Every cache should be associated with either MemoryPolicy or
> PersistentMemoryPolicy, but not both. By default, caches on startup are
> associated with default MemoryPolicy. Users can always change it to some
> PersistentMemoryPolicy, if persistence needs to be enabled.
>
> If we agree on the above, do we really need a persistenceEnabled flag at
> all?
>
> D.
>
> On Tue, Sep 12, 2017 at 2:57 AM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> We surely can, but:
>>   * we should then have two configuration settings for default memory policy
>> size (one in-memory and one persisted)
>>   * a user still may configure multiple custom memory policies. In this
>> case, the requirement to have this flag the same in a memory policy is
>> still valid, so a user still can get exceptions.
>>
>> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
>>
>>> Alex,
>>>
>>> Can we have two default memory policies - one for in-memory and another
>> one
>>> for persistence cases?
>>>
>>> On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com> wrote:
>>>
>>>> This is possible, but then if two caches belong to the same memory
>>> policy,
>>>> they must be both either persistence-enabled or persistence-disabled.
>> We
>>>> can add this validation, but I think this will lead to a greater
>>> confusion
>>>> for a user.
>>>>
>>>> 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
>>>>
>>>>> Agree with Vladimir.
>>>>>
>>>>> Currently we enable persistence cluster-wide by setting
>>>>> IgniteConfiguration.persistentStoreConfiguration.
>>>>> Ideally CacheConfiguration.persistenceEnabled should be the only
>>>> setting I
>>>>> need to set.
>>>>>
>>>>> Thanks,
>>>>> Pavel
>>>>>
>>>>> On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
>>> vozerov@gridgain.com>
>>>>> wrote:
>>>>>
>>>>>> As a user I would definitely prefer to control persistence through
>>> flag
>>>>> on
>>>>>> cache configuration. I do not even want to know what "memory
>> policy"
>>>> is.
>>>>>> E.g. CacheConfiguration.persistenceEnabled.
>>>>>>
>>>>>> On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>
>>>>>>> Igniters,
>>>>>>>
>>>>>>> I am currently reviewing a change allowing to enable persistence
>>> on a
>>>>>>> per-memory-policy basis (thanks to K. Dudkov!) and have a
>> question
>>>>>>> regarding the changes in configuration.
>>>>>>>
>>>>>>> The suggested change is to add a flag "persistenceEnabled"
>>> (defaults
>>>> to
>>>>>>> true) to the memory policy configuration. To keep configuration
>>>>>>> compatibility, the logic is as follows:
>>>>>>>
>>>>>>> If PersistentStoreConfiguration is set, then only memory policies
>>>> with
>>>>>>> persistenceEnabled=true flag will be persisted, which is
>> consistent
>>>>> with
>>>>>>> the current behavior. To disable persistence, persistenceEnabled
>>> flag
>>>>>>> should be explicitly set to false.
>>>>>>>
>>>>>>> If PersistentStoreConfiguration is not set, then all caches are
>>>> stored
>>>>>>> in-memory and persistenceEnabled is ignored.
>>>>>>>
>>>>>>> While personally, I like this change, I would like to check if
>>> there
>>>>> are
>>>>>>> any thoughts or objections to this approach.
>>>>>>>
>>>>>>> --
>>>>>>> Thanks,
>>>>>>> AG
>>>>>>>

Re: Persistence per memory policy configuration

[Dmitriy Setrakyan <ds...@apache.org>]

To extend on the idea of 2 different policies for memory and persistence,
can we have 2 completely different configuration classes?

 - MemoryPolicy
 - PersistentMemoryPolicy (extends MemoryPolicy?)

Every cache should be associated with either MemoryPolicy or
PersistentMemoryPolicy, but not both. By default, caches on startup are
associated with default MemoryPolicy. Users can always change it to some
PersistentMemoryPolicy, if persistence needs to be enabled.

If we agree on the above, do we really need a persistenceEnabled flag at
all?

D.

On Tue, Sep 12, 2017 at 2:57 AM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> We surely can, but:
>  * we should then have two configuration settings for default memory policy
> size (one in-memory and one persisted)
>  * a user still may configure multiple custom memory policies. In this
> case, the requirement to have this flag the same in a memory policy is
> still valid, so a user still can get exceptions.
>
> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
>
> > Alex,
> >
> > Can we have two default memory policies - one for in-memory and another
> one
> > for persistence cases?
> >
> > On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > This is possible, but then if two caches belong to the same memory
> > policy,
> > > they must be both either persistence-enabled or persistence-disabled.
> We
> > > can add this validation, but I think this will lead to a greater
> > confusion
> > > for a user.
> > >
> > > 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> > >
> > > > Agree with Vladimir.
> > > >
> > > > Currently we enable persistence cluster-wide by setting
> > > > IgniteConfiguration.persistentStoreConfiguration.
> > > > Ideally CacheConfiguration.persistenceEnabled should be the only
> > > setting I
> > > > need to set.
> > > >
> > > > Thanks,
> > > > Pavel
> > > >
> > > > On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > As a user I would definitely prefer to control persistence through
> > flag
> > > > on
> > > > > cache configuration. I do not even want to know what "memory
> policy"
> > > is.
> > > > > E.g. CacheConfiguration.persistenceEnabled.
> > > > >
> > > > > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I am currently reviewing a change allowing to enable persistence
> > on a
> > > > > > per-memory-policy basis (thanks to K. Dudkov!) and have a
> question
> > > > > > regarding the changes in configuration.
> > > > > >
> > > > > > The suggested change is to add a flag "persistenceEnabled"
> > (defaults
> > > to
> > > > > > true) to the memory policy configuration. To keep configuration
> > > > > > compatibility, the logic is as follows:
> > > > > >
> > > > > > If PersistentStoreConfiguration is set, then only memory policies
> > > with
> > > > > > persistenceEnabled=true flag will be persisted, which is
> consistent
> > > > with
> > > > > > the current behavior. To disable persistence, persistenceEnabled
> > flag
> > > > > > should be explicitly set to false.
> > > > > >
> > > > > > If PersistentStoreConfiguration is not set, then all caches are
> > > stored
> > > > > > in-memory and persistenceEnabled is ignored.
> > > > > >
> > > > > > While personally, I like this change, I would like to check if
> > there
> > > > are
> > > > > > any thoughts or objections to this approach.
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > AG
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

Yakov,

The default=true will be used only if the PersistentStoreConfiguration is
set in the IgniteConfiguration, this is done only to maintain configuration
compatibility.

A user is allowed to use persisted and in-memory caches, he will be only
forced to put them into separate memory policies.

2017-09-12 13:39 GMT+03:00 Yakov Zhdanov <yz...@apache.org>:

> Agree that implicit actions are always source of issues of different kinds.
>
> Alex, having persistence enabled by default does not seem to be a good
> idea.
>
> >I think if we go with "persistence per cache" path, we must just enforce
> the
> correct configuration and let users configure the rest
>
> What do you mean by enforce? What if user mixes persisted and in-mem
> caches?
>
> --Yakov
>

Re: Persistence per memory policy configuration

[Yakov Zhdanov <yz...@apache.org>]

Agree that implicit actions are always source of issues of different kinds.

Alex, having persistence enabled by default does not seem to be a good idea.

>I think if we go with "persistence per cache" path, we must just enforce the
correct configuration and let users configure the rest

What do you mean by enforce? What if user mixes persisted and in-mem caches?

--Yakov

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

This may work. But I do not like any implicit memory policy cloning because
this will double the expected memory consumption and result in questions
"why does my Ignite take 2x more memory when I enable persistence" from
users.

I think if we go with "persistence per cache" path, we must just enforce
the correct configuration and let users configure the rest. Any implicit
'magic' will lead to confusion.

2017-09-12 13:10 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:

> Can we leave the default policy as is, but clone it with persistence
> enabled on demand?
>
> E.g:
> - user starts Ignite with default config
> - createCache without persistence - use default policy
> - createCache with persistence - clone default policy with enabled
> persistence and some predefined postfix
> - createCache without persistence - use default policy
> - createCache with persistence - reuse cloned policy
>
> etc etc.
>
> We can think of CacheConfiguration.persistenceEnabled as an override.
>
> On Tue, Sep 12, 2017 at 12:57 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > We surely can, but:
> >  * we should then have two configuration settings for default memory
> policy
> > size (one in-memory and one persisted)
> >  * a user still may configure multiple custom memory policies. In this
> > case, the requirement to have this flag the same in a memory policy is
> > still valid, so a user still can get exceptions.
> >
> > 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
> >
> > > Alex,
> > >
> > > Can we have two default memory policies - one for in-memory and another
> > one
> > > for persistence cases?
> > >
> > > On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > This is possible, but then if two caches belong to the same memory
> > > policy,
> > > > they must be both either persistence-enabled or persistence-disabled.
> > We
> > > > can add this validation, but I think this will lead to a greater
> > > confusion
> > > > for a user.
> > > >
> > > > 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> > > >
> > > > > Agree with Vladimir.
> > > > >
> > > > > Currently we enable persistence cluster-wide by setting
> > > > > IgniteConfiguration.persistentStoreConfiguration.
> > > > > Ideally CacheConfiguration.persistenceEnabled should be the only
> > > > setting I
> > > > > need to set.
> > > > >
> > > > > Thanks,
> > > > > Pavel
> > > > >
> > > > > On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> > > vozerov@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > As a user I would definitely prefer to control persistence
> through
> > > flag
> > > > > on
> > > > > > cache configuration. I do not even want to know what "memory
> > policy"
> > > > is.
> > > > > > E.g. CacheConfiguration.persistenceEnabled.
> > > > > >
> > > > > > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > > > > > alexey.goncharuk@gmail.com> wrote:
> > > > > >
> > > > > > > Igniters,
> > > > > > >
> > > > > > > I am currently reviewing a change allowing to enable
> persistence
> > > on a
> > > > > > > per-memory-policy basis (thanks to K. Dudkov!) and have a
> > question
> > > > > > > regarding the changes in configuration.
> > > > > > >
> > > > > > > The suggested change is to add a flag "persistenceEnabled"
> > > (defaults
> > > > to
> > > > > > > true) to the memory policy configuration. To keep configuration
> > > > > > > compatibility, the logic is as follows:
> > > > > > >
> > > > > > > If PersistentStoreConfiguration is set, then only memory
> policies
> > > > with
> > > > > > > persistenceEnabled=true flag will be persisted, which is
> > consistent
> > > > > with
> > > > > > > the current behavior. To disable persistence,
> persistenceEnabled
> > > flag
> > > > > > > should be explicitly set to false.
> > > > > > >
> > > > > > > If PersistentStoreConfiguration is not set, then all caches are
> > > > stored
> > > > > > > in-memory and persistenceEnabled is ignored.
> > > > > > >
> > > > > > > While personally, I like this change, I would like to check if
> > > there
> > > > > are
> > > > > > > any thoughts or objections to this approach.
> > > > > > >
> > > > > > > --
> > > > > > > Thanks,
> > > > > > > AG
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Pavel Tupitsyn <pt...@apache.org>]

Can we leave the default policy as is, but clone it with persistence
enabled on demand?

E.g:
- user starts Ignite with default config
- createCache without persistence - use default policy
- createCache with persistence - clone default policy with enabled
persistence and some predefined postfix
- createCache without persistence - use default policy
- createCache with persistence - reuse cloned policy

etc etc.

We can think of CacheConfiguration.persistenceEnabled as an override.

On Tue, Sep 12, 2017 at 12:57 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> We surely can, but:
>  * we should then have two configuration settings for default memory policy
> size (one in-memory and one persisted)
>  * a user still may configure multiple custom memory policies. In this
> case, the requirement to have this flag the same in a memory policy is
> still valid, so a user still can get exceptions.
>
> 2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:
>
> > Alex,
> >
> > Can we have two default memory policies - one for in-memory and another
> one
> > for persistence cases?
> >
> > On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > This is possible, but then if two caches belong to the same memory
> > policy,
> > > they must be both either persistence-enabled or persistence-disabled.
> We
> > > can add this validation, but I think this will lead to a greater
> > confusion
> > > for a user.
> > >
> > > 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> > >
> > > > Agree with Vladimir.
> > > >
> > > > Currently we enable persistence cluster-wide by setting
> > > > IgniteConfiguration.persistentStoreConfiguration.
> > > > Ideally CacheConfiguration.persistenceEnabled should be the only
> > > setting I
> > > > need to set.
> > > >
> > > > Thanks,
> > > > Pavel
> > > >
> > > > On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> > vozerov@gridgain.com>
> > > > wrote:
> > > >
> > > > > As a user I would definitely prefer to control persistence through
> > flag
> > > > on
> > > > > cache configuration. I do not even want to know what "memory
> policy"
> > > is.
> > > > > E.g. CacheConfiguration.persistenceEnabled.
> > > > >
> > > > > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > > > > alexey.goncharuk@gmail.com> wrote:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I am currently reviewing a change allowing to enable persistence
> > on a
> > > > > > per-memory-policy basis (thanks to K. Dudkov!) and have a
> question
> > > > > > regarding the changes in configuration.
> > > > > >
> > > > > > The suggested change is to add a flag "persistenceEnabled"
> > (defaults
> > > to
> > > > > > true) to the memory policy configuration. To keep configuration
> > > > > > compatibility, the logic is as follows:
> > > > > >
> > > > > > If PersistentStoreConfiguration is set, then only memory policies
> > > with
> > > > > > persistenceEnabled=true flag will be persisted, which is
> consistent
> > > > with
> > > > > > the current behavior. To disable persistence, persistenceEnabled
> > flag
> > > > > > should be explicitly set to false.
> > > > > >
> > > > > > If PersistentStoreConfiguration is not set, then all caches are
> > > stored
> > > > > > in-memory and persistenceEnabled is ignored.
> > > > > >
> > > > > > While personally, I like this change, I would like to check if
> > there
> > > > are
> > > > > > any thoughts or objections to this approach.
> > > > > >
> > > > > > --
> > > > > > Thanks,
> > > > > > AG
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

We surely can, but:
 * we should then have two configuration settings for default memory policy
size (one in-memory and one persisted)
 * a user still may configure multiple custom memory policies. In this
case, the requirement to have this flag the same in a memory policy is
still valid, so a user still can get exceptions.

2017-09-12 12:44 GMT+03:00 Vladimir Ozerov <vo...@gridgain.com>:

> Alex,
>
> Can we have two default memory policies - one for in-memory and another one
> for persistence cases?
>
> On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > This is possible, but then if two caches belong to the same memory
> policy,
> > they must be both either persistence-enabled or persistence-disabled. We
> > can add this validation, but I think this will lead to a greater
> confusion
> > for a user.
> >
> > 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
> >
> > > Agree with Vladimir.
> > >
> > > Currently we enable persistence cluster-wide by setting
> > > IgniteConfiguration.persistentStoreConfiguration.
> > > Ideally CacheConfiguration.persistenceEnabled should be the only
> > setting I
> > > need to set.
> > >
> > > Thanks,
> > > Pavel
> > >
> > > On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <
> vozerov@gridgain.com>
> > > wrote:
> > >
> > > > As a user I would definitely prefer to control persistence through
> flag
> > > on
> > > > cache configuration. I do not even want to know what "memory policy"
> > is.
> > > > E.g. CacheConfiguration.persistenceEnabled.
> > > >
> > > > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > > > alexey.goncharuk@gmail.com> wrote:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I am currently reviewing a change allowing to enable persistence
> on a
> > > > > per-memory-policy basis (thanks to K. Dudkov!) and have a question
> > > > > regarding the changes in configuration.
> > > > >
> > > > > The suggested change is to add a flag "persistenceEnabled"
> (defaults
> > to
> > > > > true) to the memory policy configuration. To keep configuration
> > > > > compatibility, the logic is as follows:
> > > > >
> > > > > If PersistentStoreConfiguration is set, then only memory policies
> > with
> > > > > persistenceEnabled=true flag will be persisted, which is consistent
> > > with
> > > > > the current behavior. To disable persistence, persistenceEnabled
> flag
> > > > > should be explicitly set to false.
> > > > >
> > > > > If PersistentStoreConfiguration is not set, then all caches are
> > stored
> > > > > in-memory and persistenceEnabled is ignored.
> > > > >
> > > > > While personally, I like this change, I would like to check if
> there
> > > are
> > > > > any thoughts or objections to this approach.
> > > > >
> > > > > --
> > > > > Thanks,
> > > > > AG
> > > > >
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

Alex,

Can we have two default memory policies - one for in-memory and another one
for persistence cases?

On Tue, Sep 12, 2017 at 12:40 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> This is possible, but then if two caches belong to the same memory policy,
> they must be both either persistence-enabled or persistence-disabled. We
> can add this validation, but I think this will lead to a greater confusion
> for a user.
>
> 2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:
>
> > Agree with Vladimir.
> >
> > Currently we enable persistence cluster-wide by setting
> > IgniteConfiguration.persistentStoreConfiguration.
> > Ideally CacheConfiguration.persistenceEnabled should be the only
> setting I
> > need to set.
> >
> > Thanks,
> > Pavel
> >
> > On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> > > As a user I would definitely prefer to control persistence through flag
> > on
> > > cache configuration. I do not even want to know what "memory policy"
> is.
> > > E.g. CacheConfiguration.persistenceEnabled.
> > >
> > > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com> wrote:
> > >
> > > > Igniters,
> > > >
> > > > I am currently reviewing a change allowing to enable persistence on a
> > > > per-memory-policy basis (thanks to K. Dudkov!) and have a question
> > > > regarding the changes in configuration.
> > > >
> > > > The suggested change is to add a flag "persistenceEnabled" (defaults
> to
> > > > true) to the memory policy configuration. To keep configuration
> > > > compatibility, the logic is as follows:
> > > >
> > > > If PersistentStoreConfiguration is set, then only memory policies
> with
> > > > persistenceEnabled=true flag will be persisted, which is consistent
> > with
> > > > the current behavior. To disable persistence, persistenceEnabled flag
> > > > should be explicitly set to false.
> > > >
> > > > If PersistentStoreConfiguration is not set, then all caches are
> stored
> > > > in-memory and persistenceEnabled is ignored.
> > > >
> > > > While personally, I like this change, I would like to check if there
> > are
> > > > any thoughts or objections to this approach.
> > > >
> > > > --
> > > > Thanks,
> > > > AG
> > > >
> > >
> >
>

Re: Persistence per memory policy configuration

[Alexey Goncharuk <al...@gmail.com>]

This is possible, but then if two caches belong to the same memory policy,
they must be both either persistence-enabled or persistence-disabled. We
can add this validation, but I think this will lead to a greater confusion
for a user.

2017-09-12 12:34 GMT+03:00 Pavel Tupitsyn <pt...@apache.org>:

> Agree with Vladimir.
>
> Currently we enable persistence cluster-wide by setting
> IgniteConfiguration.persistentStoreConfiguration.
> Ideally CacheConfiguration.persistenceEnabled should be the only setting I
> need to set.
>
> Thanks,
> Pavel
>
> On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
> > As a user I would definitely prefer to control persistence through flag
> on
> > cache configuration. I do not even want to know what "memory policy" is.
> > E.g. CacheConfiguration.persistenceEnabled.
> >
> > On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com> wrote:
> >
> > > Igniters,
> > >
> > > I am currently reviewing a change allowing to enable persistence on a
> > > per-memory-policy basis (thanks to K. Dudkov!) and have a question
> > > regarding the changes in configuration.
> > >
> > > The suggested change is to add a flag "persistenceEnabled" (defaults to
> > > true) to the memory policy configuration. To keep configuration
> > > compatibility, the logic is as follows:
> > >
> > > If PersistentStoreConfiguration is set, then only memory policies with
> > > persistenceEnabled=true flag will be persisted, which is consistent
> with
> > > the current behavior. To disable persistence, persistenceEnabled flag
> > > should be explicitly set to false.
> > >
> > > If PersistentStoreConfiguration is not set, then all caches are stored
> > > in-memory and persistenceEnabled is ignored.
> > >
> > > While personally, I like this change, I would like to check if there
> are
> > > any thoughts or objections to this approach.
> > >
> > > --
> > > Thanks,
> > > AG
> > >
> >
>

Re: Persistence per memory policy configuration

[Pavel Tupitsyn <pt...@apache.org>]

Agree with Vladimir.

Currently we enable persistence cluster-wide by setting
IgniteConfiguration.persistentStoreConfiguration.
Ideally CacheConfiguration.persistenceEnabled should be the only setting I
need to set.

Thanks,
Pavel

On Tue, Sep 12, 2017 at 12:28 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:

> As a user I would definitely prefer to control persistence through flag on
> cache configuration. I do not even want to know what "memory policy" is.
> E.g. CacheConfiguration.persistenceEnabled.
>
> On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
> > Igniters,
> >
> > I am currently reviewing a change allowing to enable persistence on a
> > per-memory-policy basis (thanks to K. Dudkov!) and have a question
> > regarding the changes in configuration.
> >
> > The suggested change is to add a flag "persistenceEnabled" (defaults to
> > true) to the memory policy configuration. To keep configuration
> > compatibility, the logic is as follows:
> >
> > If PersistentStoreConfiguration is set, then only memory policies with
> > persistenceEnabled=true flag will be persisted, which is consistent with
> > the current behavior. To disable persistence, persistenceEnabled flag
> > should be explicitly set to false.
> >
> > If PersistentStoreConfiguration is not set, then all caches are stored
> > in-memory and persistenceEnabled is ignored.
> >
> > While personally, I like this change, I would like to check if there are
> > any thoughts or objections to this approach.
> >
> > --
> > Thanks,
> > AG
> >
>

Re: Persistence per memory policy configuration

[Vladimir Ozerov <vo...@gridgain.com>]

As a user I would definitely prefer to control persistence through flag on
cache configuration. I do not even want to know what "memory policy" is.
E.g. CacheConfiguration.persistenceEnabled.

On Tue, Sep 12, 2017 at 12:24 PM, Alexey Goncharuk <
alexey.goncharuk@gmail.com> wrote:

> Igniters,
>
> I am currently reviewing a change allowing to enable persistence on a
> per-memory-policy basis (thanks to K. Dudkov!) and have a question
> regarding the changes in configuration.
>
> The suggested change is to add a flag "persistenceEnabled" (defaults to
> true) to the memory policy configuration. To keep configuration
> compatibility, the logic is as follows:
>
> If PersistentStoreConfiguration is set, then only memory policies with
> persistenceEnabled=true flag will be persisted, which is consistent with
> the current behavior. To disable persistence, persistenceEnabled flag
> should be explicitly set to false.
>
> If PersistentStoreConfiguration is not set, then all caches are stored
> in-memory and persistenceEnabled is ignored.
>
> While personally, I like this change, I would like to check if there are
> any thoughts or objections to this approach.
>
> --
> Thanks,
> AG
>