You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Dmitriy Pavlov <dp...@gmail.com> on 2018/08/01 17:43:48 UTC
Re: Adding experimental support for Intel Optane DC Persistent Memory
Hi,
This link works for me
https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
Sincerely,
Dmitriy Pavlov
чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <st...@gmail.com>:
> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s fine.
>
> From: Stanislav Lukyanov
> Sent: 26 июля 2018 г. 15:12
> To: dev@ignite.apache.org
> Subject: RE: Adding experimental support for Intel Optane DC Persistent
> Memory
>
> Hi,
>
> The link you’ve shared gives me 404.
> Perhaps you need to add a permission for everyone to access the page?
>
> Thanks,
> Stan
>
> From: Mammo, Mulugeta
> Sent: 26 июля 2018 г. 2:44
> To: dev@ignite.apache.org
> Subject: Adding experimental support for Intel Optane DC Persistent Memory
>
> Hi,
>
> I have added a new proposal to support Intel Optane DC Persistent Memory
> for Ignite here:
> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
> .
>
> I'm looking forward to your feedback and collaboration on this.
>
> Thanks,
> Mulugeta
>
>
>
>
Re: Adding experimental support for Intel Optane DC Persistent
Memory
Posted by "steve.hostettler@gmail.com" <st...@gmail.com>.
Dear all,
while I probably cannot help a lot with the implementation, I have access to
a machine with Optane memory. So if I can help in running benchmarks, let
me know.
Steve
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Hope that we will be better prepared for new iteration.
чт, 17 окт. 2019 г. в 02:29, Denis Magda <dm...@apache.org>:
>
> Ivan,
>
> There is no argument here, I see what you mean. All we know is that Optane
> is a new class of a storage device that is slower than RAM but faster than
> disk. Plus, it's denser than RAM, and non-volatile that let us store
> many-more terabytes of data on a single machine that can be accessed via
> the memory bus and don't worry about data preloading upon restarts. So,
> there definitely will be a benefit for certain use cases once Optane is
> supported natively - much more data can be stored and accessed locally at
> (almost) speed of RAM.
>
> The current integration is not finished yet (based on the review) but once
> we get hardware, agree, that the benchmarks can be executed even over what
> was developed so far.
>
>
> -
> Denis
>
>
> On Tue, Oct 15, 2019 at 10:44 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
>
> > Denis,
> >
> > I suppose we are using not optimal strategy here. And motivation is simple:
> > 1. Code review is "expensive", benchmarking (usually) are "cheap".
> > 2. Code review is "subjective", benchmarking are "objective".
> >
> > And my personal opinion here is that there could be a performance win
> > with Optane in proposed implementation in particular cases. It depends
> > on how much slower is Optane performance comparing to RAM. But we do
> > not know it.
> >
> > ср, 16 окт. 2019 г. в 01:10, Denis Magda <dm...@apache.org>:
> > >
> > > Alex, thanks for an extensive summary and proposing different
> > > implementation trails. Presently, the JEP-352 approach seems the most
> > > reasonable - JDK folks will take care of the integration and will be
> > > testing/maintaining the feature going forward, while the Ignite community
> > > will reuse what JDK provides. However, let's see what Eric and Mulugeta
> > of
> > > Intel (both copied) think about this and other approaches.
> > >
> > > Ivan, I'm not sure if any performance testing was done for the given
> > > pull-request. Anyway, it might be premature considering Alex's summary.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com>
> > wrote:
> > >
> > > > By the way, did we have a change to run some existing benchmarks
> > > > against proposed implementation?
> > > >
> > > > пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > > >:
> > > > >
> > > > > Hello Mulugeta, Igniters.
> > > > >
> > > > > Thanks for your interest and efforts in integrating the persistent
> > memory
> > > > > to Ignite. Here are my thoughts on the PR:
> > > > >
> > > > > First of all I was questioning whether we want to use the integration
> > > > with
> > > > > the pmem library via JNI. Java 14 will have native support for NVME
> > via
> > > > > native mapped byte buffers [1] since the corresponding tickets are
> > > > already
> > > > > resolved [2],[3]. The suggested integration uses JNI calls quite
> > often
> > > > (if
> > > > > I read the PR right, there will be tens of JNI calls per a single
> > > > > operation), which may and most likely will undermine the benefits of
> > > > using
> > > > > PM. On the other hand, there is a PoC/project [4] which successfully
> > > > > demonstrates how the new integration can be used to natively access
> > > > > persistent memory from Java. Additionally, new Unsafe have several
> > > > > performance issues - each put* methods has a map lookup, and CAS
> > method
> > > > > uses a global lock. Removing these issues will change the PR and
> > > > > architecture dramatically.
> > > > >
> > > > > Other than performance and an extra library dependency, there are
> > > > > additional reasons to use custom implementation of PM persistence
> > > > > primitives:
> > > > >
> > > > > - Ignite significantly benefits from having WAL as a single point
> > of
> > > > > data for both failure recovery and historical rebalance. As far
> > as I
> > > > can
> > > > > guess, the suggested library also uses some sort of journaling in
> > > > order to
> > > > > support crash recovery. Since we do not want give up on the
> > historical
> > > > > rebalance, we would need to write an additional journal, which
> > then
> > > > raises
> > > > > a question of how to transact between this journal.
> > > > > - Ignite is supposed to work with large indexes, a I not aware how
> > > > LLPL
> > > > > handles them. Several researches [5],[6] show that since PM
> > memory has
> > > > > larger access latency than regular RAM, it is beneficial to buffer
> > > > index
> > > > > pages in regular memory and store leaf pages in PM. There is a
> > generic
> > > > > approach to convert in-memory indexes to PM ones [7], I think it
> > makes
> > > > > sense to investigate whether our existing indexes can be
> > converted to
> > > > PM.
> > > > > - There is an active IEP which is targeted for a file-based
> > > > rebalancing.
> > > > > In the suggested implementation this optimization would not work.
> > A
> > > > better
> > > > > integration should allow to take a snapshot of a partition and
> > > > transfer it
> > > > > to another node for fast rebalancing.
> > > > >
> > > > > Overall, I think we may take the following path:
> > > > >
> > > > > - Target JEP352 APIs; for now keep using the whole region msync
> > for
> > > > flush
> > > > > - Introduce PM-based isolated components to support WAL iterator
> > and
> > > > > transaction management. We can start with WAL, then implement a
> > > > PM-based
> > > > > index based on current research, then a tuple heap with free space
> > > > > management. Each of the components may be developed and tested
> > > > independently
> > > > > - Do Ignite internals refactoring to properly abstract storage
> > layer
> > > > > from transactions/communication. Currently, there is an
> > abstraction
> > > > leak in
> > > > > the storage layer, which makes it aware of particular Ignite
> > > > implementation
> > > > > details (Binary Objects, partition eviction, etc)
> > > > > - Once the storage is separated, we should be able to plug the new
> > > > > components instead of the "classical" ones
> > > > >
> > > > > WDYT? I believe we should create a detailed IEP with concrete
> > proposals
> > > > on
> > > > > both PM structures design and Ignite internals changes before making
> > any
> > > > > further code changes.
> > > > >
> > > > > [1] https://openjdk.java.net/jeps/352
> > > > > [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> > > > > [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> > > > > [4] https://github.com/jhalliday/mashona
> > > > > [5]
> > https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> > > > > [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> > > > > [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> > > > > [8]
> > > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
> > > > >
> > > > > вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > > >:
> > > > >
> > > > > > Igniters,
> > > > > >
> > > > > > I would like to resurrect this discussion and will review the
> > change
> > > > again
> > > > > > shortly. If anyone want to join the review - you are welcome!
> > > > > >
> > > > > > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> > > > > >
> > > > > >> Hi Dmitry,
> > > > > >>
> > > > > >> That's a BSD-3-Clause license if to believe this statement
> > > > > >> "SPDX-License-Identifier: BSD-3-Clause":
> > > > > >> https://github.com/pmem/llpl/blob/master/LICENSE
> > > > > >>
> > > > > >> This license can be used with ASF software:
> > > > > >> https://www.apache.org/legal/resolved.html#category-a
> > > > > >>
> > > > > >> --
> > > > > >> Denis
> > > > > >>
> > > > > >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <
> > dpavlov.spb@gmail.com
> > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Denis,
> > > > > >> >
> > > > > >> > Could you please double check if we can refer to any library
> > > > licensed to
> > > > > >> > Intel. Can we develop code only version of this support (without
> > > > > >> shipping
> > > > > >> > it in release)?
> > > > > >> >
> > > > > >> > https://github.com/apache/ignite/pull/4381 is quite huge
> > change,
> > > > > >> > including 128 files changed, patch review will require resources
> > > > from
> > > > > >> > community members to review. I would like to be sure we can
> > include
> > > > this
> > > > > >> > patch from the legal point of view.
> > > > > >> >
> > > > > >> > Sincerely,
> > > > > >> > Dmitriy Pavlov
> > > > > >> >
> > > > > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <
> > dpavlov.spb@gmail.com>:
> > > > > >> >
> > > > > >> >> Hi Mulugeta,
> > > > > >> >>
> > > > > >> >> I appologise, I've missed that license is already there. So I
> > > > guess it
> > > > > >> is
> > > > > >> >> not standard open-source license, it is seems it is not listed
> > in
> > > > > >> >> https://www.apache.org/legal/resolved.html#category-a
> > > > > >> >>
> > > > > >> >> So there can be legal concern related to including this lib as
> > > > > >> dependency
> > > > > >> >> into Apache product. It should not block review, we can later
> > > > > >> >> consult Secretary/Legal to find out how we can correctly
> > include
> > > > > >> reference
> > > > > >> >> to lib.
> > > > > >> >>
> > > > > >> >> Sincerely,
> > > > > >> >> Dmitriy Pavlov
> > > > > >> >>
> > > > > >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <
> > > > mulugeta.mammo@intel.com>:
> > > > > >> >>
> > > > > >> >>> Hi Dmitriy,
> > > > > >> >>>
> > > > > >> >>> Do you mean our LLPL library? It has a license, please look
> > here:
> > > > > >> >>> https://github.com/pmem/llpl
> > > > > >> >>>
> > > > > >> >>> Regarding the changes made to Ignite, you may refer to the
> > pull
> > > > > >> request
> > > > > >> >>> here: https://github.com/apache/ignite/pull/4381
> > > > > >> >>>
> > > > > >> >>> Thanks,
> > > > > >> >>> Mulugeta
> > > > > >> >>>
> > > > > >> >>> -----Original Message-----
> > > > > >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> > > > > >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> > > > > >> >>> To: dev@ignite.apache.org
> > > > > >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> > > > > >> Persistent
> > > > > >> >>> Memory
> > > > > >> >>>
> > > > > >> >>> Hi Mulugeta Mammo,
> > > > > >> >>>
> > > > > >> >>> I've just noticed that repository, what you refer is full
> > fork of
> > > > > >> Ignite.
> > > > > >> >>> How can I see differences with original Ignite?
> > > > > >> >>>
> > > > > >> >>> One more thing, library which you're referencing seems to not
> > > > contain
> > > > > >> >>> license, at least github can not parse it. Apache product has
> > > > > >> limitations
> > > > > >> >>> which libraries may be used (see
> > > > > >> >>> https://www.apache.org/legal/resolved.html#category-a and
> > > > > >> >>> https://www.apache.org/legal/resolved.html#category-b)
> > > > > >> >>>
> > > > > >> >>> Could you please comment if there is some legal risk?
> > > > > >> >>>
> > > > > >> >>> Sincerely,
> > > > > >> >>> Dmitriy Pavlov
> > > > > >> >>>
> > > > > >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <
> > dpavlov.spb@gmail.com
> > > > >:
> > > > > >> >>>
> > > > > >> >>> > Hi,
> > > > > >> >>> >
> > > > > >> >>> > This link works for me
> > > > > >> >>> >
> > > > > >> >>> >
> > > > > >>
> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> > > > > >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > > > > >> >>> >
> > > > > >> >>> > Sincerely,
> > > > > >> >>> > Dmitriy Pavlov
> > > > > >> >>> >
> > > > > >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> > > > > >> >>> stanlukyanov@gmail.com>:
> > > > > >> >>> >
> > > > > >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed
> > it
> > > > and
> > > > > >> it’s
> > > > > >> >>> >> fine.
> > > > > >> >>> >>
> > > > > >> >>> >> From: Stanislav Lukyanov
> > > > > >> >>> >> Sent: 26 июля 2018 г. 15:12
> > > > > >> >>> >> To: dev@ignite.apache.org
> > > > > >> >>> >> Subject: RE: Adding experimental support for Intel Optane
> > DC
> > > > > >> >>> >> Persistent Memory
> > > > > >> >>> >>
> > > > > >> >>> >> Hi,
> > > > > >> >>> >>
> > > > > >> >>> >> The link you’ve shared gives me 404.
> > > > > >> >>> >> Perhaps you need to add a permission for everyone to
> > access the
> > > > > >> page?
> > > > > >> >>> >>
> > > > > >> >>> >> Thanks,
> > > > > >> >>> >> Stan
> > > > > >> >>> >>
> > > > > >> >>> >> From: Mammo, Mulugeta
> > > > > >> >>> >> Sent: 26 июля 2018 г. 2:44
> > > > > >> >>> >> To: dev@ignite.apache.org
> > > > > >> >>> >> Subject: Adding experimental support for Intel Optane DC
> > > > Persistent
> > > > > >> >>> >> Memory
> > > > > >> >>> >>
> > > > > >> >>> >> Hi,
> > > > > >> >>> >>
> > > > > >> >>> >> I have added a new proposal to support Intel Optane DC
> > > > Persistent
> > > > > >> >>> >> Memory for Ignite here:
> > > > > >> >>> >>
> > > > > >>
> > https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> > > > > >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> > > > > >> >>> >> .
> > > > > >> >>> >>
> > > > > >> >>> >> I'm looking forward to your feedback and collaboration on
> > this.
> > > > > >> >>> >>
> > > > > >> >>> >> Thanks,
> > > > > >> >>> >> Mulugeta
> > > > > >> >>> >>
> > > > > >> >>> >>
> > > > > >> >>> >>
> > > > > >> >>> >>
> > > > > >> >>>
> > > > > >> >>
> > > > > >>
> > > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Best regards,
> > > > Ivan Pavlukhin
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
--
Best regards,
Ivan Pavlukhin
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Denis Magda <dm...@apache.org>.
Ivan,
There is no argument here, I see what you mean. All we know is that Optane
is a new class of a storage device that is slower than RAM but faster than
disk. Plus, it's denser than RAM, and non-volatile that let us store
many-more terabytes of data on a single machine that can be accessed via
the memory bus and don't worry about data preloading upon restarts. So,
there definitely will be a benefit for certain use cases once Optane is
supported natively - much more data can be stored and accessed locally at
(almost) speed of RAM.
The current integration is not finished yet (based on the review) but once
we get hardware, agree, that the benchmarks can be executed even over what
was developed so far.
-
Denis
On Tue, Oct 15, 2019 at 10:44 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Denis,
>
> I suppose we are using not optimal strategy here. And motivation is simple:
> 1. Code review is "expensive", benchmarking (usually) are "cheap".
> 2. Code review is "subjective", benchmarking are "objective".
>
> And my personal opinion here is that there could be a performance win
> with Optane in proposed implementation in particular cases. It depends
> on how much slower is Optane performance comparing to RAM. But we do
> not know it.
>
> ср, 16 окт. 2019 г. в 01:10, Denis Magda <dm...@apache.org>:
> >
> > Alex, thanks for an extensive summary and proposing different
> > implementation trails. Presently, the JEP-352 approach seems the most
> > reasonable - JDK folks will take care of the integration and will be
> > testing/maintaining the feature going forward, while the Ignite community
> > will reuse what JDK provides. However, let's see what Eric and Mulugeta
> of
> > Intel (both copied) think about this and other approaches.
> >
> > Ivan, I'm not sure if any performance testing was done for the given
> > pull-request. Anyway, it might be premature considering Alex's summary.
> >
> > -
> > Denis
> >
> >
> > On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
> >
> > > By the way, did we have a change to run some existing benchmarks
> > > against proposed implementation?
> > >
> > > пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > > >:
> > > >
> > > > Hello Mulugeta, Igniters.
> > > >
> > > > Thanks for your interest and efforts in integrating the persistent
> memory
> > > > to Ignite. Here are my thoughts on the PR:
> > > >
> > > > First of all I was questioning whether we want to use the integration
> > > with
> > > > the pmem library via JNI. Java 14 will have native support for NVME
> via
> > > > native mapped byte buffers [1] since the corresponding tickets are
> > > already
> > > > resolved [2],[3]. The suggested integration uses JNI calls quite
> often
> > > (if
> > > > I read the PR right, there will be tens of JNI calls per a single
> > > > operation), which may and most likely will undermine the benefits of
> > > using
> > > > PM. On the other hand, there is a PoC/project [4] which successfully
> > > > demonstrates how the new integration can be used to natively access
> > > > persistent memory from Java. Additionally, new Unsafe have several
> > > > performance issues - each put* methods has a map lookup, and CAS
> method
> > > > uses a global lock. Removing these issues will change the PR and
> > > > architecture dramatically.
> > > >
> > > > Other than performance and an extra library dependency, there are
> > > > additional reasons to use custom implementation of PM persistence
> > > > primitives:
> > > >
> > > > - Ignite significantly benefits from having WAL as a single point
> of
> > > > data for both failure recovery and historical rebalance. As far
> as I
> > > can
> > > > guess, the suggested library also uses some sort of journaling in
> > > order to
> > > > support crash recovery. Since we do not want give up on the
> historical
> > > > rebalance, we would need to write an additional journal, which
> then
> > > raises
> > > > a question of how to transact between this journal.
> > > > - Ignite is supposed to work with large indexes, a I not aware how
> > > LLPL
> > > > handles them. Several researches [5],[6] show that since PM
> memory has
> > > > larger access latency than regular RAM, it is beneficial to buffer
> > > index
> > > > pages in regular memory and store leaf pages in PM. There is a
> generic
> > > > approach to convert in-memory indexes to PM ones [7], I think it
> makes
> > > > sense to investigate whether our existing indexes can be
> converted to
> > > PM.
> > > > - There is an active IEP which is targeted for a file-based
> > > rebalancing.
> > > > In the suggested implementation this optimization would not work.
> A
> > > better
> > > > integration should allow to take a snapshot of a partition and
> > > transfer it
> > > > to another node for fast rebalancing.
> > > >
> > > > Overall, I think we may take the following path:
> > > >
> > > > - Target JEP352 APIs; for now keep using the whole region msync
> for
> > > flush
> > > > - Introduce PM-based isolated components to support WAL iterator
> and
> > > > transaction management. We can start with WAL, then implement a
> > > PM-based
> > > > index based on current research, then a tuple heap with free space
> > > > management. Each of the components may be developed and tested
> > > independently
> > > > - Do Ignite internals refactoring to properly abstract storage
> layer
> > > > from transactions/communication. Currently, there is an
> abstraction
> > > leak in
> > > > the storage layer, which makes it aware of particular Ignite
> > > implementation
> > > > details (Binary Objects, partition eviction, etc)
> > > > - Once the storage is separated, we should be able to plug the new
> > > > components instead of the "classical" ones
> > > >
> > > > WDYT? I believe we should create a detailed IEP with concrete
> proposals
> > > on
> > > > both PM structures design and Ignite internals changes before making
> any
> > > > further code changes.
> > > >
> > > > [1] https://openjdk.java.net/jeps/352
> > > > [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> > > > [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> > > > [4] https://github.com/jhalliday/mashona
> > > > [5]
> https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> > > > [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> > > > [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> > > > [8]
> > > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
> > > >
> > > > вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > > >:
> > > >
> > > > > Igniters,
> > > > >
> > > > > I would like to resurrect this discussion and will review the
> change
> > > again
> > > > > shortly. If anyone want to join the review - you are welcome!
> > > > >
> > > > > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> > > > >
> > > > >> Hi Dmitry,
> > > > >>
> > > > >> That's a BSD-3-Clause license if to believe this statement
> > > > >> "SPDX-License-Identifier: BSD-3-Clause":
> > > > >> https://github.com/pmem/llpl/blob/master/LICENSE
> > > > >>
> > > > >> This license can be used with ASF software:
> > > > >> https://www.apache.org/legal/resolved.html#category-a
> > > > >>
> > > > >> --
> > > > >> Denis
> > > > >>
> > > > >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <
> dpavlov.spb@gmail.com
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Denis,
> > > > >> >
> > > > >> > Could you please double check if we can refer to any library
> > > licensed to
> > > > >> > Intel. Can we develop code only version of this support (without
> > > > >> shipping
> > > > >> > it in release)?
> > > > >> >
> > > > >> > https://github.com/apache/ignite/pull/4381 is quite huge
> change,
> > > > >> > including 128 files changed, patch review will require resources
> > > from
> > > > >> > community members to review. I would like to be sure we can
> include
> > > this
> > > > >> > patch from the legal point of view.
> > > > >> >
> > > > >> > Sincerely,
> > > > >> > Dmitriy Pavlov
> > > > >> >
> > > > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <
> dpavlov.spb@gmail.com>:
> > > > >> >
> > > > >> >> Hi Mulugeta,
> > > > >> >>
> > > > >> >> I appologise, I've missed that license is already there. So I
> > > guess it
> > > > >> is
> > > > >> >> not standard open-source license, it is seems it is not listed
> in
> > > > >> >> https://www.apache.org/legal/resolved.html#category-a
> > > > >> >>
> > > > >> >> So there can be legal concern related to including this lib as
> > > > >> dependency
> > > > >> >> into Apache product. It should not block review, we can later
> > > > >> >> consult Secretary/Legal to find out how we can correctly
> include
> > > > >> reference
> > > > >> >> to lib.
> > > > >> >>
> > > > >> >> Sincerely,
> > > > >> >> Dmitriy Pavlov
> > > > >> >>
> > > > >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <
> > > mulugeta.mammo@intel.com>:
> > > > >> >>
> > > > >> >>> Hi Dmitriy,
> > > > >> >>>
> > > > >> >>> Do you mean our LLPL library? It has a license, please look
> here:
> > > > >> >>> https://github.com/pmem/llpl
> > > > >> >>>
> > > > >> >>> Regarding the changes made to Ignite, you may refer to the
> pull
> > > > >> request
> > > > >> >>> here: https://github.com/apache/ignite/pull/4381
> > > > >> >>>
> > > > >> >>> Thanks,
> > > > >> >>> Mulugeta
> > > > >> >>>
> > > > >> >>> -----Original Message-----
> > > > >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> > > > >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> > > > >> >>> To: dev@ignite.apache.org
> > > > >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> > > > >> Persistent
> > > > >> >>> Memory
> > > > >> >>>
> > > > >> >>> Hi Mulugeta Mammo,
> > > > >> >>>
> > > > >> >>> I've just noticed that repository, what you refer is full
> fork of
> > > > >> Ignite.
> > > > >> >>> How can I see differences with original Ignite?
> > > > >> >>>
> > > > >> >>> One more thing, library which you're referencing seems to not
> > > contain
> > > > >> >>> license, at least github can not parse it. Apache product has
> > > > >> limitations
> > > > >> >>> which libraries may be used (see
> > > > >> >>> https://www.apache.org/legal/resolved.html#category-a and
> > > > >> >>> https://www.apache.org/legal/resolved.html#category-b)
> > > > >> >>>
> > > > >> >>> Could you please comment if there is some legal risk?
> > > > >> >>>
> > > > >> >>> Sincerely,
> > > > >> >>> Dmitriy Pavlov
> > > > >> >>>
> > > > >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <
> dpavlov.spb@gmail.com
> > > >:
> > > > >> >>>
> > > > >> >>> > Hi,
> > > > >> >>> >
> > > > >> >>> > This link works for me
> > > > >> >>> >
> > > > >> >>> >
> > > > >>
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> > > > >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > > > >> >>> >
> > > > >> >>> > Sincerely,
> > > > >> >>> > Dmitriy Pavlov
> > > > >> >>> >
> > > > >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> > > > >> >>> stanlukyanov@gmail.com>:
> > > > >> >>> >
> > > > >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed
> it
> > > and
> > > > >> it’s
> > > > >> >>> >> fine.
> > > > >> >>> >>
> > > > >> >>> >> From: Stanislav Lukyanov
> > > > >> >>> >> Sent: 26 июля 2018 г. 15:12
> > > > >> >>> >> To: dev@ignite.apache.org
> > > > >> >>> >> Subject: RE: Adding experimental support for Intel Optane
> DC
> > > > >> >>> >> Persistent Memory
> > > > >> >>> >>
> > > > >> >>> >> Hi,
> > > > >> >>> >>
> > > > >> >>> >> The link you’ve shared gives me 404.
> > > > >> >>> >> Perhaps you need to add a permission for everyone to
> access the
> > > > >> page?
> > > > >> >>> >>
> > > > >> >>> >> Thanks,
> > > > >> >>> >> Stan
> > > > >> >>> >>
> > > > >> >>> >> From: Mammo, Mulugeta
> > > > >> >>> >> Sent: 26 июля 2018 г. 2:44
> > > > >> >>> >> To: dev@ignite.apache.org
> > > > >> >>> >> Subject: Adding experimental support for Intel Optane DC
> > > Persistent
> > > > >> >>> >> Memory
> > > > >> >>> >>
> > > > >> >>> >> Hi,
> > > > >> >>> >>
> > > > >> >>> >> I have added a new proposal to support Intel Optane DC
> > > Persistent
> > > > >> >>> >> Memory for Ignite here:
> > > > >> >>> >>
> > > > >>
> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> > > > >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> > > > >> >>> >> .
> > > > >> >>> >>
> > > > >> >>> >> I'm looking forward to your feedback and collaboration on
> this.
> > > > >> >>> >>
> > > > >> >>> >> Thanks,
> > > > >> >>> >> Mulugeta
> > > > >> >>> >>
> > > > >> >>> >>
> > > > >> >>> >>
> > > > >> >>> >>
> > > > >> >>>
> > > > >> >>
> > > > >>
> > > > >
> > >
> > >
> > >
> > > --
> > > Best regards,
> > > Ivan Pavlukhin
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Denis,
I suppose we are using not optimal strategy here. And motivation is simple:
1. Code review is "expensive", benchmarking (usually) are "cheap".
2. Code review is "subjective", benchmarking are "objective".
And my personal opinion here is that there could be a performance win
with Optane in proposed implementation in particular cases. It depends
on how much slower is Optane performance comparing to RAM. But we do
not know it.
ср, 16 окт. 2019 г. в 01:10, Denis Magda <dm...@apache.org>:
>
> Alex, thanks for an extensive summary and proposing different
> implementation trails. Presently, the JEP-352 approach seems the most
> reasonable - JDK folks will take care of the integration and will be
> testing/maintaining the feature going forward, while the Ignite community
> will reuse what JDK provides. However, let's see what Eric and Mulugeta of
> Intel (both copied) think about this and other approaches.
>
> Ivan, I'm not sure if any performance testing was done for the given
> pull-request. Anyway, it might be premature considering Alex's summary.
>
> -
> Denis
>
>
> On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
>
> > By the way, did we have a change to run some existing benchmarks
> > against proposed implementation?
> >
> > пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <alexey.goncharuk@gmail.com
> > >:
> > >
> > > Hello Mulugeta, Igniters.
> > >
> > > Thanks for your interest and efforts in integrating the persistent memory
> > > to Ignite. Here are my thoughts on the PR:
> > >
> > > First of all I was questioning whether we want to use the integration
> > with
> > > the pmem library via JNI. Java 14 will have native support for NVME via
> > > native mapped byte buffers [1] since the corresponding tickets are
> > already
> > > resolved [2],[3]. The suggested integration uses JNI calls quite often
> > (if
> > > I read the PR right, there will be tens of JNI calls per a single
> > > operation), which may and most likely will undermine the benefits of
> > using
> > > PM. On the other hand, there is a PoC/project [4] which successfully
> > > demonstrates how the new integration can be used to natively access
> > > persistent memory from Java. Additionally, new Unsafe have several
> > > performance issues - each put* methods has a map lookup, and CAS method
> > > uses a global lock. Removing these issues will change the PR and
> > > architecture dramatically.
> > >
> > > Other than performance and an extra library dependency, there are
> > > additional reasons to use custom implementation of PM persistence
> > > primitives:
> > >
> > > - Ignite significantly benefits from having WAL as a single point of
> > > data for both failure recovery and historical rebalance. As far as I
> > can
> > > guess, the suggested library also uses some sort of journaling in
> > order to
> > > support crash recovery. Since we do not want give up on the historical
> > > rebalance, we would need to write an additional journal, which then
> > raises
> > > a question of how to transact between this journal.
> > > - Ignite is supposed to work with large indexes, a I not aware how
> > LLPL
> > > handles them. Several researches [5],[6] show that since PM memory has
> > > larger access latency than regular RAM, it is beneficial to buffer
> > index
> > > pages in regular memory and store leaf pages in PM. There is a generic
> > > approach to convert in-memory indexes to PM ones [7], I think it makes
> > > sense to investigate whether our existing indexes can be converted to
> > PM.
> > > - There is an active IEP which is targeted for a file-based
> > rebalancing.
> > > In the suggested implementation this optimization would not work. A
> > better
> > > integration should allow to take a snapshot of a partition and
> > transfer it
> > > to another node for fast rebalancing.
> > >
> > > Overall, I think we may take the following path:
> > >
> > > - Target JEP352 APIs; for now keep using the whole region msync for
> > flush
> > > - Introduce PM-based isolated components to support WAL iterator and
> > > transaction management. We can start with WAL, then implement a
> > PM-based
> > > index based on current research, then a tuple heap with free space
> > > management. Each of the components may be developed and tested
> > independently
> > > - Do Ignite internals refactoring to properly abstract storage layer
> > > from transactions/communication. Currently, there is an abstraction
> > leak in
> > > the storage layer, which makes it aware of particular Ignite
> > implementation
> > > details (Binary Objects, partition eviction, etc)
> > > - Once the storage is separated, we should be able to plug the new
> > > components instead of the "classical" ones
> > >
> > > WDYT? I believe we should create a detailed IEP with concrete proposals
> > on
> > > both PM structures design and Ignite internals changes before making any
> > > further code changes.
> > >
> > > [1] https://openjdk.java.net/jeps/352
> > > [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> > > [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> > > [4] https://github.com/jhalliday/mashona
> > > [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> > > [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> > > [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> > > [8]
> > >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
> > >
> > > вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <alexey.goncharuk@gmail.com
> > >:
> > >
> > > > Igniters,
> > > >
> > > > I would like to resurrect this discussion and will review the change
> > again
> > > > shortly. If anyone want to join the review - you are welcome!
> > > >
> > > > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> > > >
> > > >> Hi Dmitry,
> > > >>
> > > >> That's a BSD-3-Clause license if to believe this statement
> > > >> "SPDX-License-Identifier: BSD-3-Clause":
> > > >> https://github.com/pmem/llpl/blob/master/LICENSE
> > > >>
> > > >> This license can be used with ASF software:
> > > >> https://www.apache.org/legal/resolved.html#category-a
> > > >>
> > > >> --
> > > >> Denis
> > > >>
> > > >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dpavlov.spb@gmail.com
> > >
> > > >> wrote:
> > > >>
> > > >> > Hi Denis,
> > > >> >
> > > >> > Could you please double check if we can refer to any library
> > licensed to
> > > >> > Intel. Can we develop code only version of this support (without
> > > >> shipping
> > > >> > it in release)?
> > > >> >
> > > >> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> > > >> > including 128 files changed, patch review will require resources
> > from
> > > >> > community members to review. I would like to be sure we can include
> > this
> > > >> > patch from the legal point of view.
> > > >> >
> > > >> > Sincerely,
> > > >> > Dmitriy Pavlov
> > > >> >
> > > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> > > >> >
> > > >> >> Hi Mulugeta,
> > > >> >>
> > > >> >> I appologise, I've missed that license is already there. So I
> > guess it
> > > >> is
> > > >> >> not standard open-source license, it is seems it is not listed in
> > > >> >> https://www.apache.org/legal/resolved.html#category-a
> > > >> >>
> > > >> >> So there can be legal concern related to including this lib as
> > > >> dependency
> > > >> >> into Apache product. It should not block review, we can later
> > > >> >> consult Secretary/Legal to find out how we can correctly include
> > > >> reference
> > > >> >> to lib.
> > > >> >>
> > > >> >> Sincerely,
> > > >> >> Dmitriy Pavlov
> > > >> >>
> > > >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <
> > mulugeta.mammo@intel.com>:
> > > >> >>
> > > >> >>> Hi Dmitriy,
> > > >> >>>
> > > >> >>> Do you mean our LLPL library? It has a license, please look here:
> > > >> >>> https://github.com/pmem/llpl
> > > >> >>>
> > > >> >>> Regarding the changes made to Ignite, you may refer to the pull
> > > >> request
> > > >> >>> here: https://github.com/apache/ignite/pull/4381
> > > >> >>>
> > > >> >>> Thanks,
> > > >> >>> Mulugeta
> > > >> >>>
> > > >> >>> -----Original Message-----
> > > >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> > > >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> > > >> >>> To: dev@ignite.apache.org
> > > >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> > > >> Persistent
> > > >> >>> Memory
> > > >> >>>
> > > >> >>> Hi Mulugeta Mammo,
> > > >> >>>
> > > >> >>> I've just noticed that repository, what you refer is full fork of
> > > >> Ignite.
> > > >> >>> How can I see differences with original Ignite?
> > > >> >>>
> > > >> >>> One more thing, library which you're referencing seems to not
> > contain
> > > >> >>> license, at least github can not parse it. Apache product has
> > > >> limitations
> > > >> >>> which libraries may be used (see
> > > >> >>> https://www.apache.org/legal/resolved.html#category-a and
> > > >> >>> https://www.apache.org/legal/resolved.html#category-b)
> > > >> >>>
> > > >> >>> Could you please comment if there is some legal risk?
> > > >> >>>
> > > >> >>> Sincerely,
> > > >> >>> Dmitriy Pavlov
> > > >> >>>
> > > >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dpavlov.spb@gmail.com
> > >:
> > > >> >>>
> > > >> >>> > Hi,
> > > >> >>> >
> > > >> >>> > This link works for me
> > > >> >>> >
> > > >> >>> >
> > > >>
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> > > >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > > >> >>> >
> > > >> >>> > Sincerely,
> > > >> >>> > Dmitriy Pavlov
> > > >> >>> >
> > > >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> > > >> >>> stanlukyanov@gmail.com>:
> > > >> >>> >
> > > >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it
> > and
> > > >> it’s
> > > >> >>> >> fine.
> > > >> >>> >>
> > > >> >>> >> From: Stanislav Lukyanov
> > > >> >>> >> Sent: 26 июля 2018 г. 15:12
> > > >> >>> >> To: dev@ignite.apache.org
> > > >> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> > > >> >>> >> Persistent Memory
> > > >> >>> >>
> > > >> >>> >> Hi,
> > > >> >>> >>
> > > >> >>> >> The link you’ve shared gives me 404.
> > > >> >>> >> Perhaps you need to add a permission for everyone to access the
> > > >> page?
> > > >> >>> >>
> > > >> >>> >> Thanks,
> > > >> >>> >> Stan
> > > >> >>> >>
> > > >> >>> >> From: Mammo, Mulugeta
> > > >> >>> >> Sent: 26 июля 2018 г. 2:44
> > > >> >>> >> To: dev@ignite.apache.org
> > > >> >>> >> Subject: Adding experimental support for Intel Optane DC
> > Persistent
> > > >> >>> >> Memory
> > > >> >>> >>
> > > >> >>> >> Hi,
> > > >> >>> >>
> > > >> >>> >> I have added a new proposal to support Intel Optane DC
> > Persistent
> > > >> >>> >> Memory for Ignite here:
> > > >> >>> >>
> > > >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> > > >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> > > >> >>> >> .
> > > >> >>> >>
> > > >> >>> >> I'm looking forward to your feedback and collaboration on this.
> > > >> >>> >>
> > > >> >>> >> Thanks,
> > > >> >>> >> Mulugeta
> > > >> >>> >>
> > > >> >>> >>
> > > >> >>> >>
> > > >> >>> >>
> > > >> >>>
> > > >> >>
> > > >>
> > > >
> >
> >
> >
> > --
> > Best regards,
> > Ivan Pavlukhin
> >
--
Best regards,
Ivan Pavlukhin
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Denis Magda <dm...@apache.org>.
Hi Steve, thanks for joining the discussion and sharing your expertise
knowledge! The JDK14 dependency is not a matter of our concern.
Steve, Erick, is there anybody from the Intel engineering who can join the
community and take part in the development? Ignite committers will help
that person(s) gain all required knowledge about Ignite internals
throughout the journey. We definitely need your help with a B+tree
implementation for Optane.
Alex Goncharuk, would you mind accepting the role of the architect for this
integration? We’ll form a team of Ignite contributors who will take part in
the development under your architectural leadership. You can count on me,
will be the first one in that group )
Igniters, is there anybody else who’s ready to play with the new exciting
storage technology developed by Intel, gain advance knowledge of Ignite
memory management and other internals? Who’s ready to contribute as a
developer?
Denis
On Thursday, October 17, 2019, Dohrmann, Steve <st...@intel.com>
wrote:
> Hi Denis,
>
> Having read through the alternate proposal that would use JEP-352, it all
> sounds very plausible and may be a great fit in this case, assuming the
> dependency on JDK14 is ok. We are familiar with the persistent ByteBuffer
> JEP and think it is a great idea.
>
> Best regards,
> Steve
>
> On Oct 15, 2019, at 3:43 PM, Kaczmarek, Eric <er...@intel.com>
> wrote:
>
> Steve,
>
> Need your help to respond to all of this.
>
> *From: *Denis Magda <dm...@apache.org>
> *Date: *Tuesday, October 15, 2019 at 3:10 PM
> *To: *dev <de...@ignite.apache.org>, "Mammo, Mulugeta" <
> mulugeta.mammo@intel.com>, "Kaczmarek, Eric" <er...@intel.com>
> *Subject: *Re: Adding experimental support for Intel Optane DC Persistent
> Memory
>
> Alex, thanks for an extensive summary and proposing different
> implementation trails. Presently, the JEP-352 approach seems the most
> reasonable - JDK folks will take care of the integration and will be
> testing/maintaining the feature going forward, while the Ignite community
> will reuse what JDK provides. However, let's see what Eric and Mulugeta of
> Intel (both copied) think about this and other approaches.
>
> Ivan, I'm not sure if any performance testing was done for the given
> pull-request. Anyway, it might be premature considering Alex's summary.
>
> -
> Denis
>
>
> On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
> By the way, did we have a change to run some existing benchmarks
> against proposed implementation?
>
> пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > Hello Mulugeta, Igniters.
> >
> > Thanks for your interest and efforts in integrating the persistent memory
> > to Ignite. Here are my thoughts on the PR:
> >
> > First of all I was questioning whether we want to use the integration
> with
> > the pmem library via JNI. Java 14 will have native support for NVME via
> > native mapped byte buffers [1] since the corresponding tickets are
> already
> > resolved [2],[3]. The suggested integration uses JNI calls quite often
> (if
> > I read the PR right, there will be tens of JNI calls per a single
> > operation), which may and most likely will undermine the benefits of
> using
> > PM. On the other hand, there is a PoC/project [4] which successfully
> > demonstrates how the new integration can be used to natively access
> > persistent memory from Java. Additionally, new Unsafe have several
> > performance issues - each put* methods has a map lookup, and CAS method
> > uses a global lock. Removing these issues will change the PR and
> > architecture dramatically.
> >
> > Other than performance and an extra library dependency, there are
> > additional reasons to use custom implementation of PM persistence
> > primitives:
> >
> > - Ignite significantly benefits from having WAL as a single point of
> > data for both failure recovery and historical rebalance. As far as I
> can
> > guess, the suggested library also uses some sort of journaling in
> order to
> > support crash recovery. Since we do not want give up on the historical
> > rebalance, we would need to write an additional journal, which then
> raises
> > a question of how to transact between this journal.
> > - Ignite is supposed to work with large indexes, a I not aware how
> LLPL
> > handles them. Several researches [5],[6] show that since PM memory has
> > larger access latency than regular RAM, it is beneficial to buffer
> index
> > pages in regular memory and store leaf pages in PM. There is a generic
> > approach to convert in-memory indexes to PM ones [7], I think it makes
> > sense to investigate whether our existing indexes can be converted to
> PM.
> > - There is an active IEP which is targeted for a file-based
> rebalancing.
> > In the suggested implementation this optimization would not work. A
> better
> > integration should allow to take a snapshot of a partition and
> transfer it
> > to another node for fast rebalancing.
> >
> > Overall, I think we may take the following path:
> >
> > - Target JEP352 APIs; for now keep using the whole region msync for
> flush
> > - Introduce PM-based isolated components to support WAL iterator and
> > transaction management. We can start with WAL, then implement a
> PM-based
> > index based on current research, then a tuple heap with free space
> > management. Each of the components may be developed and tested
> independently
> > - Do Ignite internals refactoring to properly abstract storage layer
> > from transactions/communication. Currently, there is an abstraction
> leak in
> > the storage layer, which makes it aware of particular Ignite
> implementation
> > details (Binary Objects, partition eviction, etc)
> > - Once the storage is separated, we should be able to plug the new
> > components instead of the "classical" ones
> >
> > WDYT? I believe we should create a detailed IEP with concrete proposals
> on
> > both PM structures design and Ignite internals changes before making any
> > further code changes.
> >
> > [1] https://openjdk.java.net/jeps/352
> > [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> > [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> > [4] https://github.com/jhalliday/mashona
> > [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> > [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> > [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> > [8]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 16%3A+Optimization+of+rebalancing
> >
> > вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > > Igniters,
> > >
> > > I would like to resurrect this discussion and will review the change
> again
> > > shortly. If anyone want to join the review - you are welcome!
> > >
> > > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> > >
> > >> Hi Dmitry,
> > >>
> > >> That's a BSD-3-Clause license if to believe this statement
> > >> "SPDX-License-Identifier: BSD-3-Clause":
> > >> https://github.com/pmem/llpl/blob/master/LICENSE
> > >>
> > >> This license can be used with ASF software:
> > >> https://www.apache.org/legal/resolved.html#category-a
> > >>
> > >> --
> > >> Denis
> > >>
> > >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dpavlov.spb@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi Denis,
> > >> >
> > >> > Could you please double check if we can refer to any library
> licensed to
> > >> > Intel. Can we develop code only version of this support (without
> > >> shipping
> > >> > it in release)?
> > >> >
> > >> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> > >> > including 128 files changed, patch review will require resources
> from
> > >> > community members to review. I would like to be sure we can include
> this
> > >> > patch from the legal point of view.
> > >> >
> > >> > Sincerely,
> > >> > Dmitriy Pavlov
> > >> >
> > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> > >> >
> > >> >> Hi Mulugeta,
> > >> >>
> > >> >> I appologise, I've missed that license is already there. So I
> guess it
> > >> is
> > >> >> not standard open-source license, it is seems it is not listed in
> > >> >> https://www.apache.org/legal/resolved.html#category-a
> > >> >>
> > >> >> So there can be legal concern related to including this lib as
> > >> dependency
> > >> >> into Apache product. It should not block review, we can later
> > >> >> consult Secretary/Legal to find out how we can correctly include
> > >> reference
> > >> >> to lib.
> > >> >>
> > >> >> Sincerely,
> > >> >> Dmitriy Pavlov
> > >> >>
> > >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <
> mulugeta.mammo@intel.com>:
> > >> >>
> > >> >>> Hi Dmitriy,
> > >> >>>
> > >> >>> Do you mean our LLPL library? It has a license, please look here:
> > >> >>> https://github.com/pmem/llpl
> > >> >>>
> > >> >>> Regarding the changes made to Ignite, you may refer to the pull
> > >> request
> > >> >>> here: https://github.com/apache/ignite/pull/4381
> > >> >>>
> > >> >>> Thanks,
> > >> >>> Mulugeta
> > >> >>>
> > >> >>> -----Original Message-----
> > >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> > >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> > >> >>> To: dev@ignite.apache.org
> > >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> > >> Persistent
> > >> >>> Memory
> > >> >>>
> > >> >>> Hi Mulugeta Mammo,
> > >> >>>
> > >> >>> I've just noticed that repository, what you refer is full fork of
> > >> Ignite.
> > >> >>> How can I see differences with original Ignite?
> > >> >>>
> > >> >>> One more thing, library which you're referencing seems to not
> contain
> > >> >>> license, at least github can not parse it. Apache product has
> > >> limitations
> > >> >>> which libraries may be used (see
> > >> >>> https://www.apache.org/legal/resolved.html#category-a and
> > >> >>> https://www.apache.org/legal/resolved.html#category-b)
> > >> >>>
> > >> >>> Could you please comment if there is some legal risk?
> > >> >>>
> > >> >>> Sincerely,
> > >> >>> Dmitriy Pavlov
> > >> >>>
> > >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dpavlov.spb@gmail.com
> >:
> > >> >>>
> > >> >>> > Hi,
> > >> >>> >
> > >> >>> > This link works for me
> > >> >>> >
> > >> >>> >
> > >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 26%3A+Adding+Ex
> > >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > >> >>> >
> > >> >>> > Sincerely,
> > >> >>> > Dmitriy Pavlov
> > >> >>> >
> > >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> > >> >>> stanlukyanov@gmail.com>:
> > >> >>> >
> > >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it
> and
> > >> it’s
> > >> >>> >> fine.
> > >> >>> >>
> > >> >>> >> From: Stanislav Lukyanov
> > >> >>> >> Sent: 26 июля 2018 г. 15:12
> > >> >>> >> To: dev@ignite.apache.org
> > >> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> > >> >>> >> Persistent Memory
> > >> >>> >>
> > >> >>> >> Hi,
> > >> >>> >>
> > >> >>> >> The link you’ve shared gives me 404.
> > >> >>> >> Perhaps you need to add a permission for everyone to access the
> > >> page?
> > >> >>> >>
> > >> >>> >> Thanks,
> > >> >>> >> Stan
> > >> >>> >>
> > >> >>> >> From: Mammo, Mulugeta
> > >> >>> >> Sent: 26 июля 2018 г. 2:44
> > >> >>> >> To: dev@ignite.apache.org
> > >> >>> >> Subject: Adding experimental support for Intel Optane DC
> Persistent
> > >> >>> >> Memory
> > >> >>> >>
> > >> >>> >> Hi,
> > >> >>> >>
> > >> >>> >> I have added a new proposal to support Intel Optane DC
> Persistent
> > >> >>> >> Memory for Ignite here:
> > >> >>> >>
> > >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> > >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> > >> >>> >> .
> > >> >>> >>
> > >> >>> >> I'm looking forward to your feedback and collaboration on this.
> > >> >>> >>
> > >> >>> >> Thanks,
> > >> >>> >> Mulugeta
> > >> >>> >>
> > >> >>> >>
> > >> >>> >>
> > >> >>> >>
> > >> >>>
> > >> >>
> > >>
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
>
>
--
-
Denis
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by "Dohrmann, Steve" <st...@intel.com>.
Hi Denis,
Having read through the alternate proposal that would use JEP-352, it all sounds very plausible and may be a great fit in this case, assuming the dependency on JDK14 is ok. We are familiar with the persistent ByteBuffer JEP and think it is a great idea.
Best regards,
Steve
On Oct 15, 2019, at 3:43 PM, Kaczmarek, Eric <er...@intel.com>> wrote:
Steve,
Need your help to respond to all of this.
From: Denis Magda <dm...@apache.org>>
Date: Tuesday, October 15, 2019 at 3:10 PM
To: dev <de...@ignite.apache.org>>, "Mammo, Mulugeta" <mu...@intel.com>>, "Kaczmarek, Eric" <er...@intel.com>>
Subject: Re: Adding experimental support for Intel Optane DC Persistent Memory
Alex, thanks for an extensive summary and proposing different implementation trails. Presently, the JEP-352 approach seems the most reasonable - JDK folks will take care of the integration and will be testing/maintaining the feature going forward, while the Ignite community will reuse what JDK provides. However, let's see what Eric and Mulugeta of Intel (both copied) think about this and other approaches.
Ivan, I'm not sure if any performance testing was done for the given pull-request. Anyway, it might be premature considering Alex's summary.
-
Denis
On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com>> wrote:
By the way, did we have a change to run some existing benchmarks
against proposed implementation?
пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <al...@gmail.com>>:
>
> Hello Mulugeta, Igniters.
>
> Thanks for your interest and efforts in integrating the persistent memory
> to Ignite. Here are my thoughts on the PR:
>
> First of all I was questioning whether we want to use the integration with
> the pmem library via JNI. Java 14 will have native support for NVME via
> native mapped byte buffers [1] since the corresponding tickets are already
> resolved [2],[3]. The suggested integration uses JNI calls quite often (if
> I read the PR right, there will be tens of JNI calls per a single
> operation), which may and most likely will undermine the benefits of using
> PM. On the other hand, there is a PoC/project [4] which successfully
> demonstrates how the new integration can be used to natively access
> persistent memory from Java. Additionally, new Unsafe have several
> performance issues - each put* methods has a map lookup, and CAS method
> uses a global lock. Removing these issues will change the PR and
> architecture dramatically.
>
> Other than performance and an extra library dependency, there are
> additional reasons to use custom implementation of PM persistence
> primitives:
>
> - Ignite significantly benefits from having WAL as a single point of
> data for both failure recovery and historical rebalance. As far as I can
> guess, the suggested library also uses some sort of journaling in order to
> support crash recovery. Since we do not want give up on the historical
> rebalance, we would need to write an additional journal, which then raises
> a question of how to transact between this journal.
> - Ignite is supposed to work with large indexes, a I not aware how LLPL
> handles them. Several researches [5],[6] show that since PM memory has
> larger access latency than regular RAM, it is beneficial to buffer index
> pages in regular memory and store leaf pages in PM. There is a generic
> approach to convert in-memory indexes to PM ones [7], I think it makes
> sense to investigate whether our existing indexes can be converted to PM.
> - There is an active IEP which is targeted for a file-based rebalancing.
> In the suggested implementation this optimization would not work. A better
> integration should allow to take a snapshot of a partition and transfer it
> to another node for fast rebalancing.
>
> Overall, I think we may take the following path:
>
> - Target JEP352 APIs; for now keep using the whole region msync for flush
> - Introduce PM-based isolated components to support WAL iterator and
> transaction management. We can start with WAL, then implement a PM-based
> index based on current research, then a tuple heap with free space
> management. Each of the components may be developed and tested independently
> - Do Ignite internals refactoring to properly abstract storage layer
> from transactions/communication. Currently, there is an abstraction leak in
> the storage layer, which makes it aware of particular Ignite implementation
> details (Binary Objects, partition eviction, etc)
> - Once the storage is separated, we should be able to plug the new
> components instead of the "classical" ones
>
> WDYT? I believe we should create a detailed IEP with concrete proposals on
> both PM structures design and Ignite internals changes before making any
> further code changes.
>
> [1] https://openjdk.java.net/jeps/352
> [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> [4] https://github.com/jhalliday/mashona
> [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> [8]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
>
> вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <al...@gmail.com>>:
>
> > Igniters,
> >
> > I would like to resurrect this discussion and will review the change again
> > shortly. If anyone want to join the review - you are welcome!
> >
> > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>>:
> >
> >> Hi Dmitry,
> >>
> >> That's a BSD-3-Clause license if to believe this statement
> >> "SPDX-License-Identifier: BSD-3-Clause":
> >> https://github.com/pmem/llpl/blob/master/LICENSE
> >>
> >> This license can be used with ASF software:
> >> https://www.apache.org/legal/resolved.html#category-a
> >>
> >> --
> >> Denis
> >>
> >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dp...@gmail.com>>
> >> wrote:
> >>
> >> > Hi Denis,
> >> >
> >> > Could you please double check if we can refer to any library licensed to
> >> > Intel. Can we develop code only version of this support (without
> >> shipping
> >> > it in release)?
> >> >
> >> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> >> > including 128 files changed, patch review will require resources from
> >> > community members to review. I would like to be sure we can include this
> >> > patch from the legal point of view.
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>>:
> >> >
> >> >> Hi Mulugeta,
> >> >>
> >> >> I appologise, I've missed that license is already there. So I guess it
> >> is
> >> >> not standard open-source license, it is seems it is not listed in
> >> >> https://www.apache.org/legal/resolved.html#category-a
> >> >>
> >> >> So there can be legal concern related to including this lib as
> >> dependency
> >> >> into Apache product. It should not block review, we can later
> >> >> consult Secretary/Legal to find out how we can correctly include
> >> reference
> >> >> to lib.
> >> >>
> >> >> Sincerely,
> >> >> Dmitriy Pavlov
> >> >>
> >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>>:
> >> >>
> >> >>> Hi Dmitriy,
> >> >>>
> >> >>> Do you mean our LLPL library? It has a license, please look here:
> >> >>> https://github.com/pmem/llpl
> >> >>>
> >> >>> Regarding the changes made to Ignite, you may refer to the pull
> >> request
> >> >>> here: https://github.com/apache/ignite/pull/4381
> >> >>>
> >> >>> Thanks,
> >> >>> Mulugeta
> >> >>>
> >> >>> -----Original Message-----
> >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com<ma...@gmail.com>]
> >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> >> >>> To: dev@ignite.apache.org<ma...@ignite.apache.org>
> >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> >> Persistent
> >> >>> Memory
> >> >>>
> >> >>> Hi Mulugeta Mammo,
> >> >>>
> >> >>> I've just noticed that repository, what you refer is full fork of
> >> Ignite.
> >> >>> How can I see differences with original Ignite?
> >> >>>
> >> >>> One more thing, library which you're referencing seems to not contain
> >> >>> license, at least github can not parse it. Apache product has
> >> limitations
> >> >>> which libraries may be used (see
> >> >>> https://www.apache.org/legal/resolved.html#category-a and
> >> >>> https://www.apache.org/legal/resolved.html#category-b)
> >> >>>
> >> >>> Could you please comment if there is some legal risk?
> >> >>>
> >> >>> Sincerely,
> >> >>> Dmitriy Pavlov
> >> >>>
> >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>>:
> >> >>>
> >> >>> > Hi,
> >> >>> >
> >> >>> > This link works for me
> >> >>> >
> >> >>> >
> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >> >>> >
> >> >>> > Sincerely,
> >> >>> > Dmitriy Pavlov
> >> >>> >
> >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> >> >>> stanlukyanov@gmail.com<ma...@gmail.com>>:
> >> >>> >
> >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and
> >> it’s
> >> >>> >> fine.
> >> >>> >>
> >> >>> >> From: Stanislav Lukyanov
> >> >>> >> Sent: 26 июля 2018 г. 15:12
> >> >>> >> To: dev@ignite.apache.org<ma...@ignite.apache.org>
> >> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> >> >>> >> Persistent Memory
> >> >>> >>
> >> >>> >> Hi,
> >> >>> >>
> >> >>> >> The link you’ve shared gives me 404.
> >> >>> >> Perhaps you need to add a permission for everyone to access the
> >> page?
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> Stan
> >> >>> >>
> >> >>> >> From: Mammo, Mulugeta
> >> >>> >> Sent: 26 июля 2018 г. 2:44
> >> >>> >> To: dev@ignite.apache.org<ma...@ignite.apache.org>
> >> >>> >> Subject: Adding experimental support for Intel Optane DC Persistent
> >> >>> >> Memory
> >> >>> >>
> >> >>> >> Hi,
> >> >>> >>
> >> >>> >> I have added a new proposal to support Intel Optane DC Persistent
> >> >>> >> Memory for Ignite here:
> >> >>> >>
> >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> >> >>> >> .
> >> >>> >>
> >> >>> >> I'm looking forward to your feedback and collaboration on this.
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> Mulugeta
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>>
> >> >>
> >>
> >
--
Best regards,
Ivan Pavlukhin
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Denis Magda <dm...@apache.org>.
Alex, thanks for an extensive summary and proposing different
implementation trails. Presently, the JEP-352 approach seems the most
reasonable - JDK folks will take care of the integration and will be
testing/maintaining the feature going forward, while the Ignite community
will reuse what JDK provides. However, let's see what Eric and Mulugeta of
Intel (both copied) think about this and other approaches.
Ivan, I'm not sure if any performance testing was done for the given
pull-request. Anyway, it might be premature considering Alex's summary.
-
Denis
On Mon, Oct 14, 2019 at 11:50 PM Ivan Pavlukhin <vo...@gmail.com> wrote:
> By the way, did we have a change to run some existing benchmarks
> against proposed implementation?
>
> пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > Hello Mulugeta, Igniters.
> >
> > Thanks for your interest and efforts in integrating the persistent memory
> > to Ignite. Here are my thoughts on the PR:
> >
> > First of all I was questioning whether we want to use the integration
> with
> > the pmem library via JNI. Java 14 will have native support for NVME via
> > native mapped byte buffers [1] since the corresponding tickets are
> already
> > resolved [2],[3]. The suggested integration uses JNI calls quite often
> (if
> > I read the PR right, there will be tens of JNI calls per a single
> > operation), which may and most likely will undermine the benefits of
> using
> > PM. On the other hand, there is a PoC/project [4] which successfully
> > demonstrates how the new integration can be used to natively access
> > persistent memory from Java. Additionally, new Unsafe have several
> > performance issues - each put* methods has a map lookup, and CAS method
> > uses a global lock. Removing these issues will change the PR and
> > architecture dramatically.
> >
> > Other than performance and an extra library dependency, there are
> > additional reasons to use custom implementation of PM persistence
> > primitives:
> >
> > - Ignite significantly benefits from having WAL as a single point of
> > data for both failure recovery and historical rebalance. As far as I
> can
> > guess, the suggested library also uses some sort of journaling in
> order to
> > support crash recovery. Since we do not want give up on the historical
> > rebalance, we would need to write an additional journal, which then
> raises
> > a question of how to transact between this journal.
> > - Ignite is supposed to work with large indexes, a I not aware how
> LLPL
> > handles them. Several researches [5],[6] show that since PM memory has
> > larger access latency than regular RAM, it is beneficial to buffer
> index
> > pages in regular memory and store leaf pages in PM. There is a generic
> > approach to convert in-memory indexes to PM ones [7], I think it makes
> > sense to investigate whether our existing indexes can be converted to
> PM.
> > - There is an active IEP which is targeted for a file-based
> rebalancing.
> > In the suggested implementation this optimization would not work. A
> better
> > integration should allow to take a snapshot of a partition and
> transfer it
> > to another node for fast rebalancing.
> >
> > Overall, I think we may take the following path:
> >
> > - Target JEP352 APIs; for now keep using the whole region msync for
> flush
> > - Introduce PM-based isolated components to support WAL iterator and
> > transaction management. We can start with WAL, then implement a
> PM-based
> > index based on current research, then a tuple heap with free space
> > management. Each of the components may be developed and tested
> independently
> > - Do Ignite internals refactoring to properly abstract storage layer
> > from transactions/communication. Currently, there is an abstraction
> leak in
> > the storage layer, which makes it aware of particular Ignite
> implementation
> > details (Binary Objects, partition eviction, etc)
> > - Once the storage is separated, we should be able to plug the new
> > components instead of the "classical" ones
> >
> > WDYT? I believe we should create a detailed IEP with concrete proposals
> on
> > both PM structures design and Ignite internals changes before making any
> > further code changes.
> >
> > [1] https://openjdk.java.net/jeps/352
> > [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> > [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> > [4] https://github.com/jhalliday/mashona
> > [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> > [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> > [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> > [8]
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
> >
> > вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
> >
> > > Igniters,
> > >
> > > I would like to resurrect this discussion and will review the change
> again
> > > shortly. If anyone want to join the review - you are welcome!
> > >
> > > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> > >
> > >> Hi Dmitry,
> > >>
> > >> That's a BSD-3-Clause license if to believe this statement
> > >> "SPDX-License-Identifier: BSD-3-Clause":
> > >> https://github.com/pmem/llpl/blob/master/LICENSE
> > >>
> > >> This license can be used with ASF software:
> > >> https://www.apache.org/legal/resolved.html#category-a
> > >>
> > >> --
> > >> Denis
> > >>
> > >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dpavlov.spb@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi Denis,
> > >> >
> > >> > Could you please double check if we can refer to any library
> licensed to
> > >> > Intel. Can we develop code only version of this support (without
> > >> shipping
> > >> > it in release)?
> > >> >
> > >> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> > >> > including 128 files changed, patch review will require resources
> from
> > >> > community members to review. I would like to be sure we can include
> this
> > >> > patch from the legal point of view.
> > >> >
> > >> > Sincerely,
> > >> > Dmitriy Pavlov
> > >> >
> > >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> > >> >
> > >> >> Hi Mulugeta,
> > >> >>
> > >> >> I appologise, I've missed that license is already there. So I
> guess it
> > >> is
> > >> >> not standard open-source license, it is seems it is not listed in
> > >> >> https://www.apache.org/legal/resolved.html#category-a
> > >> >>
> > >> >> So there can be legal concern related to including this lib as
> > >> dependency
> > >> >> into Apache product. It should not block review, we can later
> > >> >> consult Secretary/Legal to find out how we can correctly include
> > >> reference
> > >> >> to lib.
> > >> >>
> > >> >> Sincerely,
> > >> >> Dmitriy Pavlov
> > >> >>
> > >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <
> mulugeta.mammo@intel.com>:
> > >> >>
> > >> >>> Hi Dmitriy,
> > >> >>>
> > >> >>> Do you mean our LLPL library? It has a license, please look here:
> > >> >>> https://github.com/pmem/llpl
> > >> >>>
> > >> >>> Regarding the changes made to Ignite, you may refer to the pull
> > >> request
> > >> >>> here: https://github.com/apache/ignite/pull/4381
> > >> >>>
> > >> >>> Thanks,
> > >> >>> Mulugeta
> > >> >>>
> > >> >>> -----Original Message-----
> > >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> > >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> > >> >>> To: dev@ignite.apache.org
> > >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> > >> Persistent
> > >> >>> Memory
> > >> >>>
> > >> >>> Hi Mulugeta Mammo,
> > >> >>>
> > >> >>> I've just noticed that repository, what you refer is full fork of
> > >> Ignite.
> > >> >>> How can I see differences with original Ignite?
> > >> >>>
> > >> >>> One more thing, library which you're referencing seems to not
> contain
> > >> >>> license, at least github can not parse it. Apache product has
> > >> limitations
> > >> >>> which libraries may be used (see
> > >> >>> https://www.apache.org/legal/resolved.html#category-a and
> > >> >>> https://www.apache.org/legal/resolved.html#category-b)
> > >> >>>
> > >> >>> Could you please comment if there is some legal risk?
> > >> >>>
> > >> >>> Sincerely,
> > >> >>> Dmitriy Pavlov
> > >> >>>
> > >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dpavlov.spb@gmail.com
> >:
> > >> >>>
> > >> >>> > Hi,
> > >> >>> >
> > >> >>> > This link works for me
> > >> >>> >
> > >> >>> >
> > >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> > >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > >> >>> >
> > >> >>> > Sincerely,
> > >> >>> > Dmitriy Pavlov
> > >> >>> >
> > >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> > >> >>> stanlukyanov@gmail.com>:
> > >> >>> >
> > >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it
> and
> > >> it’s
> > >> >>> >> fine.
> > >> >>> >>
> > >> >>> >> From: Stanislav Lukyanov
> > >> >>> >> Sent: 26 июля 2018 г. 15:12
> > >> >>> >> To: dev@ignite.apache.org
> > >> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> > >> >>> >> Persistent Memory
> > >> >>> >>
> > >> >>> >> Hi,
> > >> >>> >>
> > >> >>> >> The link you’ve shared gives me 404.
> > >> >>> >> Perhaps you need to add a permission for everyone to access the
> > >> page?
> > >> >>> >>
> > >> >>> >> Thanks,
> > >> >>> >> Stan
> > >> >>> >>
> > >> >>> >> From: Mammo, Mulugeta
> > >> >>> >> Sent: 26 июля 2018 г. 2:44
> > >> >>> >> To: dev@ignite.apache.org
> > >> >>> >> Subject: Adding experimental support for Intel Optane DC
> Persistent
> > >> >>> >> Memory
> > >> >>> >>
> > >> >>> >> Hi,
> > >> >>> >>
> > >> >>> >> I have added a new proposal to support Intel Optane DC
> Persistent
> > >> >>> >> Memory for Ignite here:
> > >> >>> >>
> > >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> > >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> > >> >>> >> .
> > >> >>> >>
> > >> >>> >> I'm looking forward to your feedback and collaboration on this.
> > >> >>> >>
> > >> >>> >> Thanks,
> > >> >>> >> Mulugeta
> > >> >>> >>
> > >> >>> >>
> > >> >>> >>
> > >> >>> >>
> > >> >>>
> > >> >>
> > >>
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Ivan Pavlukhin <vo...@gmail.com>.
By the way, did we have a change to run some existing benchmarks
against proposed implementation?
пт, 11 окт. 2019 г. в 17:49, Alexey Goncharuk <al...@gmail.com>:
>
> Hello Mulugeta, Igniters.
>
> Thanks for your interest and efforts in integrating the persistent memory
> to Ignite. Here are my thoughts on the PR:
>
> First of all I was questioning whether we want to use the integration with
> the pmem library via JNI. Java 14 will have native support for NVME via
> native mapped byte buffers [1] since the corresponding tickets are already
> resolved [2],[3]. The suggested integration uses JNI calls quite often (if
> I read the PR right, there will be tens of JNI calls per a single
> operation), which may and most likely will undermine the benefits of using
> PM. On the other hand, there is a PoC/project [4] which successfully
> demonstrates how the new integration can be used to natively access
> persistent memory from Java. Additionally, new Unsafe have several
> performance issues - each put* methods has a map lookup, and CAS method
> uses a global lock. Removing these issues will change the PR and
> architecture dramatically.
>
> Other than performance and an extra library dependency, there are
> additional reasons to use custom implementation of PM persistence
> primitives:
>
> - Ignite significantly benefits from having WAL as a single point of
> data for both failure recovery and historical rebalance. As far as I can
> guess, the suggested library also uses some sort of journaling in order to
> support crash recovery. Since we do not want give up on the historical
> rebalance, we would need to write an additional journal, which then raises
> a question of how to transact between this journal.
> - Ignite is supposed to work with large indexes, a I not aware how LLPL
> handles them. Several researches [5],[6] show that since PM memory has
> larger access latency than regular RAM, it is beneficial to buffer index
> pages in regular memory and store leaf pages in PM. There is a generic
> approach to convert in-memory indexes to PM ones [7], I think it makes
> sense to investigate whether our existing indexes can be converted to PM.
> - There is an active IEP which is targeted for a file-based rebalancing.
> In the suggested implementation this optimization would not work. A better
> integration should allow to take a snapshot of a partition and transfer it
> to another node for fast rebalancing.
>
> Overall, I think we may take the following path:
>
> - Target JEP352 APIs; for now keep using the whole region msync for flush
> - Introduce PM-based isolated components to support WAL iterator and
> transaction management. We can start with WAL, then implement a PM-based
> index based on current research, then a tuple heap with free space
> management. Each of the components may be developed and tested independently
> - Do Ignite internals refactoring to properly abstract storage layer
> from transactions/communication. Currently, there is an abstraction leak in
> the storage layer, which makes it aware of particular Ignite implementation
> details (Binary Objects, partition eviction, etc)
> - Once the storage is separated, we should be able to plug the new
> components instead of the "classical" ones
>
> WDYT? I believe we should create a detailed IEP with concrete proposals on
> both PM structures design and Ignite internals changes before making any
> further code changes.
>
> [1] https://openjdk.java.net/jeps/352
> [2] https://bugs.openjdk.java.net/browse/JDK-8221397
> [3] https://bugs.openjdk.java.net/browse/JDK-8221696
> [4] https://github.com/jhalliday/mashona
> [5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
> [6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
> [7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
> [8]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
>
> вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <al...@gmail.com>:
>
> > Igniters,
> >
> > I would like to resurrect this discussion and will review the change again
> > shortly. If anyone want to join the review - you are welcome!
> >
> > ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> >
> >> Hi Dmitry,
> >>
> >> That's a BSD-3-Clause license if to believe this statement
> >> "SPDX-License-Identifier: BSD-3-Clause":
> >> https://github.com/pmem/llpl/blob/master/LICENSE
> >>
> >> This license can be used with ASF software:
> >> https://www.apache.org/legal/resolved.html#category-a
> >>
> >> --
> >> Denis
> >>
> >> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dp...@gmail.com>
> >> wrote:
> >>
> >> > Hi Denis,
> >> >
> >> > Could you please double check if we can refer to any library licensed to
> >> > Intel. Can we develop code only version of this support (without
> >> shipping
> >> > it in release)?
> >> >
> >> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> >> > including 128 files changed, patch review will require resources from
> >> > community members to review. I would like to be sure we can include this
> >> > patch from the legal point of view.
> >> >
> >> > Sincerely,
> >> > Dmitriy Pavlov
> >> >
> >> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> >> >
> >> >> Hi Mulugeta,
> >> >>
> >> >> I appologise, I've missed that license is already there. So I guess it
> >> is
> >> >> not standard open-source license, it is seems it is not listed in
> >> >> https://www.apache.org/legal/resolved.html#category-a
> >> >>
> >> >> So there can be legal concern related to including this lib as
> >> dependency
> >> >> into Apache product. It should not block review, we can later
> >> >> consult Secretary/Legal to find out how we can correctly include
> >> reference
> >> >> to lib.
> >> >>
> >> >> Sincerely,
> >> >> Dmitriy Pavlov
> >> >>
> >> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
> >> >>
> >> >>> Hi Dmitriy,
> >> >>>
> >> >>> Do you mean our LLPL library? It has a license, please look here:
> >> >>> https://github.com/pmem/llpl
> >> >>>
> >> >>> Regarding the changes made to Ignite, you may refer to the pull
> >> request
> >> >>> here: https://github.com/apache/ignite/pull/4381
> >> >>>
> >> >>> Thanks,
> >> >>> Mulugeta
> >> >>>
> >> >>> -----Original Message-----
> >> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> >> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> >> >>> To: dev@ignite.apache.org
> >> >>> Subject: Re: Adding experimental support for Intel Optane DC
> >> Persistent
> >> >>> Memory
> >> >>>
> >> >>> Hi Mulugeta Mammo,
> >> >>>
> >> >>> I've just noticed that repository, what you refer is full fork of
> >> Ignite.
> >> >>> How can I see differences with original Ignite?
> >> >>>
> >> >>> One more thing, library which you're referencing seems to not contain
> >> >>> license, at least github can not parse it. Apache product has
> >> limitations
> >> >>> which libraries may be used (see
> >> >>> https://www.apache.org/legal/resolved.html#category-a and
> >> >>> https://www.apache.org/legal/resolved.html#category-b)
> >> >>>
> >> >>> Could you please comment if there is some legal risk?
> >> >>>
> >> >>> Sincerely,
> >> >>> Dmitriy Pavlov
> >> >>>
> >> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
> >> >>>
> >> >>> > Hi,
> >> >>> >
> >> >>> > This link works for me
> >> >>> >
> >> >>> >
> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> >> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >> >>> >
> >> >>> > Sincerely,
> >> >>> > Dmitriy Pavlov
> >> >>> >
> >> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> >> >>> stanlukyanov@gmail.com>:
> >> >>> >
> >> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and
> >> it’s
> >> >>> >> fine.
> >> >>> >>
> >> >>> >> From: Stanislav Lukyanov
> >> >>> >> Sent: 26 июля 2018 г. 15:12
> >> >>> >> To: dev@ignite.apache.org
> >> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> >> >>> >> Persistent Memory
> >> >>> >>
> >> >>> >> Hi,
> >> >>> >>
> >> >>> >> The link you’ve shared gives me 404.
> >> >>> >> Perhaps you need to add a permission for everyone to access the
> >> page?
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> Stan
> >> >>> >>
> >> >>> >> From: Mammo, Mulugeta
> >> >>> >> Sent: 26 июля 2018 г. 2:44
> >> >>> >> To: dev@ignite.apache.org
> >> >>> >> Subject: Adding experimental support for Intel Optane DC Persistent
> >> >>> >> Memory
> >> >>> >>
> >> >>> >> Hi,
> >> >>> >>
> >> >>> >> I have added a new proposal to support Intel Optane DC Persistent
> >> >>> >> Memory for Ignite here:
> >> >>> >>
> >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> >> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> >> >>> >> .
> >> >>> >>
> >> >>> >> I'm looking forward to your feedback and collaboration on this.
> >> >>> >>
> >> >>> >> Thanks,
> >> >>> >> Mulugeta
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>>
> >> >>
> >>
> >
--
Best regards,
Ivan Pavlukhin
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Alexey Goncharuk <al...@gmail.com>.
Hello Mulugeta, Igniters.
Thanks for your interest and efforts in integrating the persistent memory
to Ignite. Here are my thoughts on the PR:
First of all I was questioning whether we want to use the integration with
the pmem library via JNI. Java 14 will have native support for NVME via
native mapped byte buffers [1] since the corresponding tickets are already
resolved [2],[3]. The suggested integration uses JNI calls quite often (if
I read the PR right, there will be tens of JNI calls per a single
operation), which may and most likely will undermine the benefits of using
PM. On the other hand, there is a PoC/project [4] which successfully
demonstrates how the new integration can be used to natively access
persistent memory from Java. Additionally, new Unsafe have several
performance issues - each put* methods has a map lookup, and CAS method
uses a global lock. Removing these issues will change the PR and
architecture dramatically.
Other than performance and an extra library dependency, there are
additional reasons to use custom implementation of PM persistence
primitives:
- Ignite significantly benefits from having WAL as a single point of
data for both failure recovery and historical rebalance. As far as I can
guess, the suggested library also uses some sort of journaling in order to
support crash recovery. Since we do not want give up on the historical
rebalance, we would need to write an additional journal, which then raises
a question of how to transact between this journal.
- Ignite is supposed to work with large indexes, a I not aware how LLPL
handles them. Several researches [5],[6] show that since PM memory has
larger access latency than regular RAM, it is beneficial to buffer index
pages in regular memory and store leaf pages in PM. There is a generic
approach to convert in-memory indexes to PM ones [7], I think it makes
sense to investigate whether our existing indexes can be converted to PM.
- There is an active IEP which is targeted for a file-based rebalancing.
In the suggested implementation this optimization would not work. A better
integration should allow to take a snapshot of a partition and transfer it
to another node for fast rebalancing.
Overall, I think we may take the following path:
- Target JEP352 APIs; for now keep using the whole region msync for flush
- Introduce PM-based isolated components to support WAL iterator and
transaction management. We can start with WAL, then implement a PM-based
index based on current research, then a tuple heap with free space
management. Each of the components may be developed and tested independently
- Do Ignite internals refactoring to properly abstract storage layer
from transactions/communication. Currently, there is an abstraction leak in
the storage layer, which makes it aware of particular Ignite implementation
details (Binary Objects, partition eviction, etc)
- Once the storage is separated, we should be able to plug the new
components instead of the "classical" ones
WDYT? I believe we should create a detailed IEP with concrete proposals on
both PM structures design and Ignite internals changes before making any
further code changes.
[1] https://openjdk.java.net/jeps/352
[2] https://bugs.openjdk.java.net/browse/JDK-8221397
[3] https://bugs.openjdk.java.net/browse/JDK-8221696
[4] https://github.com/jhalliday/mashona
[5] https://www.usenix.org/system/files/conference/fast17/fast17-lee.pdf
[6] http://www.vldb.org/pvldb/vol8/p786-chen.pdf
[7] https://www.cs.utexas.edu/~vijay/papers/sosp19-recipe.pdf
[8]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-16%3A+Optimization+of+rebalancing
вт, 8 окт. 2019 г. в 08:15, Alexey Goncharuk <al...@gmail.com>:
> Igniters,
>
> I would like to resurrect this discussion and will review the change again
> shortly. If anyone want to join the review - you are welcome!
>
> ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
>
>> Hi Dmitry,
>>
>> That's a BSD-3-Clause license if to believe this statement
>> "SPDX-License-Identifier: BSD-3-Clause":
>> https://github.com/pmem/llpl/blob/master/LICENSE
>>
>> This license can be used with ASF software:
>> https://www.apache.org/legal/resolved.html#category-a
>>
>> --
>> Denis
>>
>> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dp...@gmail.com>
>> wrote:
>>
>> > Hi Denis,
>> >
>> > Could you please double check if we can refer to any library licensed to
>> > Intel. Can we develop code only version of this support (without
>> shipping
>> > it in release)?
>> >
>> > https://github.com/apache/ignite/pull/4381 is quite huge change,
>> > including 128 files changed, patch review will require resources from
>> > community members to review. I would like to be sure we can include this
>> > patch from the legal point of view.
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
>> >
>> >> Hi Mulugeta,
>> >>
>> >> I appologise, I've missed that license is already there. So I guess it
>> is
>> >> not standard open-source license, it is seems it is not listed in
>> >> https://www.apache.org/legal/resolved.html#category-a
>> >>
>> >> So there can be legal concern related to including this lib as
>> dependency
>> >> into Apache product. It should not block review, we can later
>> >> consult Secretary/Legal to find out how we can correctly include
>> reference
>> >> to lib.
>> >>
>> >> Sincerely,
>> >> Dmitriy Pavlov
>> >>
>> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
>> >>
>> >>> Hi Dmitriy,
>> >>>
>> >>> Do you mean our LLPL library? It has a license, please look here:
>> >>> https://github.com/pmem/llpl
>> >>>
>> >>> Regarding the changes made to Ignite, you may refer to the pull
>> request
>> >>> here: https://github.com/apache/ignite/pull/4381
>> >>>
>> >>> Thanks,
>> >>> Mulugeta
>> >>>
>> >>> -----Original Message-----
>> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
>> >>> Sent: Wednesday, August 1, 2018 10:49 AM
>> >>> To: dev@ignite.apache.org
>> >>> Subject: Re: Adding experimental support for Intel Optane DC
>> Persistent
>> >>> Memory
>> >>>
>> >>> Hi Mulugeta Mammo,
>> >>>
>> >>> I've just noticed that repository, what you refer is full fork of
>> Ignite.
>> >>> How can I see differences with original Ignite?
>> >>>
>> >>> One more thing, library which you're referencing seems to not contain
>> >>> license, at least github can not parse it. Apache product has
>> limitations
>> >>> which libraries may be used (see
>> >>> https://www.apache.org/legal/resolved.html#category-a and
>> >>> https://www.apache.org/legal/resolved.html#category-b)
>> >>>
>> >>> Could you please comment if there is some legal risk?
>> >>>
>> >>> Sincerely,
>> >>> Dmitriy Pavlov
>> >>>
>> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
>> >>>
>> >>> > Hi,
>> >>> >
>> >>> > This link works for me
>> >>> >
>> >>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
>> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
>> >>> >
>> >>> > Sincerely,
>> >>> > Dmitriy Pavlov
>> >>> >
>> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
>> >>> stanlukyanov@gmail.com>:
>> >>> >
>> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and
>> it’s
>> >>> >> fine.
>> >>> >>
>> >>> >> From: Stanislav Lukyanov
>> >>> >> Sent: 26 июля 2018 г. 15:12
>> >>> >> To: dev@ignite.apache.org
>> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
>> >>> >> Persistent Memory
>> >>> >>
>> >>> >> Hi,
>> >>> >>
>> >>> >> The link you’ve shared gives me 404.
>> >>> >> Perhaps you need to add a permission for everyone to access the
>> page?
>> >>> >>
>> >>> >> Thanks,
>> >>> >> Stan
>> >>> >>
>> >>> >> From: Mammo, Mulugeta
>> >>> >> Sent: 26 июля 2018 г. 2:44
>> >>> >> To: dev@ignite.apache.org
>> >>> >> Subject: Adding experimental support for Intel Optane DC Persistent
>> >>> >> Memory
>> >>> >>
>> >>> >> Hi,
>> >>> >>
>> >>> >> I have added a new proposal to support Intel Optane DC Persistent
>> >>> >> Memory for Ignite here:
>> >>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
>> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
>> >>> >> .
>> >>> >>
>> >>> >> I'm looking forward to your feedback and collaboration on this.
>> >>> >>
>> >>> >> Thanks,
>> >>> >> Mulugeta
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>>
>> >>
>>
>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Alexey Goncharuk <al...@gmail.com>.
Igniters,
I would like to resurrect this discussion and will review the change again
shortly. If anyone want to join the review - you are welcome!
ср, 22 авг. 2018 г. в 18:49, Denis Magda <dm...@apache.org>:
> Hi Dmitry,
>
> That's a BSD-3-Clause license if to believe this statement
> "SPDX-License-Identifier: BSD-3-Clause":
> https://github.com/pmem/llpl/blob/master/LICENSE
>
> This license can be used with ASF software:
> https://www.apache.org/legal/resolved.html#category-a
>
> --
> Denis
>
> On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dp...@gmail.com>
> wrote:
>
> > Hi Denis,
> >
> > Could you please double check if we can refer to any library licensed to
> > Intel. Can we develop code only version of this support (without shipping
> > it in release)?
> >
> > https://github.com/apache/ignite/pull/4381 is quite huge change,
> > including 128 files changed, patch review will require resources from
> > community members to review. I would like to be sure we can include this
> > patch from the legal point of view.
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> >
> >> Hi Mulugeta,
> >>
> >> I appologise, I've missed that license is already there. So I guess it
> is
> >> not standard open-source license, it is seems it is not listed in
> >> https://www.apache.org/legal/resolved.html#category-a
> >>
> >> So there can be legal concern related to including this lib as
> dependency
> >> into Apache product. It should not block review, we can later
> >> consult Secretary/Legal to find out how we can correctly include
> reference
> >> to lib.
> >>
> >> Sincerely,
> >> Dmitriy Pavlov
> >>
> >> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
> >>
> >>> Hi Dmitriy,
> >>>
> >>> Do you mean our LLPL library? It has a license, please look here:
> >>> https://github.com/pmem/llpl
> >>>
> >>> Regarding the changes made to Ignite, you may refer to the pull request
> >>> here: https://github.com/apache/ignite/pull/4381
> >>>
> >>> Thanks,
> >>> Mulugeta
> >>>
> >>> -----Original Message-----
> >>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> >>> Sent: Wednesday, August 1, 2018 10:49 AM
> >>> To: dev@ignite.apache.org
> >>> Subject: Re: Adding experimental support for Intel Optane DC Persistent
> >>> Memory
> >>>
> >>> Hi Mulugeta Mammo,
> >>>
> >>> I've just noticed that repository, what you refer is full fork of
> Ignite.
> >>> How can I see differences with original Ignite?
> >>>
> >>> One more thing, library which you're referencing seems to not contain
> >>> license, at least github can not parse it. Apache product has
> limitations
> >>> which libraries may be used (see
> >>> https://www.apache.org/legal/resolved.html#category-a and
> >>> https://www.apache.org/legal/resolved.html#category-b)
> >>>
> >>> Could you please comment if there is some legal risk?
> >>>
> >>> Sincerely,
> >>> Dmitriy Pavlov
> >>>
> >>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
> >>>
> >>> > Hi,
> >>> >
> >>> > This link works for me
> >>> >
> >>> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> >>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >>> >
> >>> > Sincerely,
> >>> > Dmitriy Pavlov
> >>> >
> >>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
> >>> stanlukyanov@gmail.com>:
> >>> >
> >>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and
> it’s
> >>> >> fine.
> >>> >>
> >>> >> From: Stanislav Lukyanov
> >>> >> Sent: 26 июля 2018 г. 15:12
> >>> >> To: dev@ignite.apache.org
> >>> >> Subject: RE: Adding experimental support for Intel Optane DC
> >>> >> Persistent Memory
> >>> >>
> >>> >> Hi,
> >>> >>
> >>> >> The link you’ve shared gives me 404.
> >>> >> Perhaps you need to add a permission for everyone to access the
> page?
> >>> >>
> >>> >> Thanks,
> >>> >> Stan
> >>> >>
> >>> >> From: Mammo, Mulugeta
> >>> >> Sent: 26 июля 2018 г. 2:44
> >>> >> To: dev@ignite.apache.org
> >>> >> Subject: Adding experimental support for Intel Optane DC Persistent
> >>> >> Memory
> >>> >>
> >>> >> Hi,
> >>> >>
> >>> >> I have added a new proposal to support Intel Optane DC Persistent
> >>> >> Memory for Ignite here:
> >>> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> >>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> >>> >> .
> >>> >>
> >>> >> I'm looking forward to your feedback and collaboration on this.
> >>> >>
> >>> >> Thanks,
> >>> >> Mulugeta
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>>
> >>
>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Denis Magda <dm...@apache.org>.
Hi Dmitry,
That's a BSD-3-Clause license if to believe this statement
"SPDX-License-Identifier: BSD-3-Clause":
https://github.com/pmem/llpl/blob/master/LICENSE
This license can be used with ASF software:
https://www.apache.org/legal/resolved.html#category-a
--
Denis
On Wed, Aug 22, 2018 at 9:28 AM Dmitriy Pavlov <dp...@gmail.com>
wrote:
> Hi Denis,
>
> Could you please double check if we can refer to any library licensed to
> Intel. Can we develop code only version of this support (without shipping
> it in release)?
>
> https://github.com/apache/ignite/pull/4381 is quite huge change,
> including 128 files changed, patch review will require resources from
> community members to review. I would like to be sure we can include this
> patch from the legal point of view.
>
> Sincerely,
> Dmitriy Pavlov
>
> пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
>
>> Hi Mulugeta,
>>
>> I appologise, I've missed that license is already there. So I guess it is
>> not standard open-source license, it is seems it is not listed in
>> https://www.apache.org/legal/resolved.html#category-a
>>
>> So there can be legal concern related to including this lib as dependency
>> into Apache product. It should not block review, we can later
>> consult Secretary/Legal to find out how we can correctly include reference
>> to lib.
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
>>
>>> Hi Dmitriy,
>>>
>>> Do you mean our LLPL library? It has a license, please look here:
>>> https://github.com/pmem/llpl
>>>
>>> Regarding the changes made to Ignite, you may refer to the pull request
>>> here: https://github.com/apache/ignite/pull/4381
>>>
>>> Thanks,
>>> Mulugeta
>>>
>>> -----Original Message-----
>>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
>>> Sent: Wednesday, August 1, 2018 10:49 AM
>>> To: dev@ignite.apache.org
>>> Subject: Re: Adding experimental support for Intel Optane DC Persistent
>>> Memory
>>>
>>> Hi Mulugeta Mammo,
>>>
>>> I've just noticed that repository, what you refer is full fork of Ignite.
>>> How can I see differences with original Ignite?
>>>
>>> One more thing, library which you're referencing seems to not contain
>>> license, at least github can not parse it. Apache product has limitations
>>> which libraries may be used (see
>>> https://www.apache.org/legal/resolved.html#category-a and
>>> https://www.apache.org/legal/resolved.html#category-b)
>>>
>>> Could you please comment if there is some legal risk?
>>>
>>> Sincerely,
>>> Dmitriy Pavlov
>>>
>>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
>>>
>>> > Hi,
>>> >
>>> > This link works for me
>>> >
>>> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
>>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
>>> >
>>> > Sincerely,
>>> > Dmitriy Pavlov
>>> >
>>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <
>>> stanlukyanov@gmail.com>:
>>> >
>>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s
>>> >> fine.
>>> >>
>>> >> From: Stanislav Lukyanov
>>> >> Sent: 26 июля 2018 г. 15:12
>>> >> To: dev@ignite.apache.org
>>> >> Subject: RE: Adding experimental support for Intel Optane DC
>>> >> Persistent Memory
>>> >>
>>> >> Hi,
>>> >>
>>> >> The link you’ve shared gives me 404.
>>> >> Perhaps you need to add a permission for everyone to access the page?
>>> >>
>>> >> Thanks,
>>> >> Stan
>>> >>
>>> >> From: Mammo, Mulugeta
>>> >> Sent: 26 июля 2018 г. 2:44
>>> >> To: dev@ignite.apache.org
>>> >> Subject: Adding experimental support for Intel Optane DC Persistent
>>> >> Memory
>>> >>
>>> >> Hi,
>>> >>
>>> >> I have added a new proposal to support Intel Optane DC Persistent
>>> >> Memory for Ignite here:
>>> >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
>>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
>>> >> .
>>> >>
>>> >> I'm looking forward to your feedback and collaboration on this.
>>> >>
>>> >> Thanks,
>>> >> Mulugeta
>>> >>
>>> >>
>>> >>
>>> >>
>>>
>>
Adding experimental support for Intel Optane DC Persistent Memory
Posted by Dmitriy Pavlov <dp...@gmail.com>.
Hi Denis,
Could you please double check if we can refer to any library licensed to
Intel. Can we develop code only version of this support (without shipping
it in release)?
https://github.com/apache/ignite/pull/4381 is quite huge change, including
128 files changed, patch review will require resources from community
members to review. I would like to be sure we can include this patch from
the legal point of view.
Sincerely,
Dmitriy Pavlov
пт, 3 авг. 2018 г. в 19:23, Dmitriy Pavlov <dp...@gmail.com>:
> Hi Mulugeta,
>
> I appologise, I've missed that license is already there. So I guess it is
> not standard open-source license, it is seems it is not listed in
> https://www.apache.org/legal/resolved.html#category-a
>
> So there can be legal concern related to including this lib as dependency
> into Apache product. It should not block review, we can later
> consult Secretary/Legal to find out how we can correctly include reference
> to lib.
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
>
>> Hi Dmitriy,
>>
>> Do you mean our LLPL library? It has a license, please look here:
>> https://github.com/pmem/llpl
>>
>> Regarding the changes made to Ignite, you may refer to the pull request
>> here: https://github.com/apache/ignite/pull/4381
>>
>> Thanks,
>> Mulugeta
>>
>> -----Original Message-----
>> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
>> Sent: Wednesday, August 1, 2018 10:49 AM
>> To: dev@ignite.apache.org
>> Subject: Re: Adding experimental support for Intel Optane DC Persistent
>> Memory
>>
>> Hi Mulugeta Mammo,
>>
>> I've just noticed that repository, what you refer is full fork of Ignite.
>> How can I see differences with original Ignite?
>>
>> One more thing, library which you're referencing seems to not contain
>> license, at least github can not parse it. Apache product has limitations
>> which libraries may be used (see
>> https://www.apache.org/legal/resolved.html#category-a and
>> https://www.apache.org/legal/resolved.html#category-b)
>>
>> Could you please comment if there is some legal risk?
>>
>> Sincerely,
>> Dmitriy Pavlov
>>
>> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
>>
>> > Hi,
>> >
>> > This link works for me
>> >
>> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
>> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
>> >
>> > Sincerely,
>> > Dmitriy Pavlov
>> >
>> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <stanlukyanov@gmail.com
>> >:
>> >
>> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s
>> >> fine.
>> >>
>> >> From: Stanislav Lukyanov
>> >> Sent: 26 июля 2018 г. 15:12
>> >> To: dev@ignite.apache.org
>> >> Subject: RE: Adding experimental support for Intel Optane DC
>> >> Persistent Memory
>> >>
>> >> Hi,
>> >>
>> >> The link you’ve shared gives me 404.
>> >> Perhaps you need to add a permission for everyone to access the page?
>> >>
>> >> Thanks,
>> >> Stan
>> >>
>> >> From: Mammo, Mulugeta
>> >> Sent: 26 июля 2018 г. 2:44
>> >> To: dev@ignite.apache.org
>> >> Subject: Adding experimental support for Intel Optane DC Persistent
>> >> Memory
>> >>
>> >> Hi,
>> >>
>> >> I have added a new proposal to support Intel Optane DC Persistent
>> >> Memory for Ignite here:
>> >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
>> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
>> >> .
>> >>
>> >> I'm looking forward to your feedback and collaboration on this.
>> >>
>> >> Thanks,
>> >> Mulugeta
>> >>
>> >>
>> >>
>> >>
>>
>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Dmitriy Pavlov <dp...@gmail.com>.
Hi Mulugeta,
I appologise, I've missed that license is already there. So I guess it is
not standard open-source license, it is seems it is not listed in
https://www.apache.org/legal/resolved.html#category-a
So there can be legal concern related to including this lib as dependency
into Apache product. It should not block review, we can later
consult Secretary/Legal to find out how we can correctly include reference
to lib.
Sincerely,
Dmitriy Pavlov
чт, 2 авг. 2018 г. в 0:24, Mammo, Mulugeta <mu...@intel.com>:
> Hi Dmitriy,
>
> Do you mean our LLPL library? It has a license, please look here:
> https://github.com/pmem/llpl
>
> Regarding the changes made to Ignite, you may refer to the pull request
> here: https://github.com/apache/ignite/pull/4381
>
> Thanks,
> Mulugeta
>
> -----Original Message-----
> From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
> Sent: Wednesday, August 1, 2018 10:49 AM
> To: dev@ignite.apache.org
> Subject: Re: Adding experimental support for Intel Optane DC Persistent
> Memory
>
> Hi Mulugeta Mammo,
>
> I've just noticed that repository, what you refer is full fork of Ignite.
> How can I see differences with original Ignite?
>
> One more thing, library which you're referencing seems to not contain
> license, at least github can not parse it. Apache product has limitations
> which libraries may be used (see
> https://www.apache.org/legal/resolved.html#category-a and
> https://www.apache.org/legal/resolved.html#category-b)
>
> Could you please comment if there is some legal risk?
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
>
> > Hi,
> >
> > This link works for me
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> > perimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >
> > Sincerely,
> > Dmitriy Pavlov
> >
> > чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <stanlukyanov@gmail.com
> >:
> >
> >> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s
> >> fine.
> >>
> >> From: Stanislav Lukyanov
> >> Sent: 26 июля 2018 г. 15:12
> >> To: dev@ignite.apache.org
> >> Subject: RE: Adding experimental support for Intel Optane DC
> >> Persistent Memory
> >>
> >> Hi,
> >>
> >> The link you’ve shared gives me 404.
> >> Perhaps you need to add a permission for everyone to access the page?
> >>
> >> Thanks,
> >> Stan
> >>
> >> From: Mammo, Mulugeta
> >> Sent: 26 июля 2018 г. 2:44
> >> To: dev@ignite.apache.org
> >> Subject: Adding experimental support for Intel Optane DC Persistent
> >> Memory
> >>
> >> Hi,
> >>
> >> I have added a new proposal to support Intel Optane DC Persistent
> >> Memory for Ignite here:
> >> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
> >> l+Support+for+Intel+Optane+DC+Persistent+Memory
> >> .
> >>
> >> I'm looking forward to your feedback and collaboration on this.
> >>
> >> Thanks,
> >> Mulugeta
> >>
> >>
> >>
> >>
>
RE: Adding experimental support for Intel Optane DC Persistent
Memory
Posted by "Mammo, Mulugeta" <mu...@intel.com>.
Hi Dmitriy,
Do you mean our LLPL library? It has a license, please look here: https://github.com/pmem/llpl
Regarding the changes made to Ignite, you may refer to the pull request here: https://github.com/apache/ignite/pull/4381
Thanks,
Mulugeta
-----Original Message-----
From: Dmitriy Pavlov [mailto:dpavlov.spb@gmail.com]
Sent: Wednesday, August 1, 2018 10:49 AM
To: dev@ignite.apache.org
Subject: Re: Adding experimental support for Intel Optane DC Persistent Memory
Hi Mulugeta Mammo,
I've just noticed that repository, what you refer is full fork of Ignite.
How can I see differences with original Ignite?
One more thing, library which you're referencing seems to not contain license, at least github can not parse it. Apache product has limitations which libraries may be used (see https://www.apache.org/legal/resolved.html#category-a and
https://www.apache.org/legal/resolved.html#category-b)
Could you please comment if there is some legal risk?
Sincerely,
Dmitriy Pavlov
ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
> Hi,
>
> This link works for me
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Ex
> perimental+Support+for+Intel+Optane+DC+Persistent+Memory
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <st...@gmail.com>:
>
>> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s
>> fine.
>>
>> From: Stanislav Lukyanov
>> Sent: 26 июля 2018 г. 15:12
>> To: dev@ignite.apache.org
>> Subject: RE: Adding experimental support for Intel Optane DC
>> Persistent Memory
>>
>> Hi,
>>
>> The link you’ve shared gives me 404.
>> Perhaps you need to add a permission for everyone to access the page?
>>
>> Thanks,
>> Stan
>>
>> From: Mammo, Mulugeta
>> Sent: 26 июля 2018 г. 2:44
>> To: dev@ignite.apache.org
>> Subject: Adding experimental support for Intel Optane DC Persistent
>> Memory
>>
>> Hi,
>>
>> I have added a new proposal to support Intel Optane DC Persistent
>> Memory for Ignite here:
>> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimenta
>> l+Support+for+Intel+Optane+DC+Persistent+Memory
>> .
>>
>> I'm looking forward to your feedback and collaboration on this.
>>
>> Thanks,
>> Mulugeta
>>
>>
>>
>>
Re: Adding experimental support for Intel Optane DC Persistent Memory
Posted by Dmitriy Pavlov <dp...@gmail.com>.
Hi Mulugeta Mammo,
I've just noticed that repository, what you refer is full fork of Ignite.
How can I see differences with original Ignite?
One more thing, library which you're referencing seems to not contain
license, at least github can not parse it. Apache product has limitations
which libraries may be used (see
https://www.apache.org/legal/resolved.html#category-a and
https://www.apache.org/legal/resolved.html#category-b)
Could you please comment if there is some legal risk?
Sincerely,
Dmitriy Pavlov
ср, 1 авг. 2018 г. в 20:43, Dmitriy Pavlov <dp...@gmail.com>:
> Hi,
>
> This link works for me
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
>
> Sincerely,
> Dmitriy Pavlov
>
> чт, 26 июл. 2018 г. в 15:31, Stanislav Lukyanov <st...@gmail.com>:
>
>> Ah, ok, it’s just the ‘.’ at the end of the link. Removed it and it’s
>> fine.
>>
>> From: Stanislav Lukyanov
>> Sent: 26 июля 2018 г. 15:12
>> To: dev@ignite.apache.org
>> Subject: RE: Adding experimental support for Intel Optane DC Persistent
>> Memory
>>
>> Hi,
>>
>> The link you’ve shared gives me 404.
>> Perhaps you need to add a permission for everyone to access the page?
>>
>> Thanks,
>> Stan
>>
>> From: Mammo, Mulugeta
>> Sent: 26 июля 2018 г. 2:44
>> To: dev@ignite.apache.org
>> Subject: Adding experimental support for Intel Optane DC Persistent Memory
>>
>> Hi,
>>
>> I have added a new proposal to support Intel Optane DC Persistent Memory
>> for Ignite here:
>> https://cwiki.apache.org/confluence/display/IGNITE/Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
>> .
>>
>> I'm looking forward to your feedback and collaboration on this.
>>
>> Thanks,
>> Mulugeta
>>
>>
>>
>>