You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Steve Hostettler <st...@gmail.com> on 2018/08/21 16:41:55 UTC

Storage Class Memory and Persistent Collections

Hello,

 

Clearly Storage Class Memory represents a breakthrough for "in memory" grids
and some people already tried it on Ignite :
http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
ml

I would like to know what is the position of the community towards  the
Persistent Collections (
<https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
STV_2017>
https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
TV_2017,  <https://pcollections.org/> https://pcollections.org/)? 
According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
massively reduce the marshalling/unmarshalling time. `

1)   Do you guys plan to test it and to support SCM such as 3DXPoint
natively?

2)   Would it mean that ignite would not serialize the objects off heap
anymore?

 

Hope it makes sense.

 

Best Regards


Re: Storage Class Memory and Persistent Collections

Posted by Steve Hostettler <st...@gmail.com>.
Hello Pavel,

well this is what Intel claims in the above video. I try to understand
exactly how but the only think I can figure out is that there is some sort
of equivalence between main memory and the 3dx point hardware.
I really really would like to understand this statemetn.

Best Regards

On Thu, Aug 23, 2018 at 2:16 PM Pavel Kovalenko <jo...@gmail.com> wrote:

> Hello Steve,
>
> I've looked at Persistent Collectors library, but don't see any big
> advantages to use it inside Ignite right now.
> Why do you think, that marshalling/unmarshalling process will be speeded up
> with this library? As I can see there are no explicit changes in
> serialization process, just added persistent ability to regular java
> collections.
>
> 2018-08-23 9:53 GMT+03:00 Steve Hostettler <st...@gmail.com>:
>
> > That's great but would you mind explaining to me how we can get rid of
> the
> > marshalling/unmarshalling? Because that would significantly speed up the
> > processes that run on the grid.
> >
> > On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <dm...@apache.org> wrote:
> >
> > > Hello Steve,
> > >
> > > Intel folks are already contributing Intel Optane Persistent Memory
> > support
> > > to Ignite:
> > >
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
> > >
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > com/Adding-experimental-support-for-Intel-Optane-DC-
> > Persistent-Memory-td33041.html
> > >
> > > Hopefully, the contribution will be reviewed and accepted soon.
> > >
> > > --
> > > Denis
> > >
> > > On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> > > steve.hostettler@gmail.com> wrote:
> > >
> > > > Hello,
> > > >
> > > >
> > > >
> > > > Clearly Storage Class Memory represents a breakthrough for "in
> memory"
> > > > grids
> > > > and some people already tried it on Ignite :
> > > >
> > > >
> > > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> > ssds-verified-on.ht
> > > > ml
> > > > <
> > > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> > ssds-verified-on.html
> > > >
> > > >
> > > > I would like to know what is the position of the community towards
> the
> > > > Persistent Collections (
> > > > <
> > > >
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=I
> > > > STV_2017
> > > > <
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=ISTV_2017
> > > >
> > > > >
> > > >
> > > >
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=IS
> > > > TV_2017
> > > > <
> > > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> > Video&utm_campaign=ISTV_2017
> > > >,
> > > > <https://pcollections.org/> https://pcollections.org/)?
> > > > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > > > massively reduce the marshalling/unmarshalling time. `
> > > >
> > > > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > > > natively?
> > > >
> > > > 2)   Would it mean that ignite would not serialize the objects off
> heap
> > > > anymore?
> > > >
> > > >
> > > >
> > > > Hope it makes sense.
> > > >
> > > >
> > > >
> > > > Best Regards
> > > >
> > > >
> > >
> >
> >
> > --
> > ===========
> > Steve Hostettler
> >
> > Université de Genève
> > CUI - Battelle bat. A
> > Route de Drize, 7
> > CH-1227 Carouge
> > Tel. +33 67 075 2843
> > Fax +41 22 379 0250
> >
>


-- 
===========
Steve Hostettler

Université de Genève
CUI - Battelle bat. A
Route de Drize, 7
CH-1227 Carouge
Tel. +33 67 075 2843
Fax +41 22 379 0250

Re: Storage Class Memory and Persistent Collections

Posted by Pavel Kovalenko <jo...@gmail.com>.
Hello Steve,

I've looked at Persistent Collectors library, but don't see any big
advantages to use it inside Ignite right now.
Why do you think, that marshalling/unmarshalling process will be speeded up
with this library? As I can see there are no explicit changes in
serialization process, just added persistent ability to regular java
collections.

2018-08-23 9:53 GMT+03:00 Steve Hostettler <st...@gmail.com>:

> That's great but would you mind explaining to me how we can get rid of the
> marshalling/unmarshalling? Because that would significantly speed up the
> processes that run on the grid.
>
> On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <dm...@apache.org> wrote:
>
> > Hello Steve,
> >
> > Intel folks are already contributing Intel Optane Persistent Memory
> support
> > to Ignite:
> >
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
> >
> > http://apache-ignite-developers.2346864.n4.nabble.
> com/Adding-experimental-support-for-Intel-Optane-DC-
> Persistent-Memory-td33041.html
> >
> > Hopefully, the contribution will be reviewed and accepted soon.
> >
> > --
> > Denis
> >
> > On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> > steve.hostettler@gmail.com> wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > > Clearly Storage Class Memory represents a breakthrough for "in memory"
> > > grids
> > > and some people already tried it on Ignite :
> > >
> > >
> > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> ssds-verified-on.ht
> > > ml
> > > <
> > http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-
> ssds-verified-on.html
> > >
> > >
> > > I would like to know what is the position of the community towards  the
> > > Persistent Collections (
> > > <
> > >
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=I
> > > STV_2017
> > > <
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=ISTV_2017
> > >
> > > >
> > >
> > >
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=IS
> > > TV_2017
> > > <
> > https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=
> Video&utm_campaign=ISTV_2017
> > >,
> > > <https://pcollections.org/> https://pcollections.org/)?
> > > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > > massively reduce the marshalling/unmarshalling time. `
> > >
> > > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > > natively?
> > >
> > > 2)   Would it mean that ignite would not serialize the objects off heap
> > > anymore?
> > >
> > >
> > >
> > > Hope it makes sense.
> > >
> > >
> > >
> > > Best Regards
> > >
> > >
> >
>
>
> --
> ===========
> Steve Hostettler
>
> Université de Genève
> CUI - Battelle bat. A
> Route de Drize, 7
> CH-1227 Carouge
> Tel. +33 67 075 2843
> Fax +41 22 379 0250
>

Re: Storage Class Memory and Persistent Collections

Posted by Steve Hostettler <st...@gmail.com>.
That's great but would you mind explaining to me how we can get rid of the
marshalling/unmarshalling? Because that would significantly speed up the
processes that run on the grid.

On Wed, Aug 22, 2018 at 3:36 AM Denis Magda <dm...@apache.org> wrote:

> Hello Steve,
>
> Intel folks are already contributing Intel Optane Persistent Memory support
> to Ignite:
>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Adding-experimental-support-for-Intel-Optane-DC-Persistent-Memory-td33041.html
>
> Hopefully, the contribution will be reviewed and accepted soon.
>
> --
> Denis
>
> On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
> steve.hostettler@gmail.com> wrote:
>
> > Hello,
> >
> >
> >
> > Clearly Storage Class Memory represents a breakthrough for "in memory"
> > grids
> > and some people already tried it on Ignite :
> >
> >
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
> > ml
> > <
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.html
> >
> >
> > I would like to know what is the position of the community towards  the
> > Persistent Collections (
> > <
> >
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
> > STV_2017
> > <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017
> >
> > >
> >
> >
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
> > TV_2017
> > <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017
> >,
> > <https://pcollections.org/> https://pcollections.org/)?
> > According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> > massively reduce the marshalling/unmarshalling time. `
> >
> > 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> > natively?
> >
> > 2)   Would it mean that ignite would not serialize the objects off heap
> > anymore?
> >
> >
> >
> > Hope it makes sense.
> >
> >
> >
> > Best Regards
> >
> >
>


-- 
===========
Steve Hostettler

Université de Genève
CUI - Battelle bat. A
Route de Drize, 7
CH-1227 Carouge
Tel. +33 67 075 2843
Fax +41 22 379 0250

Re: Storage Class Memory and Persistent Collections

Posted by Denis Magda <dm...@apache.org>.
Hello Steve,

Intel folks are already contributing Intel Optane Persistent Memory support
to Ignite:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-26%3A+Adding+Experimental+Support+for+Intel+Optane+DC+Persistent+Memory
http://apache-ignite-developers.2346864.n4.nabble.com/Adding-experimental-support-for-Intel-Optane-DC-Persistent-Memory-td33041.html

Hopefully, the contribution will be reviewed and accepted soon.

--
Denis

On Tue, Aug 21, 2018 at 12:42 PM Steve Hostettler <
steve.hostettler@gmail.com> wrote:

> Hello,
>
>
>
> Clearly Storage Class Memory represents a breakthrough for "in memory"
> grids
> and some people already tried it on Ignite :
>
> http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.ht
> ml
> <http://dmagda.blogspot.com/2017/10/3d-xpoint-outperforms-ssds-verified-on.html>
>
> I would like to know what is the position of the community towards  the
> Persistent Collections (
> <
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=I
> STV_2017
> <https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017>
> >
>
> https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=IS
> TV_2017
> <https://github.com/pmem/pcj?utm_source=ISTV&utm_medium=Video&utm_campaign=ISTV_2017>,
> <https://pcollections.org/> https://pcollections.org/)?
> According to https://www.youtube.com/watch?v=ZuLAF3ppCzs , it could
> massively reduce the marshalling/unmarshalling time. `
>
> 1)   Do you guys plan to test it and to support SCM such as 3DXPoint
> natively?
>
> 2)   Would it mean that ignite would not serialize the objects off heap
> anymore?
>
>
>
> Hope it makes sense.
>
>
>
> Best Regards
>
>