You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by vveider <mr...@gmail.com> on 2018/03/14 09:36:01 UTC
Apache Ignite RPM packaging (Stage II)
Hi, Igniters!
Release 2.4 is almost there, at least binary part of it, so I'd like to move
forward to further improve and widen AI delivery through packages.
As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
improve usability and significantly reduce required download sizes, I
purpose that in 2.5 release we introduce splitted delivery as follows:
- CORE
- bin
- config
- libs (!optional)
- OPTIONAL LIBS
- BENCHMARKS
- DOCS (?)
- EXAMPLES
- .NET PLATFORM FILES
- C++ PLATFORM FILES
This architecture, as I assume, will add flexibility (no reason to download
all 280M+ of binaries where you are to run only core node functionality) and
maintainability (you are in full control of what is installed on your
system).
After successful architecture choice, same scheme are planned to be used in
DEB packages as well.
WDYT?
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I am not sure that an email thread of over 20 messages is a good medium to
discuss proposals. In Ignite, we create IEPs. Can you please summarize your
proposal there and send a link there? Please explain not only the change
itself, but the reason why we need it.
D.
On Thu, Apr 12, 2018 at 4:00 PM, Denis Magda <dm...@apache.org> wrote:
> Petr,
>
> I wouldn't postpone this until 2.6 that will be out nor earlier than 3
> months from now.
>
> *Anton V.*, could review and sign off the changes? Not sure we have a
> better person in the community who can do that.
>
> --
> Denis
>
> On Thu, Apr 12, 2018 at 9:10 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
> > If someone from PMCы or Committers still sees necessity about including
> > these tasks into Apache Ignite 2.5 release, this is the last chance to do
> > so.
> > Otherwise this task will be moved to at 2.6 release at least, or even
> > moved to backlog indefinitely.
> >
> >
> >
> > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> > >
> > > To top new RPM architecture off, update to release process is
> introduced
> > — [1] [2].
> > >
> > > Both tasks (this one and IGNITE-7647) are ready for review and should
> be
> > merged simultaneously.
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > [2] https://github.com/apache/ignite-release/pull/1
> > >
> > >
> > >
> > >
> > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com>
> > wrote:
> > >>
> > >> Hello!
> > >>
> > >> Let me share my idea of how this shoud work. Splitting package into
> > >> sub-packages should be dependency-driven.
> > >>
> > >> It means that all Ignite modules without dependencies or with small
> > >> dependencies (such as ignite-log4j) should be included in ignite-core.
> > It
> > >> doesn't make sense to make a zillion RPM packages.
> > >>
> > >> Critical things like ignite-spring and ignite-indexing should be in
> > >> ignite-core of course, even if they have dependencies. Ignite-core
> > should
> > >> be fully self-sufficient and feature-complete.
> > >>
> > >> However, e.g. .net API should probably be in a separate package,
> > because it
> > >> should depend on mono | net-core. We may also have ignite-devel
> package
> > >> which should include all modules which only make sense for developers
> > who
> > >> write code. Such as hibernate integration.
> > >>
> > >> I'm not sure about MR modules. The main question should be, does it
> have
> > >> dependencies? Can it run stand-alone without writing code?
> > >>
> > >> Hope this helps,
> > >>
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > >>
> > >>> Hi, Igniters!
> > >>>
> > >>>
> > >>> Here are some news on our RPM packages initiative.
> > >>>
> > >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> > >>> packages [1]. Main “new feature” is — split design. Also I’ve added
> > >>> package.sh script for automating package building process which will
> > help
> > >>> organise corresponding builds in TC as well as simplify process for
> > >>> developers who wishes to have custom packages.
> > >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> > Apache
> > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > reviewer.
> > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> > >>> repositories on Apache Bintray. Though they are already prepared for
> > >>> hosting RPM and DEB packages respectively, and there is a way of
> > linking
> > >>> them to apache.org/dist/ignite page, there is possible alternative
> in
> > >>> storing there only plain directory layout corresponding to each
> > repository
> > >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> > >>> versions, etc.) by ourselves, having more control over repositories
> but
> > >>> lacking some simplicity of deploying new releases. WDYT? Should we
> try
> > >>> Cassandra approach? They are storing their DEB packages as I
> described
> > >>> above [5].
> > >>>
> > >>> Also — a question arose while I was working on this issue: which OSes
> > (and
> > >>> which versions of each) are we going to support (if we are going) in
> > terms
> > >>> of step-by-step list? Currently RPM packages are tested only with
> > latest
> > >>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> > >>> distributives [6] some of which are more o less popular among OS
> > community
> > >>> (ALT, Fedora, openSUSE, etc.).
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > >>> [2] https://github.com/apache/ignite/pull/3703
> > >>> [3] https://bintray.com/apache/ignite-rpm
> > >>> [4] https://bintray.com/apache/ignite-deb
> > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > distributions
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> > >>>>
> > >>>> I suppose that most everything if not all from libs/options will go
> to
> > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > >>>> More precise lib selection (if something from optional would better
> to
> > >>> have in core package) will be discussed right after preliminary split
> > >>> architecture agreement.
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> I like idea of keeping simple system of modules, so +1 from me.
> > >>>>>
> > >>>>> Where optional libs (e.g Direct IO plugin) would be included, would
> > it
> > >>> be
> > >>>>> core or optional?
> > >>>>>
> > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> > >>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> How big would be a final core module?
> > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
> > it’s
> > >>> own
> > >>>>>>> package.
> > >>>>>>
> > >>>>>>
> > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
> > and
> > >>>>>> propose the simplest modular system:
> > >>>>>>
> > >>>>>> - core module that includes basic Ignite capabilities including
> SQL,
> > >>>>>> compute grid, service grid, k/v
> > >>>>>> - optional module hosts the rest - ML, streamers integration
> (kafka,
> > >>>>>> flink), kubernetes, etc.
> > >>>>>>
> > >>>>>> What do you think?
> > >>>>>>
> > >>>>>> --
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> mr.weider@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> *DEB package
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> > wrote:
> > >>>>>>>>
> > >>>>>>>> Considering that DEV package for now is almost platform
> > independent
> > >>>>>> (its
> > >>>>>>> a java application more or less), that package will work almost
> on
> > any
> > >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
> etc.
> > >>>>>>>> The only restriction is existence of systemctl (systemd) service
> > >>>>>> manager
> > >>>>>>> — we are dependent on it.
> > >>>>>>>>
> > >>>>>>>> Thats why, for instance, our RPM repository is called simply
> ‘rpm’
> > >>> and
> > >>>>>>> package has no arch or dist suffix — it will work on CentOS,
> RHEL,
> > >>>>>> Fedora,
> > >>>>>>> etc. with presence of aforementioned systemd.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Will Debian package work for Ubuntu?
> > >>>>>>>>>
> > >>>>>>>>> D.
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > mr.weider@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> > official
> > >>>>>>>>>> repositories, there can be a problem from OS community.
> > >>>>>>>>>>
> > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> > package
> > >>>>>>> build
> > >>>>>>>>>> (package layout / install scripts) for speeding up things and
> > >>>>>> excluding
> > >>>>>>>>>> possible duplication and desynchronisation, so its a matter of
> > ’sit
> > >>>>>>> and do’
> > >>>>>>>>>> rather then some technical research. Thats why I rose
> discussion
> > >>>>>> about
> > >>>>>>>>>> future package architecture, so that after agreement I'm be
> > able to
> > >>>>>>> pack
> > >>>>>>>>>> both RPM and DEB identically.
> > >>>>>>>>>>
> > >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> > current
> > >>> RPM
> > >>>>>>>>>> layout in no time.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > >>> dsetrakyan@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Peter,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I don't think the package size of 280M is going to be a
> > problem at
> > >>>>>>> all,
> > >>>>>>>>>> but
> > >>>>>>>>>>> what you are suggesting can be an improvement down the road.
> > >>>>>>>>>>>
> > >>>>>>>>>>> In the mean time, I think our top priority should be to
> provide
> > >>>>>>> packages
> > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Agree?
> > >>>>>>>>>>>
> > >>>>>>>>>>> D.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> mr.weider@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi, Igniters!
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so
> > I'd
> > >>>>>> like
> > >>>>>>> to
> > >>>>>>>>>>>> move
> > >>>>>>>>>>>> forward to further improve and widen AI delivery through
> > >>> packages.
> > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> > >>> 280M+
> > >>>>>>> and,
> > >>>>>>>>>> to
> > >>>>>>>>>>>> improve usability and significantly reduce required download
> > >>>>>> sizes, I
> > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery
> as
> > >>>>>>> follows:
> > >>>>>>>>>>>> - CORE
> > >>>>>>>>>>>> - bin
> > >>>>>>>>>>>> - config
> > >>>>>>>>>>>> - libs (!optional)
> > >>>>>>>>>>>> - OPTIONAL LIBS
> > >>>>>>>>>>>> - BENCHMARKS
> > >>>>>>>>>>>> - DOCS (?)
> > >>>>>>>>>>>> - EXAMPLES
> > >>>>>>>>>>>> - .NET PLATFORM FILES
> > >>>>>>>>>>>> - C++ PLATFORM FILES
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> > reason
> > >>> to
> > >>>>>>>>>> download
> > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> > >>>>>>> functionality)
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>> maintainability (you are in full control of what is
> installed
> > on
> > >>>>>> your
> > >>>>>>>>>>>> system).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> After successful architecture choice, same scheme are
> planned
> > to
> > >>> be
> > >>>>>>>>>> used in
> > >>>>>>>>>>>> DEB packages as well.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Sent from: http://apache-ignite-
> developers.2346864.n4.nabble.
> > >>> com/
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>>
> > >
> >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Denis Magda <dm...@apache.org>.
Petr,
I wouldn't postpone this until 2.6 that will be out nor earlier than 3
months from now.
*Anton V.*, could review and sign off the changes? Not sure we have a
better person in the community who can do that.
--
Denis
On Thu, Apr 12, 2018 at 9:10 AM, Petr Ivanov <mr...@gmail.com> wrote:
> If someone from PMCы or Committers still sees necessity about including
> these tasks into Apache Ignite 2.5 release, this is the last chance to do
> so.
> Otherwise this task will be moved to at 2.6 release at least, or even
> moved to backlog indefinitely.
>
>
>
> > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> >
> > To top new RPM architecture off, update to release process is introduced
> — [1] [2].
> >
> > Both tasks (this one and IGNITE-7647) are ready for review and should be
> merged simultaneously.
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > [2] https://github.com/apache/ignite-release/pull/1
> >
> >
> >
> >
> >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com>
> wrote:
> >>
> >> Hello!
> >>
> >> Let me share my idea of how this shoud work. Splitting package into
> >> sub-packages should be dependency-driven.
> >>
> >> It means that all Ignite modules without dependencies or with small
> >> dependencies (such as ignite-log4j) should be included in ignite-core.
> It
> >> doesn't make sense to make a zillion RPM packages.
> >>
> >> Critical things like ignite-spring and ignite-indexing should be in
> >> ignite-core of course, even if they have dependencies. Ignite-core
> should
> >> be fully self-sufficient and feature-complete.
> >>
> >> However, e.g. .net API should probably be in a separate package,
> because it
> >> should depend on mono | net-core. We may also have ignite-devel package
> >> which should include all modules which only make sense for developers
> who
> >> write code. Such as hibernate integration.
> >>
> >> I'm not sure about MR modules. The main question should be, does it have
> >> dependencies? Can it run stand-alone without writing code?
> >>
> >> Hope this helps,
> >>
> >> --
> >> Ilya Kasnacheev
> >>
> >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>
> >>> Hi, Igniters!
> >>>
> >>>
> >>> Here are some news on our RPM packages initiative.
> >>>
> >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> >>> packages [1]. Main “new feature” is — split design. Also I’ve added
> >>> package.sh script for automating package building process which will
> help
> >>> organise corresponding builds in TC as well as simplify process for
> >>> developers who wishes to have custom packages.
> >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> Apache
> >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> reviewer.
> >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> >>> repositories on Apache Bintray. Though they are already prepared for
> >>> hosting RPM and DEB packages respectively, and there is a way of
> linking
> >>> them to apache.org/dist/ignite page, there is possible alternative in
> >>> storing there only plain directory layout corresponding to each
> repository
> >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> >>> versions, etc.) by ourselves, having more control over repositories but
> >>> lacking some simplicity of deploying new releases. WDYT? Should we try
> >>> Cassandra approach? They are storing their DEB packages as I described
> >>> above [5].
> >>>
> >>> Also — a question arose while I was working on this issue: which OSes
> (and
> >>> which versions of each) are we going to support (if we are going) in
> terms
> >>> of step-by-step list? Currently RPM packages are tested only with
> latest
> >>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> >>> distributives [6] some of which are more o less popular among OS
> community
> >>> (ALT, Fedora, openSUSE, etc.).
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> >>> [2] https://github.com/apache/ignite/pull/3703
> >>> [3] https://bintray.com/apache/ignite-rpm
> >>> [4] https://bintray.com/apache/ignite-deb
> >>> [5] https://bintray.com/apache/cassandra/debian#files/
> >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> distributions
> >>>
> >>>
> >>>
> >>>
> >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>
> >>>> I suppose that most everything if not all from libs/options will go to
> >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> >>>> More precise lib selection (if something from optional would better to
> >>> have in core package) will be discussed right after preliminary split
> >>> architecture agreement.
> >>>>
> >>>>
> >>>>
> >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> wrote:
> >>>>>
> >>>>> I like idea of keeping simple system of modules, so +1 from me.
> >>>>>
> >>>>> Where optional libs (e.g Direct IO plugin) would be included, would
> it
> >>> be
> >>>>> core or optional?
> >>>>>
> >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> How big would be a final core module?
> >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
> it’s
> >>> own
> >>>>>>> package.
> >>>>>>
> >>>>>>
> >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
> and
> >>>>>> propose the simplest modular system:
> >>>>>>
> >>>>>> - core module that includes basic Ignite capabilities including SQL,
> >>>>>> compute grid, service grid, k/v
> >>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
> >>>>>> flink), kubernetes, etc.
> >>>>>>
> >>>>>> What do you think?
> >>>>>>
> >>>>>> --
> >>>>>> Denis
> >>>>>>
> >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> *DEB package
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> wrote:
> >>>>>>>>
> >>>>>>>> Considering that DEV package for now is almost platform
> independent
> >>>>>> (its
> >>>>>>> a java application more or less), that package will work almost on
> any
> >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>>>>> The only restriction is existence of systemctl (systemd) service
> >>>>>> manager
> >>>>>>> — we are dependent on it.
> >>>>>>>>
> >>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> >>> and
> >>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>>>>> Fedora,
> >>>>>>> etc. with presence of aforementioned systemd.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Will Debian package work for Ubuntu?
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> mr.weider@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> official
> >>>>>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>>>>
> >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> package
> >>>>>>> build
> >>>>>>>>>> (package layout / install scripts) for speeding up things and
> >>>>>> excluding
> >>>>>>>>>> possible duplication and desynchronisation, so its a matter of
> ’sit
> >>>>>>> and do’
> >>>>>>>>>> rather then some technical research. Thats why I rose discussion
> >>>>>> about
> >>>>>>>>>> future package architecture, so that after agreement I'm be
> able to
> >>>>>>> pack
> >>>>>>>>>> both RPM and DEB identically.
> >>>>>>>>>>
> >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> current
> >>> RPM
> >>>>>>>>>> layout in no time.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Peter,
> >>>>>>>>>>>
> >>>>>>>>>>> I don't think the package size of 280M is going to be a
> problem at
> >>>>>>> all,
> >>>>>>>>>> but
> >>>>>>>>>>> what you are suggesting can be an improvement down the road.
> >>>>>>>>>>>
> >>>>>>>>>>> In the mean time, I think our top priority should be to provide
> >>>>>>> packages
> >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>>>>>>>>
> >>>>>>>>>>> Agree?
> >>>>>>>>>>>
> >>>>>>>>>>> D.
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so
> I'd
> >>>>>> like
> >>>>>>> to
> >>>>>>>>>>>> move
> >>>>>>>>>>>> forward to further improve and widen AI delivery through
> >>> packages.
> >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> >>> 280M+
> >>>>>>> and,
> >>>>>>>>>> to
> >>>>>>>>>>>> improve usability and significantly reduce required download
> >>>>>> sizes, I
> >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
> >>>>>>> follows:
> >>>>>>>>>>>> - CORE
> >>>>>>>>>>>> - bin
> >>>>>>>>>>>> - config
> >>>>>>>>>>>> - libs (!optional)
> >>>>>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>>>>> - BENCHMARKS
> >>>>>>>>>>>> - DOCS (?)
> >>>>>>>>>>>> - EXAMPLES
> >>>>>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>>>>
> >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> reason
> >>> to
> >>>>>>>>>> download
> >>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> >>>>>>> functionality)
> >>>>>>>>>>>> and
> >>>>>>>>>>>> maintainability (you are in full control of what is installed
> on
> >>>>>> your
> >>>>>>>>>>>> system).
> >>>>>>>>>>>>
> >>>>>>>>>>>> After successful architecture choice, same scheme are planned
> to
> >>> be
> >>>>>>>>>> used in
> >>>>>>>>>>>> DEB packages as well.
> >>>>>>>>>>>>
> >>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> >>> com/
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
+1
On Thu, Apr 19, 2018 at 5:45 AM, Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I recommend inclusion of this change so that it can make way into 2.5.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2018-04-19 9:03 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
> > There is not much time left for Apache Ignite 2.5 release, so let’s move
> > stage II of packaging architecture implementation (with additional split
> > scheme discussion) to 2.6 scope.
> >
> > Instead, I’d like to include DEB package to Apache Ignite 2.5 release.
> > Corresponding PR is already prepared [1].
> >
> > Also, I’ll try to prepare modifications for release procedure scripts to
> > automate uploading packages to our new Bintray RPM and DEB repositories
> > before the code freeze.
> >
> >
> > [1] https://github.com/apache/ignite/pull/3869 <
> https://github.com/apache/
> > ignite/pull/3869>
> >
> >
> >
> > > On 18 Apr 2018, at 14:44, Ilya Kasnacheev <il...@gmail.com>
> > wrote:
> > >
> > > Copying anything manually to anywhere /usr (excluding /usr/local) is an
> > > example of slackwarization that package users and creators try to
> avoid.
> > >
> > >> By linux file hierarchy convention, home dir should be in /usr/lib
> > >
> > > Citation needed! I bet you're interpreting it wrong, since I've listed
> a
> > > random dir in /var/lib, and:
> > > ~/w/incubator-ignite% sudo ls -al /var/lib/lightdm
> > > drwxr-x--- 6 lightdm lightdm 4096 авг 14 2017 .
> > > drwxr-xr-x 89 root root 4096 мар 10 22:37 ..
> > > drwxrwxr-x 4 lightdm lightdm 4096 июл 19 2017 .cache
> > > drwxr-xr-x 5 lightdm lightdm 4096 июл 19 2017 .config
> > > drwx------ 2 lightdm lightdm 4096 апр 11 18:25 .gconf
> > > drwxr-xr-x 3 lightdm lightdm 4096 июл 19 2017 .local
> > > -rw------- 1 lightdm lightdm 283 апр 11 18:25 .Xauthority
> > >
> > > ...it definitely looks like a home dir. Which goes with additional
> > benefit
> > > of being writable by end-user.
> > >
> > > --
> > > Ilya Kasnacheev
> > >
> > > 2018-04-16 9:22 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > >
> > >>
> > >>
> > >>> On 15 Apr 2018, at 10:19, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >
> > >> wrote:
> > >>>
> > >>> Hello!
> > >>>
> > >>> With existing binary archive, user can move directories from
> > >>> apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to
> > >> activate
> > >>> them.
> > >>>
> > >>> But with RPM, user should not contemplate moving directories from
> > >>> /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
> > >>
> > >> I always thought of that option as COPYING optional libs, not MOVING.
> > >>
> > >>
> > >>>
> > >>>
> > >>> User lacks permissions for that operation and it would defeat the
> > purpose
> > >>> of having a RPM (imagine trying to upgrade Ignite RPM version with a
> > >> setup
> > >>> like that). "Turning distribution into Slackware" should not be an
> > >> offering.
> > >>
> > >> Can’t imagine use case where Apache Ignite software is installed by
> one
> > >> user, and run by another, without sudo/root permissions.
> > >> Yet, ignite user’s login shell is disabled indeed and without
> sudo/root
> > >> permissions optional libs cannot be copied.
> > >> Also — the symlinks is a good idea, but I’ve already thought of it and
> > I’m
> > >> planning addition of special utility for enabling / disabling optional
> > libs
> > >> (like a2enable/a2disable) in next development iteration.
> > >>
> > >>
> > >>>
> > >>> How it could work: Imagine we create /var/lib/ignite, use it as
> > >>> IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite.
> > This
> > >>> way, we can add and remove symlinks to modules in optional/, and thus
> > >>> enable and disable them as user sees fit.
> > >>
> > >> By linux file hierarchy convention, home dir should be in /usr/lib.
> > >> /var/lib/ is currently used for working files (MySQL alike).
> > >>
> > >>
> > >>>
> > >>> This also answers the problem of "what's IGNITE_HOME for visor
> launched
> > >>> from /usr/bin”
> > >>
> > >> That is addressed in separate task and will be fixed with minor
> startup
> > >> script redesign with universal location-independent solution.
> > >>
> > >>
> > >>>
> > >>> But obviously this will require dedication and effort.
> > >>
> > >> No problems with efforts after final design is discussed an agreed.
> > >>
> > >>
> > >>>
> > >>>
> > >>> --
> > >>> Ilya Kasnacheev
> > >>>
> > >>> 2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> > >>>
> > >>>> Current packages design (after installation) does not differ from
> > binary
> > >>>> archive - everything works (except necessity to run service instead
> > >>>> ignite.sh) just the same way, including libs/optional.
> > >>>>
> > >>>> Also, there can be issues with system JDK version by default, but
> > every
> > >>>> problem will be in journalctl and/or /var/log, and package has
> strong
> > >>>> dependency on any implementation of JDK 1.8.
> > >>>>
> > >>>>
> > >>>> I am lacking enough feedback about Apache Ignite from packages from
> > real
> > >>>> users, don’t know real use cases so still "moving in the dark".
> > >>>>
> > >>>>
> > >>>> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org>
> wrote:
> > >>>>
> > >>>>> Ilya,
> > >>>>>
> > >>>>> Thanks for your inputs. The reason why we decided to split Ignite
> > into
> > >>>>> several packages mimics the reason why Java community introduced
> > >> modular
> > >>>>> subsystem for JDK. That's all about size. Ignite distribution is
> too
> > >> big,
> > >>>>> and we're trying to separate it into several components so that
> > people
> > >>>> can
> > >>>>> install only the features they need.
> > >>>>>
> > >>>>> The point of a package is to ship something into root file system
> > that
> > >>>> can
> > >>>>>> be used from root file system. If cpp files require compilation we
> > >>>> should
> > >>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks.
> If
> > >>>>>> there's no mechanism to add optional libs to Ignite classpath, we
> > >>>> should
> > >>>>>> not ship optional libs. Moreover, some of 'optional' modules such
> as
> > >>>> yarn
> > >>>>>> don't make sense here because they're not supposed to be used with
> > >>>>>> standalone Ignite.
> > >>>>>
> > >>>>>
> > >>>>> Agree that we need to ship the code that is ready to be run. As for
> > the
> > >>>>> classpath thing, if an optional package is installed into the root
> > >> (core)
> > >>>>> package directory, then its jars have to be added to "ignite/libs"
> > >>>> folder.
> > >>>>> After that, the one needs to restart a cluster node, nd it will add
> > the
> > >>>>> just installed optional libs to the classpath. *Petr*, does it work
> > >> this
> > >>>>> way or can be implemented this way to address Ilya's concerns?
> > >>>>>
> > >>>>> --
> > >>>>> Denis
> > >>>>>
> > >>>>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> > >>>>> ilya.kasnacheev@gmail.com>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> > >>>>>>
> > >>>>>>> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> > >>>>> ilya.kasnacheev@gmail.com
> > >>>>>>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Moreover I did not find a way to start service if default
> > installed
> > >>>>> JVM
> > >>>>>>> is
> > >>>>>>>> Java 7 :( I understand it's EOL, still this is something that
> hit
> > >>>> me.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Apache Ignite >=2.4 does not support Java 7 - it is said in
> > >>>>>> documentation,
> > >>>>>>> DEVNOTES and even in startup scripts.
> > >>>>>>>
> > >>>>>>> I have Java 8 too, but I could not get service from package to
> > start
> > >>>>>> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for
> that
> > >>>>>> matter). Is it possible to specify it while running packaged
> Ignite?
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> apache-ignite-libs is a totally unexpected package name.
> > >>>>> apache-ignite
> > >>>>>>> core
> > >>>>>>>> doesn't depend on it. It doesn't enable anything out of the box.
> > >>>> The
> > >>>>>>>> package is huge.
> > >>>>>>>
> > >>>>>>> ‘apache-ignite-libs’ is an aggregation package (for now) for all
> > >>>>> optional
> > >>>>>>> libs we are delivering. Possibly later they will be split more
> > >>>> granular
> > >>>>>> or
> > >>>>>>> even package per lib (like php, perl, python, etc. do for their
> > >>>> libs).
> > >>>>>>> This package dependency on ‘apache-ignite-core’ may seem
> confusing
> > >>>>>> though,
> > >>>>>>> I will try to explain it in IEP at least for current iteration.
> > >>>>>>>
> > >>>>>>
> > >>>>>> Okay, but how do you add optional libs to be included into Ignite
> > >>>>> classpath
> > >>>>>> while being launched by service? Is it even possible? If it
> isn't, I
> > >>>>> think
> > >>>>>> it doesn't make sense to ship apache-ignite-libs at all.
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>> Further naming may become clear when we’ll start initiative on
> > >>>>> including
> > >>>>>>> packages to popular Linux distributions and theirs community will
> > >>>> join
> > >>>>>>> naming discussions.
> > >>>>>>>
> > >>>>>> Renaming packages once they're deployed widely will be a pain
> point
> > to
> > >>>>> out
> > >>>>>> users. Some things should probably be thought out in advance.
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> Frankly speaking, I'm not sure that improvements over Stage I
> are
> > >>>>>> enough
> > >>>>>>> as
> > >>>>>>>> of now. For demo-like activity, we can probably go with one
> > package
> > >>>>>> fits
> > >>>>>>>> all.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> The process of finding the best package architecture is
> iterative,
> > >>>> but
> > >>>>>>> previously community agreed in split design proposed for 2.5
> > release.
> > >>>>>>>
> > >>>>>>> Also, split architecture is half of proposed improvements. The
> > other
> > >>>>>> half -
> > >>>>>>> new process for deploying packages to Bintray (with virtually
> > >>>>> indefinite
> > >>>>>>> storage capabilities).
> > >>>>>>>
> > >>>>>> I think we could drop the split for now, or at least drop
> > >>>>>> apache-ignite-libs package at all. Probably also drop
> > >> apache-ignite-cpp
> > >>>>>> package and maybe apache-ignite-benchmarks.
> > >>>>>>
> > >>>>>> The point of a package is to ship something into root file system
> > that
> > >>>>> can
> > >>>>>> be used from root file system. If cpp files require compilation we
> > >>>> should
> > >>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks.
> If
> > >>>>>> there's no mechanism to add optional libs to Ignite classpath, we
> > >>>> should
> > >>>>>> not ship optional libs. Moreover, some of 'optional' modules such
> as
> > >>>> yarn
> > >>>>>> don't make sense here because they're not supposed to be used with
> > >>>>>> standalone Ignite.
> > >>>>>>
> > >>>>>> IMO it is not right to try and shove every file from Ignite
> > >>>> distribution
> > >>>>>> into some package. We should only put in packages things that can
> be
> > >>>>> used.
> > >>>>>> If something can't be used without copying it to a different FS
> > >>>> location,
> > >>>>>> it should be in examples or not packaged at all.
> > >>>>>>
> > >>>>>> In my opinion, it doesn't make sense to implement an underwhelming
> > >>>>> package
> > >>>>>> split right now just because we have agreed to have *some* package
> > >>>> split
> > >>>>> in
> > >>>>>> 2.5. Let's aim for happiness.
> > >>>>>>
> > >>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> --
> > >>>>>>>> Ilya Kasnacheev
> > >>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>>>
> > >>>>>>>> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > >>>>>>>>
> > >>>>>>>>> If someone from PMCы or Committers still sees necessity about
> > >>>>>> including
> > >>>>>>>>> these tasks into Apache Ignite 2.5 release, this is the last
> > >>>> chance
> > >>>>>> to
> > >>>>>>> do
> > >>>>>>>>> so.
> > >>>>>>>>> Otherwise this task will be moved to at 2.6 release at least,
> or
> > >>>>> even
> > >>>>>>>>> moved to backlog indefinitely.
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
> > >>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> To top new RPM architecture off, update to release process is
> > >>>>>>>> introduced
> > >>>>>>>>> — [1] [2].
> > >>>>>>>>>>
> > >>>>>>>>>> Both tasks (this one and IGNITE-7647) are ready for review and
> > >>>>>> should
> > >>>>>>>> be
> > >>>>>>>>> merged simultaneously.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > >>>>>>>>>> [2] https://github.com/apache/ignite-release/pull/1
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> > >>>>>> ilya.kasnacheev@gmail.com
> > >>>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hello!
> > >>>>>>>>>>>
> > >>>>>>>>>>> Let me share my idea of how this shoud work. Splitting
> package
> > >>>>>> into
> > >>>>>>>>>>> sub-packages should be dependency-driven.
> > >>>>>>>>>>>
> > >>>>>>>>>>> It means that all Ignite modules without dependencies or with
> > >>>>>> small
> > >>>>>>>>>>> dependencies (such as ignite-log4j) should be included in
> > >>>>>>> ignite-core.
> > >>>>>>>>> It
> > >>>>>>>>>>> doesn't make sense to make a zillion RPM packages.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Critical things like ignite-spring and ignite-indexing should
> > >>>> be
> > >>>>>> in
> > >>>>>>>>>>> ignite-core of course, even if they have dependencies.
> > >>>>> Ignite-core
> > >>>>>>>>> should
> > >>>>>>>>>>> be fully self-sufficient and feature-complete.
> > >>>>>>>>>>>
> > >>>>>>>>>>> However, e.g. .net API should probably be in a separate
> > >>>> package,
> > >>>>>>>>> because it
> > >>>>>>>>>>> should depend on mono | net-core. We may also have
> > >>>> ignite-devel
> > >>>>>>>> package
> > >>>>>>>>>>> which should include all modules which only make sense for
> > >>>>>>> developers
> > >>>>>>>>> who
> > >>>>>>>>>>> write code. Such as hibernate integration.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I'm not sure about MR modules. The main question should be,
> > >>>> does
> > >>>>>> it
> > >>>>>>>> have
> > >>>>>>>>>>> dependencies? Can it run stand-alone without writing code?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hope this helps,
> > >>>>>>>>>>>
> > >>>>>>>>>>> --
> > >>>>>>>>>>> Ilya Kasnacheev
> > >>>>>>>>>>>
> > >>>>>>>>>>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr.weider@gmail.com
> >:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi, Igniters!
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Here are some news on our RPM packages initiative.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1. I’ve finished preliminary developing of Stage II version
> > >>>> of
> > >>>>>> RPM
> > >>>>>>>>>>>> packages [1]. Main “new feature” is — split design. Also
> I’ve
> > >>>>>> added
> > >>>>>>>>>>>> package.sh script for automating package building process
> > >>>> which
> > >>>>>>> will
> > >>>>>>>>> help
> > >>>>>>>>>>>> organise corresponding builds in TC as well as simplify
> > >>>> process
> > >>>>>> for
> > >>>>>>>>>>>> developers who wishes to have custom packages.
> > >>>>>>>>>>>> PR#3703 [2] is ready for review. Denis, in order to catch up
> > >>>>> with
> > >>>>>>>>> Apache
> > >>>>>>>>>>>> Ignite 2.5 release, I’d greatly appreciate your help in
> > >>>> finding
> > >>>>>>>>> reviewer.
> > >>>>>>>>>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and
> > >>>> DEB
> > >>>>>> [4]
> > >>>>>>>>>>>> repositories on Apache Bintray. Though they are already
> > >>>>> prepared
> > >>>>>>> for
> > >>>>>>>>>>>> hosting RPM and DEB packages respectively, and there is a
> way
> > >>>>> of
> > >>>>>>>>> linking
> > >>>>>>>>>>>> them to apache.org/dist/ignite page, there is possible
> > >>>>>> alternative
> > >>>>>>>> in
> > >>>>>>>>>>>> storing there only plain directory layout corresponding to
> > >>>> each
> > >>>>>>>>> repository
> > >>>>>>>>>>>> type (RPM and DEB) and manage this layout (repodata,
> > >>>>>> distributions,
> > >>>>>>>>>>>> versions, etc.) by ourselves, having more control over
> > >>>>>> repositories
> > >>>>>>>> but
> > >>>>>>>>>>>> lacking some simplicity of deploying new releases. WDYT?
> > >>>> Should
> > >>>>>> we
> > >>>>>>>> try
> > >>>>>>>>>>>> Cassandra approach? They are storing their DEB packages as I
> > >>>>>>>> described
> > >>>>>>>>>>>> above [5].
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Also — a question arose while I was working on this issue:
> > >>>>> which
> > >>>>>>> OSes
> > >>>>>>>>> (and
> > >>>>>>>>>>>> which versions of each) are we going to support (if we are
> > >>>>> going)
> > >>>>>>> in
> > >>>>>>>>> terms
> > >>>>>>>>>>>> of step-by-step list? Currently RPM packages are tested only
> > >>>>> with
> > >>>>>>>>> latest
> > >>>>>>>>>>>> CentOS (and, respectively — RHEL), but there are a lot more
> > >>>>>>> RPM-based
> > >>>>>>>>>>>> distributives [6] some of which are more o less popular
> among
> > >>>>> OS
> > >>>>>>>>> community
> > >>>>>>>>>>>> (ALT, Fedora, openSUSE, etc.).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > >>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/3703
> > >>>>>>>>>>>> [3] https://bintray.com/apache/ignite-rpm
> > >>>>>>>>>>>> [4] https://bintray.com/apache/ignite-deb
> > >>>>>>>>>>>> [5] https://bintray.com/apache/cassandra/debian#files/
> > >>>>>>>>>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > >>>>>>>>> distributions
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr.weider@gmail.com
> >
> > >>>>>>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I suppose that most everything if not all from libs/options
> > >>>>> will
> > >>>>>>> go
> > >>>>>>>> to
> > >>>>>>>>>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > >>>>>>>>>>>>> More precise lib selection (if something from optional
> would
> > >>>>>>> better
> > >>>>>>>> to
> > >>>>>>>>>>>> have in core package) will be discussed right after
> > >>>> preliminary
> > >>>>>>> split
> > >>>>>>>>>>>> architecture agreement.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
> > >>>>> dpavlov.spb@gmail.com
> > >>>>>>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> I like idea of keeping simple system of modules, so +1
> from
> > >>>>> me.
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Where optional libs (e.g Direct IO plugin) would be
> > >>>> included,
> > >>>>>>> would
> > >>>>>>>>> it
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>> core or optional?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
> > >>>> dmagda@apache.org
> > >>>>>> :
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> How big would be a final core module?
> > >>>>>>>>>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor
> and
> > >>>>>>> create
> > >>>>>>>>> it’s
> > >>>>>>>>>>>> own
> > >>>>>>>>>>>>>>>> package.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
> > >>>>> with
> > >>>>>>> Petr
> > >>>>>>>>> and
> > >>>>>>>>>>>>>>> propose the simplest modular system:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> - core module that includes basic Ignite capabilities
> > >>>>>> including
> > >>>>>>>> SQL,
> > >>>>>>>>>>>>>>> compute grid, service grid, k/v
> > >>>>>>>>>>>>>>> - optional module hosts the rest - ML, streamers
> > >>>> integration
> > >>>>>>>> (kafka,
> > >>>>>>>>>>>>>>> flink), kubernetes, etc.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> What do you think?
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> > >>>>>>>> mr.weider@gmail.com>
> > >>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> *DEB package
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
> > >>>>> mr.weider@gmail.com>
> > >>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Considering that DEV package for now is almost platform
> > >>>>>>>>> independent
> > >>>>>>>>>>>>>>> (its
> > >>>>>>>>>>>>>>>> a java application more or less), that package will work
> > >>>>>> almost
> > >>>>>>>> on
> > >>>>>>>>> any
> > >>>>>>>>>>>>>>>> DEB-based linux, including but not limited to Ubuntu,
> > >>>>> Debian,
> > >>>>>>>> etc.
> > >>>>>>>>>>>>>>>>> The only restriction is existence of systemctl
> (systemd)
> > >>>>>>> service
> > >>>>>>>>>>>>>>> manager
> > >>>>>>>>>>>>>>>> — we are dependent on it.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Thats why, for instance, our RPM repository is called
> > >>>>> simply
> > >>>>>>>> ‘rpm’
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>> package has no arch or dist suffix — it will work on
> > >>>>> CentOS,
> > >>>>>>>> RHEL,
> > >>>>>>>>>>>>>>> Fedora,
> > >>>>>>>>>>>>>>>> etc. with presence of aforementioned systemd.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > >>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Will Debian package work for Ubuntu?
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > >>>>>>>>> mr.weider@gmail.com>
> > >>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
> > >>>> move
> > >>>>> to
> > >>>>>>>>> official
> > >>>>>>>>>>>>>>>>>>> repositories, there can be a problem from OS
> > >>>> community.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
> > >>>> for
> > >>>>>> DEB
> > >>>>>>>>> package
> > >>>>>>>>>>>>>>>> build
> > >>>>>>>>>>>>>>>>>>> (package layout / install scripts) for speeding up
> > >>>>> things
> > >>>>>>> and
> > >>>>>>>>>>>>>>> excluding
> > >>>>>>>>>>>>>>>>>>> possible duplication and desynchronisation, so its a
> > >>>>>> matter
> > >>>>>>> of
> > >>>>>>>>> ’sit
> > >>>>>>>>>>>>>>>> and do’
> > >>>>>>>>>>>>>>>>>>> rather then some technical research. Thats why I rose
> > >>>>>>>> discussion
> > >>>>>>>>>>>>>>> about
> > >>>>>>>>>>>>>>>>>>> future package architecture, so that after agreement
> > >>>> I'm
> > >>>>>> be
> > >>>>>>>>> able to
> > >>>>>>>>>>>>>>>> pack
> > >>>>>>>>>>>>>>>>>>> both RPM and DEB identically.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Yet, if you insist, I can create DEB package
> according
> > >>>>> to
> > >>>>>>>>> current
> > >>>>>>>>>>>> RPM
> > >>>>>>>>>>>>>>>>>>> layout in no time.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > >>>>>>>>>>>> dsetrakyan@apache.org>
> > >>>>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Peter,
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> I don't think the package size of 280M is going to
> > >>>> be a
> > >>>>>>>>> problem at
> > >>>>>>>>>>>>>>>> all,
> > >>>>>>>>>>>>>>>>>>> but
> > >>>>>>>>>>>>>>>>>>>> what you are suggesting can be an improvement down
> > >>>> the
> > >>>>>>> road.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> In the mean time, I think our top priority should be
> > >>>> to
> > >>>>>>>> provide
> > >>>>>>>>>>>>>>>> packages
> > >>>>>>>>>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not
> nearly
> > >>>>>>> enough.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Agree?
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> > >>>>>>>> mr.weider@gmail.com>
> > >>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Hi, Igniters!
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part
> of
> > >>>>> it,
> > >>>>>>> so
> > >>>>>>>>> I'd
> > >>>>>>>>>>>>>>> like
> > >>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>> move
> > >>>>>>>>>>>>>>>>>>>>> forward to further improve and widen AI delivery
> > >>>>> through
> > >>>>>>>>>>>> packages.
> > >>>>>>>>>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
> > >>>> weighing
> > >>>>>>> about
> > >>>>>>>>>>>> 280M+
> > >>>>>>>>>>>>>>>> and,
> > >>>>>>>>>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>>>> improve usability and significantly reduce required
> > >>>>>>> download
> > >>>>>>>>>>>>>>> sizes, I
> > >>>>>>>>>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> > >>>>>> delivery
> > >>>>>>>> as
> > >>>>>>>>>>>>>>>> follows:
> > >>>>>>>>>>>>>>>>>>>>> - CORE
> > >>>>>>>>>>>>>>>>>>>>> - bin
> > >>>>>>>>>>>>>>>>>>>>> - config
> > >>>>>>>>>>>>>>>>>>>>> - libs (!optional)
> > >>>>>>>>>>>>>>>>>>>>> - OPTIONAL LIBS
> > >>>>>>>>>>>>>>>>>>>>> - BENCHMARKS
> > >>>>>>>>>>>>>>>>>>>>> - DOCS (?)
> > >>>>>>>>>>>>>>>>>>>>> - EXAMPLES
> > >>>>>>>>>>>>>>>>>>>>> - .NET PLATFORM FILES
> > >>>>>>>>>>>>>>>>>>>>> - C++ PLATFORM FILES
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> This architecture, as I assume, will add
> flexibility
> > >>>>> (no
> > >>>>>>>>> reason
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>>>>>> download
> > >>>>>>>>>>>>>>>>>>>>> all 280M+ of binaries where you are to run only
> core
> > >>>>>> node
> > >>>>>>>>>>>>>>>> functionality)
> > >>>>>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>> maintainability (you are in full control of what is
> > >>>>>>>> installed
> > >>>>>>>>> on
> > >>>>>>>>>>>>>>> your
> > >>>>>>>>>>>>>>>>>>>>> system).
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> After successful architecture choice, same scheme
> > >>>> are
> > >>>>>>>> planned
> > >>>>>>>>> to
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>>>> used in
> > >>>>>>>>>>>>>>>>>>>>> DEB packages as well.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> --
> > >>>>>>>>>>>>>>>>>>>>> Sent from:
> > >>>>>>>> http://apache-ignite-developers.2346864.n4.nabble.
> > >>>>>>>>>>>> com/
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>
> > >>
> >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I recommend inclusion of this change so that it can make way into 2.5.
Regards,
--
Ilya Kasnacheev
2018-04-19 9:03 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> There is not much time left for Apache Ignite 2.5 release, so let’s move
> stage II of packaging architecture implementation (with additional split
> scheme discussion) to 2.6 scope.
>
> Instead, I’d like to include DEB package to Apache Ignite 2.5 release.
> Corresponding PR is already prepared [1].
>
> Also, I’ll try to prepare modifications for release procedure scripts to
> automate uploading packages to our new Bintray RPM and DEB repositories
> before the code freeze.
>
>
> [1] https://github.com/apache/ignite/pull/3869 <https://github.com/apache/
> ignite/pull/3869>
>
>
>
> > On 18 Apr 2018, at 14:44, Ilya Kasnacheev <il...@gmail.com>
> wrote:
> >
> > Copying anything manually to anywhere /usr (excluding /usr/local) is an
> > example of slackwarization that package users and creators try to avoid.
> >
> >> By linux file hierarchy convention, home dir should be in /usr/lib
> >
> > Citation needed! I bet you're interpreting it wrong, since I've listed a
> > random dir in /var/lib, and:
> > ~/w/incubator-ignite% sudo ls -al /var/lib/lightdm
> > drwxr-x--- 6 lightdm lightdm 4096 авг 14 2017 .
> > drwxr-xr-x 89 root root 4096 мар 10 22:37 ..
> > drwxrwxr-x 4 lightdm lightdm 4096 июл 19 2017 .cache
> > drwxr-xr-x 5 lightdm lightdm 4096 июл 19 2017 .config
> > drwx------ 2 lightdm lightdm 4096 апр 11 18:25 .gconf
> > drwxr-xr-x 3 lightdm lightdm 4096 июл 19 2017 .local
> > -rw------- 1 lightdm lightdm 283 апр 11 18:25 .Xauthority
> >
> > ...it definitely looks like a home dir. Which goes with additional
> benefit
> > of being writable by end-user.
> >
> > --
> > Ilya Kasnacheev
> >
> > 2018-04-16 9:22 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >
> >>
> >>
> >>> On 15 Apr 2018, at 10:19, Ilya Kasnacheev <il...@gmail.com>
> >> wrote:
> >>>
> >>> Hello!
> >>>
> >>> With existing binary archive, user can move directories from
> >>> apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to
> >> activate
> >>> them.
> >>>
> >>> But with RPM, user should not contemplate moving directories from
> >>> /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
> >>
> >> I always thought of that option as COPYING optional libs, not MOVING.
> >>
> >>
> >>>
> >>>
> >>> User lacks permissions for that operation and it would defeat the
> purpose
> >>> of having a RPM (imagine trying to upgrade Ignite RPM version with a
> >> setup
> >>> like that). "Turning distribution into Slackware" should not be an
> >> offering.
> >>
> >> Can’t imagine use case where Apache Ignite software is installed by one
> >> user, and run by another, without sudo/root permissions.
> >> Yet, ignite user’s login shell is disabled indeed and without sudo/root
> >> permissions optional libs cannot be copied.
> >> Also — the symlinks is a good idea, but I’ve already thought of it and
> I’m
> >> planning addition of special utility for enabling / disabling optional
> libs
> >> (like a2enable/a2disable) in next development iteration.
> >>
> >>
> >>>
> >>> How it could work: Imagine we create /var/lib/ignite, use it as
> >>> IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite.
> This
> >>> way, we can add and remove symlinks to modules in optional/, and thus
> >>> enable and disable them as user sees fit.
> >>
> >> By linux file hierarchy convention, home dir should be in /usr/lib.
> >> /var/lib/ is currently used for working files (MySQL alike).
> >>
> >>
> >>>
> >>> This also answers the problem of "what's IGNITE_HOME for visor launched
> >>> from /usr/bin”
> >>
> >> That is addressed in separate task and will be fixed with minor startup
> >> script redesign with universal location-independent solution.
> >>
> >>
> >>>
> >>> But obviously this will require dedication and effort.
> >>
> >> No problems with efforts after final design is discussed an agreed.
> >>
> >>
> >>>
> >>>
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>> 2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> >>>
> >>>> Current packages design (after installation) does not differ from
> binary
> >>>> archive - everything works (except necessity to run service instead
> >>>> ignite.sh) just the same way, including libs/optional.
> >>>>
> >>>> Also, there can be issues with system JDK version by default, but
> every
> >>>> problem will be in journalctl and/or /var/log, and package has strong
> >>>> dependency on any implementation of JDK 1.8.
> >>>>
> >>>>
> >>>> I am lacking enough feedback about Apache Ignite from packages from
> real
> >>>> users, don’t know real use cases so still "moving in the dark".
> >>>>
> >>>>
> >>>> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
> >>>>
> >>>>> Ilya,
> >>>>>
> >>>>> Thanks for your inputs. The reason why we decided to split Ignite
> into
> >>>>> several packages mimics the reason why Java community introduced
> >> modular
> >>>>> subsystem for JDK. That's all about size. Ignite distribution is too
> >> big,
> >>>>> and we're trying to separate it into several components so that
> people
> >>>> can
> >>>>> install only the features they need.
> >>>>>
> >>>>> The point of a package is to ship something into root file system
> that
> >>>> can
> >>>>>> be used from root file system. If cpp files require compilation we
> >>>> should
> >>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> >>>>>> there's no mechanism to add optional libs to Ignite classpath, we
> >>>> should
> >>>>>> not ship optional libs. Moreover, some of 'optional' modules such as
> >>>> yarn
> >>>>>> don't make sense here because they're not supposed to be used with
> >>>>>> standalone Ignite.
> >>>>>
> >>>>>
> >>>>> Agree that we need to ship the code that is ready to be run. As for
> the
> >>>>> classpath thing, if an optional package is installed into the root
> >> (core)
> >>>>> package directory, then its jars have to be added to "ignite/libs"
> >>>> folder.
> >>>>> After that, the one needs to restart a cluster node, nd it will add
> the
> >>>>> just installed optional libs to the classpath. *Petr*, does it work
> >> this
> >>>>> way or can be implemented this way to address Ilya's concerns?
> >>>>>
> >>>>> --
> >>>>> Denis
> >>>>>
> >>>>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> >>>>> ilya.kasnacheev@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> >>>>>>
> >>>>>>> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> >>>>> ilya.kasnacheev@gmail.com
> >>>>>>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Moreover I did not find a way to start service if default
> installed
> >>>>> JVM
> >>>>>>> is
> >>>>>>>> Java 7 :( I understand it's EOL, still this is something that hit
> >>>> me.
> >>>>>>>
> >>>>>>>
> >>>>>>> Apache Ignite >=2.4 does not support Java 7 - it is said in
> >>>>>> documentation,
> >>>>>>> DEVNOTES and even in startup scripts.
> >>>>>>>
> >>>>>>> I have Java 8 too, but I could not get service from package to
> start
> >>>>>> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> >>>>>> matter). Is it possible to specify it while running packaged Ignite?
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> apache-ignite-libs is a totally unexpected package name.
> >>>>> apache-ignite
> >>>>>>> core
> >>>>>>>> doesn't depend on it. It doesn't enable anything out of the box.
> >>>> The
> >>>>>>>> package is huge.
> >>>>>>>
> >>>>>>> ‘apache-ignite-libs’ is an aggregation package (for now) for all
> >>>>> optional
> >>>>>>> libs we are delivering. Possibly later they will be split more
> >>>> granular
> >>>>>> or
> >>>>>>> even package per lib (like php, perl, python, etc. do for their
> >>>> libs).
> >>>>>>> This package dependency on ‘apache-ignite-core’ may seem confusing
> >>>>>> though,
> >>>>>>> I will try to explain it in IEP at least for current iteration.
> >>>>>>>
> >>>>>>
> >>>>>> Okay, but how do you add optional libs to be included into Ignite
> >>>>> classpath
> >>>>>> while being launched by service? Is it even possible? If it isn't, I
> >>>>> think
> >>>>>> it doesn't make sense to ship apache-ignite-libs at all.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>> Further naming may become clear when we’ll start initiative on
> >>>>> including
> >>>>>>> packages to popular Linux distributions and theirs community will
> >>>> join
> >>>>>>> naming discussions.
> >>>>>>>
> >>>>>> Renaming packages once they're deployed widely will be a pain point
> to
> >>>>> out
> >>>>>> users. Some things should probably be thought out in advance.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> Frankly speaking, I'm not sure that improvements over Stage I are
> >>>>>> enough
> >>>>>>> as
> >>>>>>>> of now. For demo-like activity, we can probably go with one
> package
> >>>>>> fits
> >>>>>>>> all.
> >>>>>>>>
> >>>>>>>
> >>>>>>> The process of finding the best package architecture is iterative,
> >>>> but
> >>>>>>> previously community agreed in split design proposed for 2.5
> release.
> >>>>>>>
> >>>>>>> Also, split architecture is half of proposed improvements. The
> other
> >>>>>> half -
> >>>>>>> new process for deploying packages to Bintray (with virtually
> >>>>> indefinite
> >>>>>>> storage capabilities).
> >>>>>>>
> >>>>>> I think we could drop the split for now, or at least drop
> >>>>>> apache-ignite-libs package at all. Probably also drop
> >> apache-ignite-cpp
> >>>>>> package and maybe apache-ignite-benchmarks.
> >>>>>>
> >>>>>> The point of a package is to ship something into root file system
> that
> >>>>> can
> >>>>>> be used from root file system. If cpp files require compilation we
> >>>> should
> >>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> >>>>>> there's no mechanism to add optional libs to Ignite classpath, we
> >>>> should
> >>>>>> not ship optional libs. Moreover, some of 'optional' modules such as
> >>>> yarn
> >>>>>> don't make sense here because they're not supposed to be used with
> >>>>>> standalone Ignite.
> >>>>>>
> >>>>>> IMO it is not right to try and shove every file from Ignite
> >>>> distribution
> >>>>>> into some package. We should only put in packages things that can be
> >>>>> used.
> >>>>>> If something can't be used without copying it to a different FS
> >>>> location,
> >>>>>> it should be in examples or not packaged at all.
> >>>>>>
> >>>>>> In my opinion, it doesn't make sense to implement an underwhelming
> >>>>> package
> >>>>>> split right now just because we have agreed to have *some* package
> >>>> split
> >>>>> in
> >>>>>> 2.5. Let's aim for happiness.
> >>>>>>
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> --
> >>>>>>>> Ilya Kasnacheev
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>>
> >>>>>>>> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>>>>>>>
> >>>>>>>>> If someone from PMCы or Committers still sees necessity about
> >>>>>> including
> >>>>>>>>> these tasks into Apache Ignite 2.5 release, this is the last
> >>>> chance
> >>>>>> to
> >>>>>>> do
> >>>>>>>>> so.
> >>>>>>>>> Otherwise this task will be moved to at 2.6 release at least, or
> >>>>> even
> >>>>>>>>> moved to backlog indefinitely.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> To top new RPM architecture off, update to release process is
> >>>>>>>> introduced
> >>>>>>>>> — [1] [2].
> >>>>>>>>>>
> >>>>>>>>>> Both tasks (this one and IGNITE-7647) are ready for review and
> >>>>>> should
> >>>>>>>> be
> >>>>>>>>> merged simultaneously.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-8172
> >>>>>>>>>> [2] https://github.com/apache/ignite-release/pull/1
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> >>>>>> ilya.kasnacheev@gmail.com
> >>>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Hello!
> >>>>>>>>>>>
> >>>>>>>>>>> Let me share my idea of how this shoud work. Splitting package
> >>>>>> into
> >>>>>>>>>>> sub-packages should be dependency-driven.
> >>>>>>>>>>>
> >>>>>>>>>>> It means that all Ignite modules without dependencies or with
> >>>>>> small
> >>>>>>>>>>> dependencies (such as ignite-log4j) should be included in
> >>>>>>> ignite-core.
> >>>>>>>>> It
> >>>>>>>>>>> doesn't make sense to make a zillion RPM packages.
> >>>>>>>>>>>
> >>>>>>>>>>> Critical things like ignite-spring and ignite-indexing should
> >>>> be
> >>>>>> in
> >>>>>>>>>>> ignite-core of course, even if they have dependencies.
> >>>>> Ignite-core
> >>>>>>>>> should
> >>>>>>>>>>> be fully self-sufficient and feature-complete.
> >>>>>>>>>>>
> >>>>>>>>>>> However, e.g. .net API should probably be in a separate
> >>>> package,
> >>>>>>>>> because it
> >>>>>>>>>>> should depend on mono | net-core. We may also have
> >>>> ignite-devel
> >>>>>>>> package
> >>>>>>>>>>> which should include all modules which only make sense for
> >>>>>>> developers
> >>>>>>>>> who
> >>>>>>>>>>> write code. Such as hibernate integration.
> >>>>>>>>>>>
> >>>>>>>>>>> I'm not sure about MR modules. The main question should be,
> >>>> does
> >>>>>> it
> >>>>>>>> have
> >>>>>>>>>>> dependencies? Can it run stand-alone without writing code?
> >>>>>>>>>>>
> >>>>>>>>>>> Hope this helps,
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Ilya Kasnacheev
> >>>>>>>>>>>
> >>>>>>>>>>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Here are some news on our RPM packages initiative.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. I’ve finished preliminary developing of Stage II version
> >>>> of
> >>>>>> RPM
> >>>>>>>>>>>> packages [1]. Main “new feature” is — split design. Also I’ve
> >>>>>> added
> >>>>>>>>>>>> package.sh script for automating package building process
> >>>> which
> >>>>>>> will
> >>>>>>>>> help
> >>>>>>>>>>>> organise corresponding builds in TC as well as simplify
> >>>> process
> >>>>>> for
> >>>>>>>>>>>> developers who wishes to have custom packages.
> >>>>>>>>>>>> PR#3703 [2] is ready for review. Denis, in order to catch up
> >>>>> with
> >>>>>>>>> Apache
> >>>>>>>>>>>> Ignite 2.5 release, I’d greatly appreciate your help in
> >>>> finding
> >>>>>>>>> reviewer.
> >>>>>>>>>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and
> >>>> DEB
> >>>>>> [4]
> >>>>>>>>>>>> repositories on Apache Bintray. Though they are already
> >>>>> prepared
> >>>>>>> for
> >>>>>>>>>>>> hosting RPM and DEB packages respectively, and there is a way
> >>>>> of
> >>>>>>>>> linking
> >>>>>>>>>>>> them to apache.org/dist/ignite page, there is possible
> >>>>>> alternative
> >>>>>>>> in
> >>>>>>>>>>>> storing there only plain directory layout corresponding to
> >>>> each
> >>>>>>>>> repository
> >>>>>>>>>>>> type (RPM and DEB) and manage this layout (repodata,
> >>>>>> distributions,
> >>>>>>>>>>>> versions, etc.) by ourselves, having more control over
> >>>>>> repositories
> >>>>>>>> but
> >>>>>>>>>>>> lacking some simplicity of deploying new releases. WDYT?
> >>>> Should
> >>>>>> we
> >>>>>>>> try
> >>>>>>>>>>>> Cassandra approach? They are storing their DEB packages as I
> >>>>>>>> described
> >>>>>>>>>>>> above [5].
> >>>>>>>>>>>>
> >>>>>>>>>>>> Also — a question arose while I was working on this issue:
> >>>>> which
> >>>>>>> OSes
> >>>>>>>>> (and
> >>>>>>>>>>>> which versions of each) are we going to support (if we are
> >>>>> going)
> >>>>>>> in
> >>>>>>>>> terms
> >>>>>>>>>>>> of step-by-step list? Currently RPM packages are tested only
> >>>>> with
> >>>>>>>>> latest
> >>>>>>>>>>>> CentOS (and, respectively — RHEL), but there are a lot more
> >>>>>>> RPM-based
> >>>>>>>>>>>> distributives [6] some of which are more o less popular among
> >>>>> OS
> >>>>>>>>> community
> >>>>>>>>>>>> (ALT, Fedora, openSUSE, etc.).
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> >>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/3703
> >>>>>>>>>>>> [3] https://bintray.com/apache/ignite-rpm
> >>>>>>>>>>>> [4] https://bintray.com/apache/ignite-deb
> >>>>>>>>>>>> [5] https://bintray.com/apache/cassandra/debian#files/
> >>>>>>>>>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> >>>>>>>>> distributions
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I suppose that most everything if not all from libs/options
> >>>>> will
> >>>>>>> go
> >>>>>>>> to
> >>>>>>>>>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> >>>>>>>>>>>>> More precise lib selection (if something from optional would
> >>>>>>> better
> >>>>>>>> to
> >>>>>>>>>>>> have in core package) will be discussed right after
> >>>> preliminary
> >>>>>>> split
> >>>>>>>>>>>> architecture agreement.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
> >>>>> dpavlov.spb@gmail.com
> >>>>>>>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I like idea of keeping simple system of modules, so +1 from
> >>>>> me.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Where optional libs (e.g Direct IO plugin) would be
> >>>> included,
> >>>>>>> would
> >>>>>>>>> it
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>> core or optional?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
> >>>> dmagda@apache.org
> >>>>>> :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> How big would be a final core module?
> >>>>>>>>>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> >>>>>>> create
> >>>>>>>>> it’s
> >>>>>>>>>>>> own
> >>>>>>>>>>>>>>>> package.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
> >>>>> with
> >>>>>>> Petr
> >>>>>>>>> and
> >>>>>>>>>>>>>>> propose the simplest modular system:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> - core module that includes basic Ignite capabilities
> >>>>>> including
> >>>>>>>> SQL,
> >>>>>>>>>>>>>>> compute grid, service grid, k/v
> >>>>>>>>>>>>>>> - optional module hosts the rest - ML, streamers
> >>>> integration
> >>>>>>>> (kafka,
> >>>>>>>>>>>>>>> flink), kubernetes, etc.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> >>>>>>>> mr.weider@gmail.com>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> *DEB package
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
> >>>>> mr.weider@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Considering that DEV package for now is almost platform
> >>>>>>>>> independent
> >>>>>>>>>>>>>>> (its
> >>>>>>>>>>>>>>>> a java application more or less), that package will work
> >>>>>> almost
> >>>>>>>> on
> >>>>>>>>> any
> >>>>>>>>>>>>>>>> DEB-based linux, including but not limited to Ubuntu,
> >>>>> Debian,
> >>>>>>>> etc.
> >>>>>>>>>>>>>>>>> The only restriction is existence of systemctl (systemd)
> >>>>>>> service
> >>>>>>>>>>>>>>> manager
> >>>>>>>>>>>>>>>> — we are dependent on it.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Thats why, for instance, our RPM repository is called
> >>>>> simply
> >>>>>>>> ‘rpm’
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>> package has no arch or dist suffix — it will work on
> >>>>> CentOS,
> >>>>>>>> RHEL,
> >>>>>>>>>>>>>>> Fedora,
> >>>>>>>>>>>>>>>> etc. with presence of aforementioned systemd.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Will Debian package work for Ubuntu?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> >>>>>>>>> mr.weider@gmail.com>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
> >>>> move
> >>>>> to
> >>>>>>>>> official
> >>>>>>>>>>>>>>>>>>> repositories, there can be a problem from OS
> >>>> community.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
> >>>> for
> >>>>>> DEB
> >>>>>>>>> package
> >>>>>>>>>>>>>>>> build
> >>>>>>>>>>>>>>>>>>> (package layout / install scripts) for speeding up
> >>>>> things
> >>>>>>> and
> >>>>>>>>>>>>>>> excluding
> >>>>>>>>>>>>>>>>>>> possible duplication and desynchronisation, so its a
> >>>>>> matter
> >>>>>>> of
> >>>>>>>>> ’sit
> >>>>>>>>>>>>>>>> and do’
> >>>>>>>>>>>>>>>>>>> rather then some technical research. Thats why I rose
> >>>>>>>> discussion
> >>>>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>>>> future package architecture, so that after agreement
> >>>> I'm
> >>>>>> be
> >>>>>>>>> able to
> >>>>>>>>>>>>>>>> pack
> >>>>>>>>>>>>>>>>>>> both RPM and DEB identically.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Yet, if you insist, I can create DEB package according
> >>>>> to
> >>>>>>>>> current
> >>>>>>>>>>>> RPM
> >>>>>>>>>>>>>>>>>>> layout in no time.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Peter,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> I don't think the package size of 280M is going to
> >>>> be a
> >>>>>>>>> problem at
> >>>>>>>>>>>>>>>> all,
> >>>>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>>>> what you are suggesting can be an improvement down
> >>>> the
> >>>>>>> road.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> In the mean time, I think our top priority should be
> >>>> to
> >>>>>>>> provide
> >>>>>>>>>>>>>>>> packages
> >>>>>>>>>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> >>>>>>> enough.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Agree?
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> >>>>>>>> mr.weider@gmail.com>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
> >>>>> it,
> >>>>>>> so
> >>>>>>>>> I'd
> >>>>>>>>>>>>>>> like
> >>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> move
> >>>>>>>>>>>>>>>>>>>>> forward to further improve and widen AI delivery
> >>>>> through
> >>>>>>>>>>>> packages.
> >>>>>>>>>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
> >>>> weighing
> >>>>>>> about
> >>>>>>>>>>>> 280M+
> >>>>>>>>>>>>>>>> and,
> >>>>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>>>> improve usability and significantly reduce required
> >>>>>>> download
> >>>>>>>>>>>>>>> sizes, I
> >>>>>>>>>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> >>>>>> delivery
> >>>>>>>> as
> >>>>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>>>> - CORE
> >>>>>>>>>>>>>>>>>>>>> - bin
> >>>>>>>>>>>>>>>>>>>>> - config
> >>>>>>>>>>>>>>>>>>>>> - libs (!optional)
> >>>>>>>>>>>>>>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>>>>>>>>>>>>>> - BENCHMARKS
> >>>>>>>>>>>>>>>>>>>>> - DOCS (?)
> >>>>>>>>>>>>>>>>>>>>> - EXAMPLES
> >>>>>>>>>>>>>>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>>>>>>>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> This architecture, as I assume, will add flexibility
> >>>>> (no
> >>>>>>>>> reason
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> download
> >>>>>>>>>>>>>>>>>>>>> all 280M+ of binaries where you are to run only core
> >>>>>> node
> >>>>>>>>>>>>>>>> functionality)
> >>>>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>> maintainability (you are in full control of what is
> >>>>>>>> installed
> >>>>>>>>> on
> >>>>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>>>> system).
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> After successful architecture choice, same scheme
> >>>> are
> >>>>>>>> planned
> >>>>>>>>> to
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>>>> used in
> >>>>>>>>>>>>>>>>>>>>> DEB packages as well.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>>>> Sent from:
> >>>>>>>> http://apache-ignite-developers.2346864.n4.nabble.
> >>>>>>>>>>>> com/
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>
> >>
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
There is not much time left for Apache Ignite 2.5 release, so let’s move stage II of packaging architecture implementation (with additional split scheme discussion) to 2.6 scope.
Instead, I’d like to include DEB package to Apache Ignite 2.5 release.
Corresponding PR is already prepared [1].
Also, I’ll try to prepare modifications for release procedure scripts to automate uploading packages to our new Bintray RPM and DEB repositories before the code freeze.
[1] https://github.com/apache/ignite/pull/3869 <https://github.com/apache/ignite/pull/3869>
> On 18 Apr 2018, at 14:44, Ilya Kasnacheev <il...@gmail.com> wrote:
>
> Copying anything manually to anywhere /usr (excluding /usr/local) is an
> example of slackwarization that package users and creators try to avoid.
>
>> By linux file hierarchy convention, home dir should be in /usr/lib
>
> Citation needed! I bet you're interpreting it wrong, since I've listed a
> random dir in /var/lib, and:
> ~/w/incubator-ignite% sudo ls -al /var/lib/lightdm
> drwxr-x--- 6 lightdm lightdm 4096 авг 14 2017 .
> drwxr-xr-x 89 root root 4096 мар 10 22:37 ..
> drwxrwxr-x 4 lightdm lightdm 4096 июл 19 2017 .cache
> drwxr-xr-x 5 lightdm lightdm 4096 июл 19 2017 .config
> drwx------ 2 lightdm lightdm 4096 апр 11 18:25 .gconf
> drwxr-xr-x 3 lightdm lightdm 4096 июл 19 2017 .local
> -rw------- 1 lightdm lightdm 283 апр 11 18:25 .Xauthority
>
> ...it definitely looks like a home dir. Which goes with additional benefit
> of being writable by end-user.
>
> --
> Ilya Kasnacheev
>
> 2018-04-16 9:22 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
>>
>>
>>> On 15 Apr 2018, at 10:19, Ilya Kasnacheev <il...@gmail.com>
>> wrote:
>>>
>>> Hello!
>>>
>>> With existing binary archive, user can move directories from
>>> apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to
>> activate
>>> them.
>>>
>>> But with RPM, user should not contemplate moving directories from
>>> /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
>>
>> I always thought of that option as COPYING optional libs, not MOVING.
>>
>>
>>>
>>>
>>> User lacks permissions for that operation and it would defeat the purpose
>>> of having a RPM (imagine trying to upgrade Ignite RPM version with a
>> setup
>>> like that). "Turning distribution into Slackware" should not be an
>> offering.
>>
>> Can’t imagine use case where Apache Ignite software is installed by one
>> user, and run by another, without sudo/root permissions.
>> Yet, ignite user’s login shell is disabled indeed and without sudo/root
>> permissions optional libs cannot be copied.
>> Also — the symlinks is a good idea, but I’ve already thought of it and I’m
>> planning addition of special utility for enabling / disabling optional libs
>> (like a2enable/a2disable) in next development iteration.
>>
>>
>>>
>>> How it could work: Imagine we create /var/lib/ignite, use it as
>>> IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
>>> way, we can add and remove symlinks to modules in optional/, and thus
>>> enable and disable them as user sees fit.
>>
>> By linux file hierarchy convention, home dir should be in /usr/lib.
>> /var/lib/ is currently used for working files (MySQL alike).
>>
>>
>>>
>>> This also answers the problem of "what's IGNITE_HOME for visor launched
>>> from /usr/bin”
>>
>> That is addressed in separate task and will be fixed with minor startup
>> script redesign with universal location-independent solution.
>>
>>
>>>
>>> But obviously this will require dedication and effort.
>>
>> No problems with efforts after final design is discussed an agreed.
>>
>>
>>>
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>> 2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
>>>
>>>> Current packages design (after installation) does not differ from binary
>>>> archive - everything works (except necessity to run service instead
>>>> ignite.sh) just the same way, including libs/optional.
>>>>
>>>> Also, there can be issues with system JDK version by default, but every
>>>> problem will be in journalctl and/or /var/log, and package has strong
>>>> dependency on any implementation of JDK 1.8.
>>>>
>>>>
>>>> I am lacking enough feedback about Apache Ignite from packages from real
>>>> users, don’t know real use cases so still "moving in the dark".
>>>>
>>>>
>>>> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>>> Ilya,
>>>>>
>>>>> Thanks for your inputs. The reason why we decided to split Ignite into
>>>>> several packages mimics the reason why Java community introduced
>> modular
>>>>> subsystem for JDK. That's all about size. Ignite distribution is too
>> big,
>>>>> and we're trying to separate it into several components so that people
>>>> can
>>>>> install only the features they need.
>>>>>
>>>>> The point of a package is to ship something into root file system that
>>>> can
>>>>>> be used from root file system. If cpp files require compilation we
>>>> should
>>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
>>>>>> there's no mechanism to add optional libs to Ignite classpath, we
>>>> should
>>>>>> not ship optional libs. Moreover, some of 'optional' modules such as
>>>> yarn
>>>>>> don't make sense here because they're not supposed to be used with
>>>>>> standalone Ignite.
>>>>>
>>>>>
>>>>> Agree that we need to ship the code that is ready to be run. As for the
>>>>> classpath thing, if an optional package is installed into the root
>> (core)
>>>>> package directory, then its jars have to be added to "ignite/libs"
>>>> folder.
>>>>> After that, the one needs to restart a cluster node, nd it will add the
>>>>> just installed optional libs to the classpath. *Petr*, does it work
>> this
>>>>> way or can be implemented this way to address Ilya's concerns?
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
>>>>> ilya.kasnacheev@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
>>>>>>
>>>>>>> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
>>>>> ilya.kasnacheev@gmail.com
>>>>>>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>> Moreover I did not find a way to start service if default installed
>>>>> JVM
>>>>>>> is
>>>>>>>> Java 7 :( I understand it's EOL, still this is something that hit
>>>> me.
>>>>>>>
>>>>>>>
>>>>>>> Apache Ignite >=2.4 does not support Java 7 - it is said in
>>>>>> documentation,
>>>>>>> DEVNOTES and even in startup scripts.
>>>>>>>
>>>>>>> I have Java 8 too, but I could not get service from package to start
>>>>>> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
>>>>>> matter). Is it possible to specify it while running packaged Ignite?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> apache-ignite-libs is a totally unexpected package name.
>>>>> apache-ignite
>>>>>>> core
>>>>>>>> doesn't depend on it. It doesn't enable anything out of the box.
>>>> The
>>>>>>>> package is huge.
>>>>>>>
>>>>>>> ‘apache-ignite-libs’ is an aggregation package (for now) for all
>>>>> optional
>>>>>>> libs we are delivering. Possibly later they will be split more
>>>> granular
>>>>>> or
>>>>>>> even package per lib (like php, perl, python, etc. do for their
>>>> libs).
>>>>>>> This package dependency on ‘apache-ignite-core’ may seem confusing
>>>>>> though,
>>>>>>> I will try to explain it in IEP at least for current iteration.
>>>>>>>
>>>>>>
>>>>>> Okay, but how do you add optional libs to be included into Ignite
>>>>> classpath
>>>>>> while being launched by service? Is it even possible? If it isn't, I
>>>>> think
>>>>>> it doesn't make sense to ship apache-ignite-libs at all.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Further naming may become clear when we’ll start initiative on
>>>>> including
>>>>>>> packages to popular Linux distributions and theirs community will
>>>> join
>>>>>>> naming discussions.
>>>>>>>
>>>>>> Renaming packages once they're deployed widely will be a pain point to
>>>>> out
>>>>>> users. Some things should probably be thought out in advance.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Frankly speaking, I'm not sure that improvements over Stage I are
>>>>>> enough
>>>>>>> as
>>>>>>>> of now. For demo-like activity, we can probably go with one package
>>>>>> fits
>>>>>>>> all.
>>>>>>>>
>>>>>>>
>>>>>>> The process of finding the best package architecture is iterative,
>>>> but
>>>>>>> previously community agreed in split design proposed for 2.5 release.
>>>>>>>
>>>>>>> Also, split architecture is half of proposed improvements. The other
>>>>>> half -
>>>>>>> new process for deploying packages to Bintray (with virtually
>>>>> indefinite
>>>>>>> storage capabilities).
>>>>>>>
>>>>>> I think we could drop the split for now, or at least drop
>>>>>> apache-ignite-libs package at all. Probably also drop
>> apache-ignite-cpp
>>>>>> package and maybe apache-ignite-benchmarks.
>>>>>>
>>>>>> The point of a package is to ship something into root file system that
>>>>> can
>>>>>> be used from root file system. If cpp files require compilation we
>>>> should
>>>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
>>>>>> there's no mechanism to add optional libs to Ignite classpath, we
>>>> should
>>>>>> not ship optional libs. Moreover, some of 'optional' modules such as
>>>> yarn
>>>>>> don't make sense here because they're not supposed to be used with
>>>>>> standalone Ignite.
>>>>>>
>>>>>> IMO it is not right to try and shove every file from Ignite
>>>> distribution
>>>>>> into some package. We should only put in packages things that can be
>>>>> used.
>>>>>> If something can't be used without copying it to a different FS
>>>> location,
>>>>>> it should be in examples or not packaged at all.
>>>>>>
>>>>>> In my opinion, it doesn't make sense to implement an underwhelming
>>>>> package
>>>>>> split right now just because we have agreed to have *some* package
>>>> split
>>>>> in
>>>>>> 2.5. Let's aim for happiness.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ilya Kasnacheev
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>>
>>>>>>>> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>
>>>>>>>>> If someone from PMCы or Committers still sees necessity about
>>>>>> including
>>>>>>>>> these tasks into Apache Ignite 2.5 release, this is the last
>>>> chance
>>>>>> to
>>>>>>> do
>>>>>>>>> so.
>>>>>>>>> Otherwise this task will be moved to at 2.6 release at least, or
>>>>> even
>>>>>>>>> moved to backlog indefinitely.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> To top new RPM architecture off, update to release process is
>>>>>>>> introduced
>>>>>>>>> — [1] [2].
>>>>>>>>>>
>>>>>>>>>> Both tasks (this one and IGNITE-7647) are ready for review and
>>>>>> should
>>>>>>>> be
>>>>>>>>> merged simultaneously.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-8172
>>>>>>>>>> [2] https://github.com/apache/ignite-release/pull/1
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
>>>>>> ilya.kasnacheev@gmail.com
>>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hello!
>>>>>>>>>>>
>>>>>>>>>>> Let me share my idea of how this shoud work. Splitting package
>>>>>> into
>>>>>>>>>>> sub-packages should be dependency-driven.
>>>>>>>>>>>
>>>>>>>>>>> It means that all Ignite modules without dependencies or with
>>>>>> small
>>>>>>>>>>> dependencies (such as ignite-log4j) should be included in
>>>>>>> ignite-core.
>>>>>>>>> It
>>>>>>>>>>> doesn't make sense to make a zillion RPM packages.
>>>>>>>>>>>
>>>>>>>>>>> Critical things like ignite-spring and ignite-indexing should
>>>> be
>>>>>> in
>>>>>>>>>>> ignite-core of course, even if they have dependencies.
>>>>> Ignite-core
>>>>>>>>> should
>>>>>>>>>>> be fully self-sufficient and feature-complete.
>>>>>>>>>>>
>>>>>>>>>>> However, e.g. .net API should probably be in a separate
>>>> package,
>>>>>>>>> because it
>>>>>>>>>>> should depend on mono | net-core. We may also have
>>>> ignite-devel
>>>>>>>> package
>>>>>>>>>>> which should include all modules which only make sense for
>>>>>>> developers
>>>>>>>>> who
>>>>>>>>>>> write code. Such as hibernate integration.
>>>>>>>>>>>
>>>>>>>>>>> I'm not sure about MR modules. The main question should be,
>>>> does
>>>>>> it
>>>>>>>> have
>>>>>>>>>>> dependencies? Can it run stand-alone without writing code?
>>>>>>>>>>>
>>>>>>>>>>> Hope this helps,
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>
>>>>>>>>>>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Here are some news on our RPM packages initiative.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. I’ve finished preliminary developing of Stage II version
>>>> of
>>>>>> RPM
>>>>>>>>>>>> packages [1]. Main “new feature” is — split design. Also I’ve
>>>>>> added
>>>>>>>>>>>> package.sh script for automating package building process
>>>> which
>>>>>>> will
>>>>>>>>> help
>>>>>>>>>>>> organise corresponding builds in TC as well as simplify
>>>> process
>>>>>> for
>>>>>>>>>>>> developers who wishes to have custom packages.
>>>>>>>>>>>> PR#3703 [2] is ready for review. Denis, in order to catch up
>>>>> with
>>>>>>>>> Apache
>>>>>>>>>>>> Ignite 2.5 release, I’d greatly appreciate your help in
>>>> finding
>>>>>>>>> reviewer.
>>>>>>>>>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and
>>>> DEB
>>>>>> [4]
>>>>>>>>>>>> repositories on Apache Bintray. Though they are already
>>>>> prepared
>>>>>>> for
>>>>>>>>>>>> hosting RPM and DEB packages respectively, and there is a way
>>>>> of
>>>>>>>>> linking
>>>>>>>>>>>> them to apache.org/dist/ignite page, there is possible
>>>>>> alternative
>>>>>>>> in
>>>>>>>>>>>> storing there only plain directory layout corresponding to
>>>> each
>>>>>>>>> repository
>>>>>>>>>>>> type (RPM and DEB) and manage this layout (repodata,
>>>>>> distributions,
>>>>>>>>>>>> versions, etc.) by ourselves, having more control over
>>>>>> repositories
>>>>>>>> but
>>>>>>>>>>>> lacking some simplicity of deploying new releases. WDYT?
>>>> Should
>>>>>> we
>>>>>>>> try
>>>>>>>>>>>> Cassandra approach? They are storing their DEB packages as I
>>>>>>>> described
>>>>>>>>>>>> above [5].
>>>>>>>>>>>>
>>>>>>>>>>>> Also — a question arose while I was working on this issue:
>>>>> which
>>>>>>> OSes
>>>>>>>>> (and
>>>>>>>>>>>> which versions of each) are we going to support (if we are
>>>>> going)
>>>>>>> in
>>>>>>>>> terms
>>>>>>>>>>>> of step-by-step list? Currently RPM packages are tested only
>>>>> with
>>>>>>>>> latest
>>>>>>>>>>>> CentOS (and, respectively — RHEL), but there are a lot more
>>>>>>> RPM-based
>>>>>>>>>>>> distributives [6] some of which are more o less popular among
>>>>> OS
>>>>>>>>> community
>>>>>>>>>>>> (ALT, Fedora, openSUSE, etc.).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/3703
>>>>>>>>>>>> [3] https://bintray.com/apache/ignite-rpm
>>>>>>>>>>>> [4] https://bintray.com/apache/ignite-deb
>>>>>>>>>>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>>>>>>>>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
>>>>>>>>> distributions
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I suppose that most everything if not all from libs/options
>>>>> will
>>>>>>> go
>>>>>>>> to
>>>>>>>>>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>>>>>>>>>>> More precise lib selection (if something from optional would
>>>>>>> better
>>>>>>>> to
>>>>>>>>>>>> have in core package) will be discussed right after
>>>> preliminary
>>>>>>> split
>>>>>>>>>>>> architecture agreement.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
>>>>> dpavlov.spb@gmail.com
>>>>>>>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I like idea of keeping simple system of modules, so +1 from
>>>>> me.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Where optional libs (e.g Direct IO plugin) would be
>>>> included,
>>>>>>> would
>>>>>>>>> it
>>>>>>>>>>>> be
>>>>>>>>>>>>>> core or optional?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
>>>> dmagda@apache.org
>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> How big would be a final core module?
>>>>>>>>>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
>>>>>>> create
>>>>>>>>> it’s
>>>>>>>>>>>> own
>>>>>>>>>>>>>>>> package.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
>>>>> with
>>>>>>> Petr
>>>>>>>>> and
>>>>>>>>>>>>>>> propose the simplest modular system:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> - core module that includes basic Ignite capabilities
>>>>>> including
>>>>>>>> SQL,
>>>>>>>>>>>>>>> compute grid, service grid, k/v
>>>>>>>>>>>>>>> - optional module hosts the rest - ML, streamers
>>>> integration
>>>>>>>> (kafka,
>>>>>>>>>>>>>>> flink), kubernetes, etc.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
>>>>>>>> mr.weider@gmail.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *DEB package
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
>>>>> mr.weider@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Considering that DEV package for now is almost platform
>>>>>>>>> independent
>>>>>>>>>>>>>>> (its
>>>>>>>>>>>>>>>> a java application more or less), that package will work
>>>>>> almost
>>>>>>>> on
>>>>>>>>> any
>>>>>>>>>>>>>>>> DEB-based linux, including but not limited to Ubuntu,
>>>>> Debian,
>>>>>>>> etc.
>>>>>>>>>>>>>>>>> The only restriction is existence of systemctl (systemd)
>>>>>>> service
>>>>>>>>>>>>>>> manager
>>>>>>>>>>>>>>>> — we are dependent on it.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thats why, for instance, our RPM repository is called
>>>>> simply
>>>>>>>> ‘rpm’
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> package has no arch or dist suffix — it will work on
>>>>> CentOS,
>>>>>>>> RHEL,
>>>>>>>>>>>>>>> Fedora,
>>>>>>>>>>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
>>>>>>>>> mr.weider@gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
>>>> move
>>>>> to
>>>>>>>>> official
>>>>>>>>>>>>>>>>>>> repositories, there can be a problem from OS
>>>> community.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
>>>> for
>>>>>> DEB
>>>>>>>>> package
>>>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>>>>>> (package layout / install scripts) for speeding up
>>>>> things
>>>>>>> and
>>>>>>>>>>>>>>> excluding
>>>>>>>>>>>>>>>>>>> possible duplication and desynchronisation, so its a
>>>>>> matter
>>>>>>> of
>>>>>>>>> ’sit
>>>>>>>>>>>>>>>> and do’
>>>>>>>>>>>>>>>>>>> rather then some technical research. Thats why I rose
>>>>>>>> discussion
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>>> future package architecture, so that after agreement
>>>> I'm
>>>>>> be
>>>>>>>>> able to
>>>>>>>>>>>>>>>> pack
>>>>>>>>>>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Yet, if you insist, I can create DEB package according
>>>>> to
>>>>>>>>> current
>>>>>>>>>>>> RPM
>>>>>>>>>>>>>>>>>>> layout in no time.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Peter,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> I don't think the package size of 280M is going to
>>>> be a
>>>>>>>>> problem at
>>>>>>>>>>>>>>>> all,
>>>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>>>> what you are suggesting can be an improvement down
>>>> the
>>>>>>> road.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> In the mean time, I think our top priority should be
>>>> to
>>>>>>>> provide
>>>>>>>>>>>>>>>> packages
>>>>>>>>>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
>>>>>>> enough.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Agree?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
>>>>>>>> mr.weider@gmail.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
>>>>> it,
>>>>>>> so
>>>>>>>>> I'd
>>>>>>>>>>>>>>> like
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> move
>>>>>>>>>>>>>>>>>>>>> forward to further improve and widen AI delivery
>>>>> through
>>>>>>>>>>>> packages.
>>>>>>>>>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
>>>> weighing
>>>>>>> about
>>>>>>>>>>>> 280M+
>>>>>>>>>>>>>>>> and,
>>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> improve usability and significantly reduce required
>>>>>>> download
>>>>>>>>>>>>>>> sizes, I
>>>>>>>>>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
>>>>>> delivery
>>>>>>>> as
>>>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>>>> - CORE
>>>>>>>>>>>>>>>>>>>>> - bin
>>>>>>>>>>>>>>>>>>>>> - config
>>>>>>>>>>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This architecture, as I assume, will add flexibility
>>>>> (no
>>>>>>>>> reason
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> download
>>>>>>>>>>>>>>>>>>>>> all 280M+ of binaries where you are to run only core
>>>>>> node
>>>>>>>>>>>>>>>> functionality)
>>>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> maintainability (you are in full control of what is
>>>>>>>> installed
>>>>>>>>> on
>>>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>>>> system).
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> After successful architecture choice, same scheme
>>>> are
>>>>>>>> planned
>>>>>>>>> to
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>>>> used in
>>>>>>>>>>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>>>> Sent from:
>>>>>>>> http://apache-ignite-developers.2346864.n4.nabble.
>>>>>>>>>>>> com/
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
Copying anything manually to anywhere /usr (excluding /usr/local) is an
example of slackwarization that package users and creators try to avoid.
> By linux file hierarchy convention, home dir should be in /usr/lib
Citation needed! I bet you're interpreting it wrong, since I've listed a
random dir in /var/lib, and:
~/w/incubator-ignite% sudo ls -al /var/lib/lightdm
drwxr-x--- 6 lightdm lightdm 4096 авг 14 2017 .
drwxr-xr-x 89 root root 4096 мар 10 22:37 ..
drwxrwxr-x 4 lightdm lightdm 4096 июл 19 2017 .cache
drwxr-xr-x 5 lightdm lightdm 4096 июл 19 2017 .config
drwx------ 2 lightdm lightdm 4096 апр 11 18:25 .gconf
drwxr-xr-x 3 lightdm lightdm 4096 июл 19 2017 .local
-rw------- 1 lightdm lightdm 283 апр 11 18:25 .Xauthority
...it definitely looks like a home dir. Which goes with additional benefit
of being writable by end-user.
--
Ilya Kasnacheev
2018-04-16 9:22 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
>
> > On 15 Apr 2018, at 10:19, Ilya Kasnacheev <il...@gmail.com>
> wrote:
> >
> > Hello!
> >
> > With existing binary archive, user can move directories from
> > apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to
> activate
> > them.
> >
> > But with RPM, user should not contemplate moving directories from
> > /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
>
> I always thought of that option as COPYING optional libs, not MOVING.
>
>
> >
> >
> > User lacks permissions for that operation and it would defeat the purpose
> > of having a RPM (imagine trying to upgrade Ignite RPM version with a
> setup
> > like that). "Turning distribution into Slackware" should not be an
> offering.
>
> Can’t imagine use case where Apache Ignite software is installed by one
> user, and run by another, without sudo/root permissions.
> Yet, ignite user’s login shell is disabled indeed and without sudo/root
> permissions optional libs cannot be copied.
> Also — the symlinks is a good idea, but I’ve already thought of it and I’m
> planning addition of special utility for enabling / disabling optional libs
> (like a2enable/a2disable) in next development iteration.
>
>
> >
> > How it could work: Imagine we create /var/lib/ignite, use it as
> > IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
> > way, we can add and remove symlinks to modules in optional/, and thus
> > enable and disable them as user sees fit.
>
> By linux file hierarchy convention, home dir should be in /usr/lib.
> /var/lib/ is currently used for working files (MySQL alike).
>
>
> >
> > This also answers the problem of "what's IGNITE_HOME for visor launched
> > from /usr/bin”
>
> That is addressed in separate task and will be fixed with minor startup
> script redesign with universal location-independent solution.
>
>
> >
> > But obviously this will require dedication and effort.
>
> No problems with efforts after final design is discussed an agreed.
>
>
> >
> >
> > --
> > Ilya Kasnacheev
> >
> > 2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> >
> >> Current packages design (after installation) does not differ from binary
> >> archive - everything works (except necessity to run service instead
> >> ignite.sh) just the same way, including libs/optional.
> >>
> >> Also, there can be issues with system JDK version by default, but every
> >> problem will be in journalctl and/or /var/log, and package has strong
> >> dependency on any implementation of JDK 1.8.
> >>
> >>
> >> I am lacking enough feedback about Apache Ignite from packages from real
> >> users, don’t know real use cases so still "moving in the dark".
> >>
> >>
> >> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
> >>
> >>> Ilya,
> >>>
> >>> Thanks for your inputs. The reason why we decided to split Ignite into
> >>> several packages mimics the reason why Java community introduced
> modular
> >>> subsystem for JDK. That's all about size. Ignite distribution is too
> big,
> >>> and we're trying to separate it into several components so that people
> >> can
> >>> install only the features they need.
> >>>
> >>> The point of a package is to ship something into root file system that
> >> can
> >>>> be used from root file system. If cpp files require compilation we
> >> should
> >>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> >>>> there's no mechanism to add optional libs to Ignite classpath, we
> >> should
> >>>> not ship optional libs. Moreover, some of 'optional' modules such as
> >> yarn
> >>>> don't make sense here because they're not supposed to be used with
> >>>> standalone Ignite.
> >>>
> >>>
> >>> Agree that we need to ship the code that is ready to be run. As for the
> >>> classpath thing, if an optional package is installed into the root
> (core)
> >>> package directory, then its jars have to be added to "ignite/libs"
> >> folder.
> >>> After that, the one needs to restart a cluster node, nd it will add the
> >>> just installed optional libs to the classpath. *Petr*, does it work
> this
> >>> way or can be implemented this way to address Ilya's concerns?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> >>> ilya.kasnacheev@gmail.com>
> >>> wrote:
> >>>
> >>>> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> >>>>
> >>>>> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> >>> ilya.kasnacheev@gmail.com
> >>>>>
> >>>>> wrote:
> >>>>>
> >>>>>>
> >>>>>> Moreover I did not find a way to start service if default installed
> >>> JVM
> >>>>> is
> >>>>>> Java 7 :( I understand it's EOL, still this is something that hit
> >> me.
> >>>>>
> >>>>>
> >>>>> Apache Ignite >=2.4 does not support Java 7 - it is said in
> >>>> documentation,
> >>>>> DEVNOTES and even in startup scripts.
> >>>>>
> >>>>> I have Java 8 too, but I could not get service from package to start
> >>>> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> >>>> matter). Is it possible to specify it while running packaged Ignite?
> >>>>
> >>>>
> >>>>>
> >>>>>>
> >>>>>> apache-ignite-libs is a totally unexpected package name.
> >>> apache-ignite
> >>>>> core
> >>>>>> doesn't depend on it. It doesn't enable anything out of the box.
> >> The
> >>>>>> package is huge.
> >>>>>
> >>>>> ‘apache-ignite-libs’ is an aggregation package (for now) for all
> >>> optional
> >>>>> libs we are delivering. Possibly later they will be split more
> >> granular
> >>>> or
> >>>>> even package per lib (like php, perl, python, etc. do for their
> >> libs).
> >>>>> This package dependency on ‘apache-ignite-core’ may seem confusing
> >>>> though,
> >>>>> I will try to explain it in IEP at least for current iteration.
> >>>>>
> >>>>
> >>>> Okay, but how do you add optional libs to be included into Ignite
> >>> classpath
> >>>> while being launched by service? Is it even possible? If it isn't, I
> >>> think
> >>>> it doesn't make sense to ship apache-ignite-libs at all.
> >>>>
> >>>>
> >>>>>
> >>>>> Further naming may become clear when we’ll start initiative on
> >>> including
> >>>>> packages to popular Linux distributions and theirs community will
> >> join
> >>>>> naming discussions.
> >>>>>
> >>>> Renaming packages once they're deployed widely will be a pain point to
> >>> out
> >>>> users. Some things should probably be thought out in advance.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> Frankly speaking, I'm not sure that improvements over Stage I are
> >>>> enough
> >>>>> as
> >>>>>> of now. For demo-like activity, we can probably go with one package
> >>>> fits
> >>>>>> all.
> >>>>>>
> >>>>>
> >>>>> The process of finding the best package architecture is iterative,
> >> but
> >>>>> previously community agreed in split design proposed for 2.5 release.
> >>>>>
> >>>>> Also, split architecture is half of proposed improvements. The other
> >>>> half -
> >>>>> new process for deploying packages to Bintray (with virtually
> >>> indefinite
> >>>>> storage capabilities).
> >>>>>
> >>>> I think we could drop the split for now, or at least drop
> >>>> apache-ignite-libs package at all. Probably also drop
> apache-ignite-cpp
> >>>> package and maybe apache-ignite-benchmarks.
> >>>>
> >>>> The point of a package is to ship something into root file system that
> >>> can
> >>>> be used from root file system. If cpp files require compilation we
> >> should
> >>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> >>>> there's no mechanism to add optional libs to Ignite classpath, we
> >> should
> >>>> not ship optional libs. Moreover, some of 'optional' modules such as
> >> yarn
> >>>> don't make sense here because they're not supposed to be used with
> >>>> standalone Ignite.
> >>>>
> >>>> IMO it is not right to try and shove every file from Ignite
> >> distribution
> >>>> into some package. We should only put in packages things that can be
> >>> used.
> >>>> If something can't be used without copying it to a different FS
> >> location,
> >>>> it should be in examples or not packaged at all.
> >>>>
> >>>> In my opinion, it doesn't make sense to implement an underwhelming
> >>> package
> >>>> split right now just because we have agreed to have *some* package
> >> split
> >>> in
> >>>> 2.5. Let's aim for happiness.
> >>>>
> >>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Ilya Kasnacheev
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>>>
> >>>>>> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>>>>>
> >>>>>>> If someone from PMCы or Committers still sees necessity about
> >>>> including
> >>>>>>> these tasks into Apache Ignite 2.5 release, this is the last
> >> chance
> >>>> to
> >>>>> do
> >>>>>>> so.
> >>>>>>> Otherwise this task will be moved to at 2.6 release at least, or
> >>> even
> >>>>>>> moved to backlog indefinitely.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>> To top new RPM architecture off, update to release process is
> >>>>>> introduced
> >>>>>>> — [1] [2].
> >>>>>>>>
> >>>>>>>> Both tasks (this one and IGNITE-7647) are ready for review and
> >>>> should
> >>>>>> be
> >>>>>>> merged simultaneously.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-8172
> >>>>>>>> [2] https://github.com/apache/ignite-release/pull/1
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> >>>> ilya.kasnacheev@gmail.com
> >>>>>>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Hello!
> >>>>>>>>>
> >>>>>>>>> Let me share my idea of how this shoud work. Splitting package
> >>>> into
> >>>>>>>>> sub-packages should be dependency-driven.
> >>>>>>>>>
> >>>>>>>>> It means that all Ignite modules without dependencies or with
> >>>> small
> >>>>>>>>> dependencies (such as ignite-log4j) should be included in
> >>>>> ignite-core.
> >>>>>>> It
> >>>>>>>>> doesn't make sense to make a zillion RPM packages.
> >>>>>>>>>
> >>>>>>>>> Critical things like ignite-spring and ignite-indexing should
> >> be
> >>>> in
> >>>>>>>>> ignite-core of course, even if they have dependencies.
> >>> Ignite-core
> >>>>>>> should
> >>>>>>>>> be fully self-sufficient and feature-complete.
> >>>>>>>>>
> >>>>>>>>> However, e.g. .net API should probably be in a separate
> >> package,
> >>>>>>> because it
> >>>>>>>>> should depend on mono | net-core. We may also have
> >> ignite-devel
> >>>>>> package
> >>>>>>>>> which should include all modules which only make sense for
> >>>>> developers
> >>>>>>> who
> >>>>>>>>> write code. Such as hibernate integration.
> >>>>>>>>>
> >>>>>>>>> I'm not sure about MR modules. The main question should be,
> >> does
> >>>> it
> >>>>>> have
> >>>>>>>>> dependencies? Can it run stand-alone without writing code?
> >>>>>>>>>
> >>>>>>>>> Hope this helps,
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Ilya Kasnacheev
> >>>>>>>>>
> >>>>>>>>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Here are some news on our RPM packages initiative.
> >>>>>>>>>>
> >>>>>>>>>> 1. I’ve finished preliminary developing of Stage II version
> >> of
> >>>> RPM
> >>>>>>>>>> packages [1]. Main “new feature” is — split design. Also I’ve
> >>>> added
> >>>>>>>>>> package.sh script for automating package building process
> >> which
> >>>>> will
> >>>>>>> help
> >>>>>>>>>> organise corresponding builds in TC as well as simplify
> >> process
> >>>> for
> >>>>>>>>>> developers who wishes to have custom packages.
> >>>>>>>>>> PR#3703 [2] is ready for review. Denis, in order to catch up
> >>> with
> >>>>>>> Apache
> >>>>>>>>>> Ignite 2.5 release, I’d greatly appreciate your help in
> >> finding
> >>>>>>> reviewer.
> >>>>>>>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and
> >> DEB
> >>>> [4]
> >>>>>>>>>> repositories on Apache Bintray. Though they are already
> >>> prepared
> >>>>> for
> >>>>>>>>>> hosting RPM and DEB packages respectively, and there is a way
> >>> of
> >>>>>>> linking
> >>>>>>>>>> them to apache.org/dist/ignite page, there is possible
> >>>> alternative
> >>>>>> in
> >>>>>>>>>> storing there only plain directory layout corresponding to
> >> each
> >>>>>>> repository
> >>>>>>>>>> type (RPM and DEB) and manage this layout (repodata,
> >>>> distributions,
> >>>>>>>>>> versions, etc.) by ourselves, having more control over
> >>>> repositories
> >>>>>> but
> >>>>>>>>>> lacking some simplicity of deploying new releases. WDYT?
> >> Should
> >>>> we
> >>>>>> try
> >>>>>>>>>> Cassandra approach? They are storing their DEB packages as I
> >>>>>> described
> >>>>>>>>>> above [5].
> >>>>>>>>>>
> >>>>>>>>>> Also — a question arose while I was working on this issue:
> >>> which
> >>>>> OSes
> >>>>>>> (and
> >>>>>>>>>> which versions of each) are we going to support (if we are
> >>> going)
> >>>>> in
> >>>>>>> terms
> >>>>>>>>>> of step-by-step list? Currently RPM packages are tested only
> >>> with
> >>>>>>> latest
> >>>>>>>>>> CentOS (and, respectively — RHEL), but there are a lot more
> >>>>> RPM-based
> >>>>>>>>>> distributives [6] some of which are more o less popular among
> >>> OS
> >>>>>>> community
> >>>>>>>>>> (ALT, Fedora, openSUSE, etc.).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> >>>>>>>>>> [2] https://github.com/apache/ignite/pull/3703
> >>>>>>>>>> [3] https://bintray.com/apache/ignite-rpm
> >>>>>>>>>> [4] https://bintray.com/apache/ignite-deb
> >>>>>>>>>> [5] https://bintray.com/apache/cassandra/debian#files/
> >>>>>>>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> >>>>>>> distributions
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> I suppose that most everything if not all from libs/options
> >>> will
> >>>>> go
> >>>>>> to
> >>>>>>>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> >>>>>>>>>>> More precise lib selection (if something from optional would
> >>>>> better
> >>>>>> to
> >>>>>>>>>> have in core package) will be discussed right after
> >> preliminary
> >>>>> split
> >>>>>>>>>> architecture agreement.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
> >>> dpavlov.spb@gmail.com
> >>>>>
> >>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> I like idea of keeping simple system of modules, so +1 from
> >>> me.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Where optional libs (e.g Direct IO plugin) would be
> >> included,
> >>>>> would
> >>>>>>> it
> >>>>>>>>>> be
> >>>>>>>>>>>> core or optional?
> >>>>>>>>>>>>
> >>>>>>>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
> >> dmagda@apache.org
> >>>> :
> >>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> How big would be a final core module?
> >>>>>>>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> >>>>> create
> >>>>>>> it’s
> >>>>>>>>>> own
> >>>>>>>>>>>>>> package.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
> >>> with
> >>>>> Petr
> >>>>>>> and
> >>>>>>>>>>>>> propose the simplest modular system:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> - core module that includes basic Ignite capabilities
> >>>> including
> >>>>>> SQL,
> >>>>>>>>>>>>> compute grid, service grid, k/v
> >>>>>>>>>>>>> - optional module hosts the rest - ML, streamers
> >> integration
> >>>>>> (kafka,
> >>>>>>>>>>>>> flink), kubernetes, etc.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> What do you think?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> >>>>>> mr.weider@gmail.com>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> *DEB package
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
> >>> mr.weider@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Considering that DEV package for now is almost platform
> >>>>>>> independent
> >>>>>>>>>>>>> (its
> >>>>>>>>>>>>>> a java application more or less), that package will work
> >>>> almost
> >>>>>> on
> >>>>>>> any
> >>>>>>>>>>>>>> DEB-based linux, including but not limited to Ubuntu,
> >>> Debian,
> >>>>>> etc.
> >>>>>>>>>>>>>>> The only restriction is existence of systemctl (systemd)
> >>>>> service
> >>>>>>>>>>>>> manager
> >>>>>>>>>>>>>> — we are dependent on it.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thats why, for instance, our RPM repository is called
> >>> simply
> >>>>>> ‘rpm’
> >>>>>>>>>> and
> >>>>>>>>>>>>>> package has no arch or dist suffix — it will work on
> >>> CentOS,
> >>>>>> RHEL,
> >>>>>>>>>>>>> Fedora,
> >>>>>>>>>>>>>> etc. with presence of aforementioned systemd.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> >>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Will Debian package work for Ubuntu?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> >>>>>>> mr.weider@gmail.com>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
> >> move
> >>> to
> >>>>>>> official
> >>>>>>>>>>>>>>>>> repositories, there can be a problem from OS
> >> community.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
> >> for
> >>>> DEB
> >>>>>>> package
> >>>>>>>>>>>>>> build
> >>>>>>>>>>>>>>>>> (package layout / install scripts) for speeding up
> >>> things
> >>>>> and
> >>>>>>>>>>>>> excluding
> >>>>>>>>>>>>>>>>> possible duplication and desynchronisation, so its a
> >>>> matter
> >>>>> of
> >>>>>>> ’sit
> >>>>>>>>>>>>>> and do’
> >>>>>>>>>>>>>>>>> rather then some technical research. Thats why I rose
> >>>>>> discussion
> >>>>>>>>>>>>> about
> >>>>>>>>>>>>>>>>> future package architecture, so that after agreement
> >> I'm
> >>>> be
> >>>>>>> able to
> >>>>>>>>>>>>>> pack
> >>>>>>>>>>>>>>>>> both RPM and DEB identically.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Yet, if you insist, I can create DEB package according
> >>> to
> >>>>>>> current
> >>>>>>>>>> RPM
> >>>>>>>>>>>>>>>>> layout in no time.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> >>>>>>>>>> dsetrakyan@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Peter,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I don't think the package size of 280M is going to
> >> be a
> >>>>>>> problem at
> >>>>>>>>>>>>>> all,
> >>>>>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>>>>> what you are suggesting can be an improvement down
> >> the
> >>>>> road.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> In the mean time, I think our top priority should be
> >> to
> >>>>>> provide
> >>>>>>>>>>>>>> packages
> >>>>>>>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> >>>>> enough.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Agree?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> >>>>>> mr.weider@gmail.com>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
> >>> it,
> >>>>> so
> >>>>>>> I'd
> >>>>>>>>>>>>> like
> >>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> move
> >>>>>>>>>>>>>>>>>>> forward to further improve and widen AI delivery
> >>> through
> >>>>>>>>>> packages.
> >>>>>>>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
> >> weighing
> >>>>> about
> >>>>>>>>>> 280M+
> >>>>>>>>>>>>>> and,
> >>>>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>>> improve usability and significantly reduce required
> >>>>> download
> >>>>>>>>>>>>> sizes, I
> >>>>>>>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> >>>> delivery
> >>>>>> as
> >>>>>>>>>>>>>> follows:
> >>>>>>>>>>>>>>>>>>> - CORE
> >>>>>>>>>>>>>>>>>>> - bin
> >>>>>>>>>>>>>>>>>>> - config
> >>>>>>>>>>>>>>>>>>> - libs (!optional)
> >>>>>>>>>>>>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>>>>>>>>>>>> - BENCHMARKS
> >>>>>>>>>>>>>>>>>>> - DOCS (?)
> >>>>>>>>>>>>>>>>>>> - EXAMPLES
> >>>>>>>>>>>>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>>>>>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> This architecture, as I assume, will add flexibility
> >>> (no
> >>>>>>> reason
> >>>>>>>>>> to
> >>>>>>>>>>>>>>>>> download
> >>>>>>>>>>>>>>>>>>> all 280M+ of binaries where you are to run only core
> >>>> node
> >>>>>>>>>>>>>> functionality)
> >>>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>> maintainability (you are in full control of what is
> >>>>>> installed
> >>>>>>> on
> >>>>>>>>>>>>> your
> >>>>>>>>>>>>>>>>>>> system).
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> After successful architecture choice, same scheme
> >> are
> >>>>>> planned
> >>>>>>> to
> >>>>>>>>>> be
> >>>>>>>>>>>>>>>>> used in
> >>>>>>>>>>>>>>>>>>> DEB packages as well.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>>>>>> Sent from:
> >>>>>> http://apache-ignite-developers.2346864.n4.nabble.
> >>>>>>>>>> com/
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
> On 15 Apr 2018, at 10:19, Ilya Kasnacheev <il...@gmail.com> wrote:
>
> Hello!
>
> With existing binary archive, user can move directories from
> apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to activate
> them.
>
> But with RPM, user should not contemplate moving directories from
> /usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
I always thought of that option as COPYING optional libs, not MOVING.
>
>
> User lacks permissions for that operation and it would defeat the purpose
> of having a RPM (imagine trying to upgrade Ignite RPM version with a setup
> like that). "Turning distribution into Slackware" should not be an offering.
Can’t imagine use case where Apache Ignite software is installed by one user, and run by another, without sudo/root permissions.
Yet, ignite user’s login shell is disabled indeed and without sudo/root permissions optional libs cannot be copied.
Also — the symlinks is a good idea, but I’ve already thought of it and I’m planning addition of special utility for enabling / disabling optional libs (like a2enable/a2disable) in next development iteration.
>
> How it could work: Imagine we create /var/lib/ignite, use it as
> IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
> way, we can add and remove symlinks to modules in optional/, and thus
> enable and disable them as user sees fit.
By linux file hierarchy convention, home dir should be in /usr/lib. /var/lib/ is currently used for working files (MySQL alike).
>
> This also answers the problem of "what's IGNITE_HOME for visor launched
> from /usr/bin”
That is addressed in separate task and will be fixed with minor startup script redesign with universal location-independent solution.
>
> But obviously this will require dedication and effort.
No problems with efforts after final design is discussed an agreed.
>
>
> --
> Ilya Kasnacheev
>
> 2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
>
>> Current packages design (after installation) does not differ from binary
>> archive - everything works (except necessity to run service instead
>> ignite.sh) just the same way, including libs/optional.
>>
>> Also, there can be issues with system JDK version by default, but every
>> problem will be in journalctl and/or /var/log, and package has strong
>> dependency on any implementation of JDK 1.8.
>>
>>
>> I am lacking enough feedback about Apache Ignite from packages from real
>> users, don’t know real use cases so still "moving in the dark".
>>
>>
>> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
>>
>>> Ilya,
>>>
>>> Thanks for your inputs. The reason why we decided to split Ignite into
>>> several packages mimics the reason why Java community introduced modular
>>> subsystem for JDK. That's all about size. Ignite distribution is too big,
>>> and we're trying to separate it into several components so that people
>> can
>>> install only the features they need.
>>>
>>> The point of a package is to ship something into root file system that
>> can
>>>> be used from root file system. If cpp files require compilation we
>> should
>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
>>>> there's no mechanism to add optional libs to Ignite classpath, we
>> should
>>>> not ship optional libs. Moreover, some of 'optional' modules such as
>> yarn
>>>> don't make sense here because they're not supposed to be used with
>>>> standalone Ignite.
>>>
>>>
>>> Agree that we need to ship the code that is ready to be run. As for the
>>> classpath thing, if an optional package is installed into the root (core)
>>> package directory, then its jars have to be added to "ignite/libs"
>> folder.
>>> After that, the one needs to restart a cluster node, nd it will add the
>>> just installed optional libs to the classpath. *Petr*, does it work this
>>> way or can be implemented this way to address Ilya's concerns?
>>>
>>> --
>>> Denis
>>>
>>> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
>>> ilya.kasnacheev@gmail.com>
>>> wrote:
>>>
>>>> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
>>>>
>>>>> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
>>> ilya.kasnacheev@gmail.com
>>>>>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Moreover I did not find a way to start service if default installed
>>> JVM
>>>>> is
>>>>>> Java 7 :( I understand it's EOL, still this is something that hit
>> me.
>>>>>
>>>>>
>>>>> Apache Ignite >=2.4 does not support Java 7 - it is said in
>>>> documentation,
>>>>> DEVNOTES and even in startup scripts.
>>>>>
>>>>> I have Java 8 too, but I could not get service from package to start
>>>> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
>>>> matter). Is it possible to specify it while running packaged Ignite?
>>>>
>>>>
>>>>>
>>>>>>
>>>>>> apache-ignite-libs is a totally unexpected package name.
>>> apache-ignite
>>>>> core
>>>>>> doesn't depend on it. It doesn't enable anything out of the box.
>> The
>>>>>> package is huge.
>>>>>
>>>>> ‘apache-ignite-libs’ is an aggregation package (for now) for all
>>> optional
>>>>> libs we are delivering. Possibly later they will be split more
>> granular
>>>> or
>>>>> even package per lib (like php, perl, python, etc. do for their
>> libs).
>>>>> This package dependency on ‘apache-ignite-core’ may seem confusing
>>>> though,
>>>>> I will try to explain it in IEP at least for current iteration.
>>>>>
>>>>
>>>> Okay, but how do you add optional libs to be included into Ignite
>>> classpath
>>>> while being launched by service? Is it even possible? If it isn't, I
>>> think
>>>> it doesn't make sense to ship apache-ignite-libs at all.
>>>>
>>>>
>>>>>
>>>>> Further naming may become clear when we’ll start initiative on
>>> including
>>>>> packages to popular Linux distributions and theirs community will
>> join
>>>>> naming discussions.
>>>>>
>>>> Renaming packages once they're deployed widely will be a pain point to
>>> out
>>>> users. Some things should probably be thought out in advance.
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> Frankly speaking, I'm not sure that improvements over Stage I are
>>>> enough
>>>>> as
>>>>>> of now. For demo-like activity, we can probably go with one package
>>>> fits
>>>>>> all.
>>>>>>
>>>>>
>>>>> The process of finding the best package architecture is iterative,
>> but
>>>>> previously community agreed in split design proposed for 2.5 release.
>>>>>
>>>>> Also, split architecture is half of proposed improvements. The other
>>>> half -
>>>>> new process for deploying packages to Bintray (with virtually
>>> indefinite
>>>>> storage capabilities).
>>>>>
>>>> I think we could drop the split for now, or at least drop
>>>> apache-ignite-libs package at all. Probably also drop apache-ignite-cpp
>>>> package and maybe apache-ignite-benchmarks.
>>>>
>>>> The point of a package is to ship something into root file system that
>>> can
>>>> be used from root file system. If cpp files require compilation we
>> should
>>>> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
>>>> there's no mechanism to add optional libs to Ignite classpath, we
>> should
>>>> not ship optional libs. Moreover, some of 'optional' modules such as
>> yarn
>>>> don't make sense here because they're not supposed to be used with
>>>> standalone Ignite.
>>>>
>>>> IMO it is not right to try and shove every file from Ignite
>> distribution
>>>> into some package. We should only put in packages things that can be
>>> used.
>>>> If something can't be used without copying it to a different FS
>> location,
>>>> it should be in examples or not packaged at all.
>>>>
>>>> In my opinion, it doesn't make sense to implement an underwhelming
>>> package
>>>> split right now just because we have agreed to have *some* package
>> split
>>> in
>>>> 2.5. Let's aim for happiness.
>>>>
>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>
>>>>
>>>>
>>>>
>>>>>>
>>>>>> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>
>>>>>>> If someone from PMCы or Committers still sees necessity about
>>>> including
>>>>>>> these tasks into Apache Ignite 2.5 release, this is the last
>> chance
>>>> to
>>>>> do
>>>>>>> so.
>>>>>>> Otherwise this task will be moved to at 2.6 release at least, or
>>> even
>>>>>>> moved to backlog indefinitely.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
>>> wrote:
>>>>>>>>
>>>>>>>> To top new RPM architecture off, update to release process is
>>>>>> introduced
>>>>>>> — [1] [2].
>>>>>>>>
>>>>>>>> Both tasks (this one and IGNITE-7647) are ready for review and
>>>> should
>>>>>> be
>>>>>>> merged simultaneously.
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-8172
>>>>>>>> [2] https://github.com/apache/ignite-release/pull/1
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
>>>> ilya.kasnacheev@gmail.com
>>>>>>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> Let me share my idea of how this shoud work. Splitting package
>>>> into
>>>>>>>>> sub-packages should be dependency-driven.
>>>>>>>>>
>>>>>>>>> It means that all Ignite modules without dependencies or with
>>>> small
>>>>>>>>> dependencies (such as ignite-log4j) should be included in
>>>>> ignite-core.
>>>>>>> It
>>>>>>>>> doesn't make sense to make a zillion RPM packages.
>>>>>>>>>
>>>>>>>>> Critical things like ignite-spring and ignite-indexing should
>> be
>>>> in
>>>>>>>>> ignite-core of course, even if they have dependencies.
>>> Ignite-core
>>>>>>> should
>>>>>>>>> be fully self-sufficient and feature-complete.
>>>>>>>>>
>>>>>>>>> However, e.g. .net API should probably be in a separate
>> package,
>>>>>>> because it
>>>>>>>>> should depend on mono | net-core. We may also have
>> ignite-devel
>>>>>> package
>>>>>>>>> which should include all modules which only make sense for
>>>>> developers
>>>>>>> who
>>>>>>>>> write code. Such as hibernate integration.
>>>>>>>>>
>>>>>>>>> I'm not sure about MR modules. The main question should be,
>> does
>>>> it
>>>>>> have
>>>>>>>>> dependencies? Can it run stand-alone without writing code?
>>>>>>>>>
>>>>>>>>> Hope this helps,
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>
>>>>>>>>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Here are some news on our RPM packages initiative.
>>>>>>>>>>
>>>>>>>>>> 1. I’ve finished preliminary developing of Stage II version
>> of
>>>> RPM
>>>>>>>>>> packages [1]. Main “new feature” is — split design. Also I’ve
>>>> added
>>>>>>>>>> package.sh script for automating package building process
>> which
>>>>> will
>>>>>>> help
>>>>>>>>>> organise corresponding builds in TC as well as simplify
>> process
>>>> for
>>>>>>>>>> developers who wishes to have custom packages.
>>>>>>>>>> PR#3703 [2] is ready for review. Denis, in order to catch up
>>> with
>>>>>>> Apache
>>>>>>>>>> Ignite 2.5 release, I’d greatly appreciate your help in
>> finding
>>>>>>> reviewer.
>>>>>>>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and
>> DEB
>>>> [4]
>>>>>>>>>> repositories on Apache Bintray. Though they are already
>>> prepared
>>>>> for
>>>>>>>>>> hosting RPM and DEB packages respectively, and there is a way
>>> of
>>>>>>> linking
>>>>>>>>>> them to apache.org/dist/ignite page, there is possible
>>>> alternative
>>>>>> in
>>>>>>>>>> storing there only plain directory layout corresponding to
>> each
>>>>>>> repository
>>>>>>>>>> type (RPM and DEB) and manage this layout (repodata,
>>>> distributions,
>>>>>>>>>> versions, etc.) by ourselves, having more control over
>>>> repositories
>>>>>> but
>>>>>>>>>> lacking some simplicity of deploying new releases. WDYT?
>> Should
>>>> we
>>>>>> try
>>>>>>>>>> Cassandra approach? They are storing their DEB packages as I
>>>>>> described
>>>>>>>>>> above [5].
>>>>>>>>>>
>>>>>>>>>> Also — a question arose while I was working on this issue:
>>> which
>>>>> OSes
>>>>>>> (and
>>>>>>>>>> which versions of each) are we going to support (if we are
>>> going)
>>>>> in
>>>>>>> terms
>>>>>>>>>> of step-by-step list? Currently RPM packages are tested only
>>> with
>>>>>>> latest
>>>>>>>>>> CentOS (and, respectively — RHEL), but there are a lot more
>>>>> RPM-based
>>>>>>>>>> distributives [6] some of which are more o less popular among
>>> OS
>>>>>>> community
>>>>>>>>>> (ALT, Fedora, openSUSE, etc.).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>>>>>>>>> [2] https://github.com/apache/ignite/pull/3703
>>>>>>>>>> [3] https://bintray.com/apache/ignite-rpm
>>>>>>>>>> [4] https://bintray.com/apache/ignite-deb
>>>>>>>>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>>>>>>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
>>>>>>> distributions
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I suppose that most everything if not all from libs/options
>>> will
>>>>> go
>>>>>> to
>>>>>>>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>>>>>>>>> More precise lib selection (if something from optional would
>>>>> better
>>>>>> to
>>>>>>>>>> have in core package) will be discussed right after
>> preliminary
>>>>> split
>>>>>>>>>> architecture agreement.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
>>> dpavlov.spb@gmail.com
>>>>>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I like idea of keeping simple system of modules, so +1 from
>>> me.
>>>>>>>>>>>>
>>>>>>>>>>>> Where optional libs (e.g Direct IO plugin) would be
>> included,
>>>>> would
>>>>>>> it
>>>>>>>>>> be
>>>>>>>>>>>> core or optional?
>>>>>>>>>>>>
>>>>>>>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
>> dmagda@apache.org
>>>> :
>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How big would be a final core module?
>>>>>>>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
>>>>> create
>>>>>>> it’s
>>>>>>>>>> own
>>>>>>>>>>>>>> package.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
>>> with
>>>>> Petr
>>>>>>> and
>>>>>>>>>>>>> propose the simplest modular system:
>>>>>>>>>>>>>
>>>>>>>>>>>>> - core module that includes basic Ignite capabilities
>>>> including
>>>>>> SQL,
>>>>>>>>>>>>> compute grid, service grid, k/v
>>>>>>>>>>>>> - optional module hosts the rest - ML, streamers
>> integration
>>>>>> (kafka,
>>>>>>>>>>>>> flink), kubernetes, etc.
>>>>>>>>>>>>>
>>>>>>>>>>>>> What do you think?
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
>>>>>> mr.weider@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> *DEB package
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
>>> mr.weider@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Considering that DEV package for now is almost platform
>>>>>>> independent
>>>>>>>>>>>>> (its
>>>>>>>>>>>>>> a java application more or less), that package will work
>>>> almost
>>>>>> on
>>>>>>> any
>>>>>>>>>>>>>> DEB-based linux, including but not limited to Ubuntu,
>>> Debian,
>>>>>> etc.
>>>>>>>>>>>>>>> The only restriction is existence of systemctl (systemd)
>>>>> service
>>>>>>>>>>>>> manager
>>>>>>>>>>>>>> — we are dependent on it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thats why, for instance, our RPM repository is called
>>> simply
>>>>>> ‘rpm’
>>>>>>>>>> and
>>>>>>>>>>>>>> package has no arch or dist suffix — it will work on
>>> CentOS,
>>>>>> RHEL,
>>>>>>>>>>>>> Fedora,
>>>>>>>>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
>>>>>>> mr.weider@gmail.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will
>> move
>>> to
>>>>>>> official
>>>>>>>>>>>>>>>>> repositories, there can be a problem from OS
>> community.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
>> for
>>>> DEB
>>>>>>> package
>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>>>> (package layout / install scripts) for speeding up
>>> things
>>>>> and
>>>>>>>>>>>>> excluding
>>>>>>>>>>>>>>>>> possible duplication and desynchronisation, so its a
>>>> matter
>>>>> of
>>>>>>> ’sit
>>>>>>>>>>>>>> and do’
>>>>>>>>>>>>>>>>> rather then some technical research. Thats why I rose
>>>>>> discussion
>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>> future package architecture, so that after agreement
>> I'm
>>>> be
>>>>>>> able to
>>>>>>>>>>>>>> pack
>>>>>>>>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Yet, if you insist, I can create DEB package according
>>> to
>>>>>>> current
>>>>>>>>>> RPM
>>>>>>>>>>>>>>>>> layout in no time.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Peter,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I don't think the package size of 280M is going to
>> be a
>>>>>>> problem at
>>>>>>>>>>>>>> all,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> what you are suggesting can be an improvement down
>> the
>>>>> road.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> In the mean time, I think our top priority should be
>> to
>>>>>> provide
>>>>>>>>>>>>>> packages
>>>>>>>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
>>>>> enough.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Agree?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
>>>>>> mr.weider@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
>>> it,
>>>>> so
>>>>>>> I'd
>>>>>>>>>>>>> like
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> move
>>>>>>>>>>>>>>>>>>> forward to further improve and widen AI delivery
>>> through
>>>>>>>>>> packages.
>>>>>>>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
>> weighing
>>>>> about
>>>>>>>>>> 280M+
>>>>>>>>>>>>>> and,
>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> improve usability and significantly reduce required
>>>>> download
>>>>>>>>>>>>> sizes, I
>>>>>>>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
>>>> delivery
>>>>>> as
>>>>>>>>>>>>>> follows:
>>>>>>>>>>>>>>>>>>> - CORE
>>>>>>>>>>>>>>>>>>> - bin
>>>>>>>>>>>>>>>>>>> - config
>>>>>>>>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> This architecture, as I assume, will add flexibility
>>> (no
>>>>>>> reason
>>>>>>>>>> to
>>>>>>>>>>>>>>>>> download
>>>>>>>>>>>>>>>>>>> all 280M+ of binaries where you are to run only core
>>>> node
>>>>>>>>>>>>>> functionality)
>>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>> maintainability (you are in full control of what is
>>>>>> installed
>>>>>>> on
>>>>>>>>>>>>> your
>>>>>>>>>>>>>>>>>>> system).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> After successful architecture choice, same scheme
>> are
>>>>>> planned
>>>>>>> to
>>>>>>>>>> be
>>>>>>>>>>>>>>>>> used in
>>>>>>>>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>>>> Sent from:
>>>>>> http://apache-ignite-developers.2346864.n4.nabble.
>>>>>>>>>> com/
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
With existing binary archive, user can move directories from
apache-ignite-fabric/libs/optional to apache-ignite-fabric/libs to activate
them.
But with RPM, user should not contemplate moving directories from
/usr/lib/apache-ignite/optional to /usr/lib/apache-ignite.
User lacks permissions for that operation and it would defeat the purpose
of having a RPM (imagine trying to upgrade Ignite RPM version with a setup
like that). "Turning distribution into Slackware" should not be an offering.
How it could work: Imagine we create /var/lib/ignite, use it as
IGNITE_HOME, add symlinks from files in /usr/lib to /var/lib/ignite. This
way, we can add and remove symlinks to modules in optional/, and thus
enable and disable them as user sees fit.
This also answers the problem of "what's IGNITE_HOME for visor launched
from /usr/bin"
But obviously this will require dedication and effort.
--
Ilya Kasnacheev
2018-04-14 8:03 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> Current packages design (after installation) does not differ from binary
> archive - everything works (except necessity to run service instead
> ignite.sh) just the same way, including libs/optional.
>
> Also, there can be issues with system JDK version by default, but every
> problem will be in journalctl and/or /var/log, and package has strong
> dependency on any implementation of JDK 1.8.
>
>
> I am lacking enough feedback about Apache Ignite from packages from real
> users, don’t know real use cases so still "moving in the dark".
>
>
> On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
>
> > Ilya,
> >
> > Thanks for your inputs. The reason why we decided to split Ignite into
> > several packages mimics the reason why Java community introduced modular
> > subsystem for JDK. That's all about size. Ignite distribution is too big,
> > and we're trying to separate it into several components so that people
> can
> > install only the features they need.
> >
> > The point of a package is to ship something into root file system that
> can
> > > be used from root file system. If cpp files require compilation we
> should
> > > not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> > > there's no mechanism to add optional libs to Ignite classpath, we
> should
> > > not ship optional libs. Moreover, some of 'optional' modules such as
> yarn
> > > don't make sense here because they're not supposed to be used with
> > > standalone Ignite.
> >
> >
> > Agree that we need to ship the code that is ready to be run. As for the
> > classpath thing, if an optional package is installed into the root (core)
> > package directory, then its jars have to be added to "ignite/libs"
> folder.
> > After that, the one needs to restart a cluster node, nd it will add the
> > just installed optional libs to the classpath. *Petr*, does it work this
> > way or can be implemented this way to address Ilya's concerns?
> >
> > --
> > Denis
> >
> > On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com>
> > wrote:
> >
> > > 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> > >
> > > > On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > Moreover I did not find a way to start service if default installed
> > JVM
> > > > is
> > > > > Java 7 :( I understand it's EOL, still this is something that hit
> me.
> > > >
> > > >
> > > > Apache Ignite >=2.4 does not support Java 7 - it is said in
> > > documentation,
> > > > DEVNOTES and even in startup scripts.
> > > >
> > > > I have Java 8 too, but I could not get service from package to start
> > > Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> > > matter). Is it possible to specify it while running packaged Ignite?
> > >
> > >
> > > >
> > > > >
> > > > > apache-ignite-libs is a totally unexpected package name.
> > apache-ignite
> > > > core
> > > > > doesn't depend on it. It doesn't enable anything out of the box.
> The
> > > > > package is huge.
> > > >
> > > > ‘apache-ignite-libs’ is an aggregation package (for now) for all
> > optional
> > > > libs we are delivering. Possibly later they will be split more
> granular
> > > or
> > > > even package per lib (like php, perl, python, etc. do for their
> libs).
> > > > This package dependency on ‘apache-ignite-core’ may seem confusing
> > > though,
> > > > I will try to explain it in IEP at least for current iteration.
> > > >
> > >
> > > Okay, but how do you add optional libs to be included into Ignite
> > classpath
> > > while being launched by service? Is it even possible? If it isn't, I
> > think
> > > it doesn't make sense to ship apache-ignite-libs at all.
> > >
> > >
> > > >
> > > > Further naming may become clear when we’ll start initiative on
> > including
> > > > packages to popular Linux distributions and theirs community will
> join
> > > > naming discussions.
> > > >
> > > Renaming packages once they're deployed widely will be a pain point to
> > out
> > > users. Some things should probably be thought out in advance.
> > >
> > >
> > > >
> > > >
> > > >
> > > > >
> > > > > Frankly speaking, I'm not sure that improvements over Stage I are
> > > enough
> > > > as
> > > > > of now. For demo-like activity, we can probably go with one package
> > > fits
> > > > > all.
> > > > >
> > > >
> > > > The process of finding the best package architecture is iterative,
> but
> > > > previously community agreed in split design proposed for 2.5 release.
> > > >
> > > > Also, split architecture is half of proposed improvements. The other
> > > half -
> > > > new process for deploying packages to Bintray (with virtually
> > indefinite
> > > > storage capabilities).
> > > >
> > > I think we could drop the split for now, or at least drop
> > > apache-ignite-libs package at all. Probably also drop apache-ignite-cpp
> > > package and maybe apache-ignite-benchmarks.
> > >
> > > The point of a package is to ship something into root file system that
> > can
> > > be used from root file system. If cpp files require compilation we
> should
> > > not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> > > there's no mechanism to add optional libs to Ignite classpath, we
> should
> > > not ship optional libs. Moreover, some of 'optional' modules such as
> yarn
> > > don't make sense here because they're not supposed to be used with
> > > standalone Ignite.
> > >
> > > IMO it is not right to try and shove every file from Ignite
> distribution
> > > into some package. We should only put in packages things that can be
> > used.
> > > If something can't be used without copying it to a different FS
> location,
> > > it should be in examples or not packaged at all.
> > >
> > > In my opinion, it doesn't make sense to implement an underwhelming
> > package
> > > split right now just because we have agreed to have *some* package
> split
> > in
> > > 2.5. Let's aim for happiness.
> > >
> > >
> > > >
> > > >
> > > >
> > > > >
> > > > > --
> > > > > Ilya Kasnacheev
> > > >
> > >
> > >
> > >
> > > > >
> > > > > 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > > >
> > > > > > If someone from PMCы or Committers still sees necessity about
> > > including
> > > > > > these tasks into Apache Ignite 2.5 release, this is the last
> chance
> > > to
> > > > do
> > > > > > so.
> > > > > > Otherwise this task will be moved to at 2.6 release at least, or
> > even
> > > > > > moved to backlog indefinitely.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
> > wrote:
> > > > > > >
> > > > > > > To top new RPM architecture off, update to release process is
> > > > > introduced
> > > > > > — [1] [2].
> > > > > > >
> > > > > > > Both tasks (this one and IGNITE-7647) are ready for review and
> > > should
> > > > > be
> > > > > > merged simultaneously.
> > > > > > >
> > > > > > >
> > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > > > > > [2] https://github.com/apache/ignite-release/pull/1
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> > > ilya.kasnacheev@gmail.com
> > > > >
> > > > > > wrote:
> > > > > > >>
> > > > > > >> Hello!
> > > > > > >>
> > > > > > >> Let me share my idea of how this shoud work. Splitting package
> > > into
> > > > > > >> sub-packages should be dependency-driven.
> > > > > > >>
> > > > > > >> It means that all Ignite modules without dependencies or with
> > > small
> > > > > > >> dependencies (such as ignite-log4j) should be included in
> > > > ignite-core.
> > > > > > It
> > > > > > >> doesn't make sense to make a zillion RPM packages.
> > > > > > >>
> > > > > > >> Critical things like ignite-spring and ignite-indexing should
> be
> > > in
> > > > > > >> ignite-core of course, even if they have dependencies.
> > Ignite-core
> > > > > > should
> > > > > > >> be fully self-sufficient and feature-complete.
> > > > > > >>
> > > > > > >> However, e.g. .net API should probably be in a separate
> package,
> > > > > > because it
> > > > > > >> should depend on mono | net-core. We may also have
> ignite-devel
> > > > > package
> > > > > > >> which should include all modules which only make sense for
> > > > developers
> > > > > > who
> > > > > > >> write code. Such as hibernate integration.
> > > > > > >>
> > > > > > >> I'm not sure about MR modules. The main question should be,
> does
> > > it
> > > > > have
> > > > > > >> dependencies? Can it run stand-alone without writing code?
> > > > > > >>
> > > > > > >> Hope this helps,
> > > > > > >>
> > > > > > >> --
> > > > > > >> Ilya Kasnacheev
> > > > > > >>
> > > > > > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > > > > >>
> > > > > > >>> Hi, Igniters!
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> Here are some news on our RPM packages initiative.
> > > > > > >>>
> > > > > > >>> 1. I’ve finished preliminary developing of Stage II version
> of
> > > RPM
> > > > > > >>> packages [1]. Main “new feature” is — split design. Also I’ve
> > > added
> > > > > > >>> package.sh script for automating package building process
> which
> > > > will
> > > > > > help
> > > > > > >>> organise corresponding builds in TC as well as simplify
> process
> > > for
> > > > > > >>> developers who wishes to have custom packages.
> > > > > > >>> PR#3703 [2] is ready for review. Denis, in order to catch up
> > with
> > > > > > Apache
> > > > > > >>> Ignite 2.5 release, I’d greatly appreciate your help in
> finding
> > > > > > reviewer.
> > > > > > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and
> DEB
> > > [4]
> > > > > > >>> repositories on Apache Bintray. Though they are already
> > prepared
> > > > for
> > > > > > >>> hosting RPM and DEB packages respectively, and there is a way
> > of
> > > > > > linking
> > > > > > >>> them to apache.org/dist/ignite page, there is possible
> > > alternative
> > > > > in
> > > > > > >>> storing there only plain directory layout corresponding to
> each
> > > > > > repository
> > > > > > >>> type (RPM and DEB) and manage this layout (repodata,
> > > distributions,
> > > > > > >>> versions, etc.) by ourselves, having more control over
> > > repositories
> > > > > but
> > > > > > >>> lacking some simplicity of deploying new releases. WDYT?
> Should
> > > we
> > > > > try
> > > > > > >>> Cassandra approach? They are storing their DEB packages as I
> > > > > described
> > > > > > >>> above [5].
> > > > > > >>>
> > > > > > >>> Also — a question arose while I was working on this issue:
> > which
> > > > OSes
> > > > > > (and
> > > > > > >>> which versions of each) are we going to support (if we are
> > going)
> > > > in
> > > > > > terms
> > > > > > >>> of step-by-step list? Currently RPM packages are tested only
> > with
> > > > > > latest
> > > > > > >>> CentOS (and, respectively — RHEL), but there are a lot more
> > > > RPM-based
> > > > > > >>> distributives [6] some of which are more o less popular among
> > OS
> > > > > > community
> > > > > > >>> (ALT, Fedora, openSUSE, etc.).
> > > > > > >>>
> > > > > > >>>
> > > > > > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > > > > > >>> [2] https://github.com/apache/ignite/pull/3703
> > > > > > >>> [3] https://bintray.com/apache/ignite-rpm
> > > > > > >>> [4] https://bintray.com/apache/ignite-deb
> > > > > > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > > > > > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > > > > > distributions
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> > > > wrote:
> > > > > > >>>>
> > > > > > >>>> I suppose that most everything if not all from libs/options
> > will
> > > > go
> > > > > to
> > > > > > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > > > > > >>>> More precise lib selection (if something from optional would
> > > > better
> > > > > to
> > > > > > >>> have in core package) will be discussed right after
> preliminary
> > > > split
> > > > > > >>> architecture agreement.
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>
> > > > > > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
> > dpavlov.spb@gmail.com
> > > >
> > > > > > wrote:
> > > > > > >>>>>
> > > > > > >>>>> I like idea of keeping simple system of modules, so +1 from
> > me.
> > > > > > >>>>>
> > > > > > >>>>> Where optional libs (e.g Direct IO plugin) would be
> included,
> > > > would
> > > > > > it
> > > > > > >>> be
> > > > > > >>>>> core or optional?
> > > > > > >>>>>
> > > > > > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <
> dmagda@apache.org
> > >:
> > > > > > >>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> How big would be a final core module?
> > > > > > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> > > > create
> > > > > > it’s
> > > > > > >>> own
> > > > > > >>>>>>> package.
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
> > with
> > > > Petr
> > > > > > and
> > > > > > >>>>>> propose the simplest modular system:
> > > > > > >>>>>>
> > > > > > >>>>>> - core module that includes basic Ignite capabilities
> > > including
> > > > > SQL,
> > > > > > >>>>>> compute grid, service grid, k/v
> > > > > > >>>>>> - optional module hosts the rest - ML, streamers
> integration
> > > > > (kafka,
> > > > > > >>>>>> flink), kubernetes, etc.
> > > > > > >>>>>>
> > > > > > >>>>>> What do you think?
> > > > > > >>>>>>
> > > > > > >>>>>> --
> > > > > > >>>>>> Denis
> > > > > > >>>>>>
> > > > > > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> > > > > mr.weider@gmail.com>
> > > > > > >>> wrote:
> > > > > > >>>>>>
> > > > > > >>>>>>> *DEB package
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
> > mr.weider@gmail.com>
> > > > > > wrote:
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Considering that DEV package for now is almost platform
> > > > > > independent
> > > > > > >>>>>> (its
> > > > > > >>>>>>> a java application more or less), that package will work
> > > almost
> > > > > on
> > > > > > any
> > > > > > >>>>>>> DEB-based linux, including but not limited to Ubuntu,
> > Debian,
> > > > > etc.
> > > > > > >>>>>>>> The only restriction is existence of systemctl (systemd)
> > > > service
> > > > > > >>>>>> manager
> > > > > > >>>>>>> — we are dependent on it.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>> Thats why, for instance, our RPM repository is called
> > simply
> > > > > ‘rpm’
> > > > > > >>> and
> > > > > > >>>>>>> package has no arch or dist suffix — it will work on
> > CentOS,
> > > > > RHEL,
> > > > > > >>>>>> Fedora,
> > > > > > >>>>>>> etc. with presence of aforementioned systemd.
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > > > > > dsetrakyan@apache.org>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> Will Debian package work for Ubuntu?
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> D.
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > > > > > mr.weider@gmail.com>
> > > > > > >>>>>>> wrote:
> > > > > > >>>>>>>>>
> > > > > > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will
> move
> > to
> > > > > > official
> > > > > > >>>>>>>>>> repositories, there can be a problem from OS
> community.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base
> for
> > > DEB
> > > > > > package
> > > > > > >>>>>>> build
> > > > > > >>>>>>>>>> (package layout / install scripts) for speeding up
> > things
> > > > and
> > > > > > >>>>>> excluding
> > > > > > >>>>>>>>>> possible duplication and desynchronisation, so its a
> > > matter
> > > > of
> > > > > > ’sit
> > > > > > >>>>>>> and do’
> > > > > > >>>>>>>>>> rather then some technical research. Thats why I rose
> > > > > discussion
> > > > > > >>>>>> about
> > > > > > >>>>>>>>>> future package architecture, so that after agreement
> I'm
> > > be
> > > > > > able to
> > > > > > >>>>>>> pack
> > > > > > >>>>>>>>>> both RPM and DEB identically.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>> Yet, if you insist, I can create DEB package according
> > to
> > > > > > current
> > > > > > >>> RPM
> > > > > > >>>>>>>>>> layout in no time.
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > > > > > >>> dsetrakyan@apache.org>
> > > > > > >>>>>>>>>> wrote:
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Peter,
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> I don't think the package size of 280M is going to
> be a
> > > > > > problem at
> > > > > > >>>>>>> all,
> > > > > > >>>>>>>>>> but
> > > > > > >>>>>>>>>>> what you are suggesting can be an improvement down
> the
> > > > road.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> In the mean time, I think our top priority should be
> to
> > > > > provide
> > > > > > >>>>>>> packages
> > > > > > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> > > > enough.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> Agree?
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> D.
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> > > > > mr.weider@gmail.com>
> > > > > > >>>>>> wrote:
> > > > > > >>>>>>>>>>>
> > > > > > >>>>>>>>>>>> Hi, Igniters!
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
> > it,
> > > > so
> > > > > > I'd
> > > > > > >>>>>> like
> > > > > > >>>>>>> to
> > > > > > >>>>>>>>>>>> move
> > > > > > >>>>>>>>>>>> forward to further improve and widen AI delivery
> > through
> > > > > > >>> packages.
> > > > > > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package
> weighing
> > > > about
> > > > > > >>> 280M+
> > > > > > >>>>>>> and,
> > > > > > >>>>>>>>>> to
> > > > > > >>>>>>>>>>>> improve usability and significantly reduce required
> > > > download
> > > > > > >>>>>> sizes, I
> > > > > > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> > > delivery
> > > > > as
> > > > > > >>>>>>> follows:
> > > > > > >>>>>>>>>>>> - CORE
> > > > > > >>>>>>>>>>>> - bin
> > > > > > >>>>>>>>>>>> - config
> > > > > > >>>>>>>>>>>> - libs (!optional)
> > > > > > >>>>>>>>>>>> - OPTIONAL LIBS
> > > > > > >>>>>>>>>>>> - BENCHMARKS
> > > > > > >>>>>>>>>>>> - DOCS (?)
> > > > > > >>>>>>>>>>>> - EXAMPLES
> > > > > > >>>>>>>>>>>> - .NET PLATFORM FILES
> > > > > > >>>>>>>>>>>> - C++ PLATFORM FILES
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> This architecture, as I assume, will add flexibility
> > (no
> > > > > > reason
> > > > > > >>> to
> > > > > > >>>>>>>>>> download
> > > > > > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core
> > > node
> > > > > > >>>>>>> functionality)
> > > > > > >>>>>>>>>>>> and
> > > > > > >>>>>>>>>>>> maintainability (you are in full control of what is
> > > > > installed
> > > > > > on
> > > > > > >>>>>> your
> > > > > > >>>>>>>>>>>> system).
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> After successful architecture choice, same scheme
> are
> > > > > planned
> > > > > > to
> > > > > > >>> be
> > > > > > >>>>>>>>>> used in
> > > > > > >>>>>>>>>>>> DEB packages as well.
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> WDYT?
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>>> --
> > > > > > >>>>>>>>>>>> Sent from:
> > > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > > > >>> com/
> > > > > > >>>>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>>>
> > > > > > >>>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>>
> > > > > > >>>>>>
> > > > > > >>>>
> > > > > > >>>
> > > > > > >>>
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Peter Ivanov <mr...@gmail.com>.
Current packages design (after installation) does not differ from binary
archive - everything works (except necessity to run service instead
ignite.sh) just the same way, including libs/optional.
Also, there can be issues with system JDK version by default, but every
problem will be in journalctl and/or /var/log, and package has strong
dependency on any implementation of JDK 1.8.
I am lacking enough feedback about Apache Ignite from packages from real
users, don’t know real use cases so still "moving in the dark".
On Fri, 13 Apr 2018 at 22:18, Denis Magda <dm...@apache.org> wrote:
> Ilya,
>
> Thanks for your inputs. The reason why we decided to split Ignite into
> several packages mimics the reason why Java community introduced modular
> subsystem for JDK. That's all about size. Ignite distribution is too big,
> and we're trying to separate it into several components so that people can
> install only the features they need.
>
> The point of a package is to ship something into root file system that can
> > be used from root file system. If cpp files require compilation we should
> > not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> > there's no mechanism to add optional libs to Ignite classpath, we should
> > not ship optional libs. Moreover, some of 'optional' modules such as yarn
> > don't make sense here because they're not supposed to be used with
> > standalone Ignite.
>
>
> Agree that we need to ship the code that is ready to be run. As for the
> classpath thing, if an optional package is installed into the root (core)
> package directory, then its jars have to be added to "ignite/libs" folder.
> After that, the one needs to restart a cluster node, nd it will add the
> just installed optional libs to the classpath. *Petr*, does it work this
> way or can be implemented this way to address Ilya's concerns?
>
> --
> Denis
>
> On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com>
> wrote:
>
> > 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> >
> > > On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com
> > >
> > > wrote:
> > >
> > > >
> > > > Moreover I did not find a way to start service if default installed
> JVM
> > > is
> > > > Java 7 :( I understand it's EOL, still this is something that hit me.
> > >
> > >
> > > Apache Ignite >=2.4 does not support Java 7 - it is said in
> > documentation,
> > > DEVNOTES and even in startup scripts.
> > >
> > > I have Java 8 too, but I could not get service from package to start
> > Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> > matter). Is it possible to specify it while running packaged Ignite?
> >
> >
> > >
> > > >
> > > > apache-ignite-libs is a totally unexpected package name.
> apache-ignite
> > > core
> > > > doesn't depend on it. It doesn't enable anything out of the box. The
> > > > package is huge.
> > >
> > > ‘apache-ignite-libs’ is an aggregation package (for now) for all
> optional
> > > libs we are delivering. Possibly later they will be split more granular
> > or
> > > even package per lib (like php, perl, python, etc. do for their libs).
> > > This package dependency on ‘apache-ignite-core’ may seem confusing
> > though,
> > > I will try to explain it in IEP at least for current iteration.
> > >
> >
> > Okay, but how do you add optional libs to be included into Ignite
> classpath
> > while being launched by service? Is it even possible? If it isn't, I
> think
> > it doesn't make sense to ship apache-ignite-libs at all.
> >
> >
> > >
> > > Further naming may become clear when we’ll start initiative on
> including
> > > packages to popular Linux distributions and theirs community will join
> > > naming discussions.
> > >
> > Renaming packages once they're deployed widely will be a pain point to
> out
> > users. Some things should probably be thought out in advance.
> >
> >
> > >
> > >
> > >
> > > >
> > > > Frankly speaking, I'm not sure that improvements over Stage I are
> > enough
> > > as
> > > > of now. For demo-like activity, we can probably go with one package
> > fits
> > > > all.
> > > >
> > >
> > > The process of finding the best package architecture is iterative, but
> > > previously community agreed in split design proposed for 2.5 release.
> > >
> > > Also, split architecture is half of proposed improvements. The other
> > half -
> > > new process for deploying packages to Bintray (with virtually
> indefinite
> > > storage capabilities).
> > >
> > I think we could drop the split for now, or at least drop
> > apache-ignite-libs package at all. Probably also drop apache-ignite-cpp
> > package and maybe apache-ignite-benchmarks.
> >
> > The point of a package is to ship something into root file system that
> can
> > be used from root file system. If cpp files require compilation we should
> > not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> > there's no mechanism to add optional libs to Ignite classpath, we should
> > not ship optional libs. Moreover, some of 'optional' modules such as yarn
> > don't make sense here because they're not supposed to be used with
> > standalone Ignite.
> >
> > IMO it is not right to try and shove every file from Ignite distribution
> > into some package. We should only put in packages things that can be
> used.
> > If something can't be used without copying it to a different FS location,
> > it should be in examples or not packaged at all.
> >
> > In my opinion, it doesn't make sense to implement an underwhelming
> package
> > split right now just because we have agreed to have *some* package split
> in
> > 2.5. Let's aim for happiness.
> >
> >
> > >
> > >
> > >
> > > >
> > > > --
> > > > Ilya Kasnacheev
> > >
> >
> >
> >
> > > >
> > > > 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > >
> > > > > If someone from PMCы or Committers still sees necessity about
> > including
> > > > > these tasks into Apache Ignite 2.5 release, this is the last chance
> > to
> > > do
> > > > > so.
> > > > > Otherwise this task will be moved to at 2.6 release at least, or
> even
> > > > > moved to backlog indefinitely.
> > > > >
> > > > >
> > > > >
> > > > > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com>
> wrote:
> > > > > >
> > > > > > To top new RPM architecture off, update to release process is
> > > > introduced
> > > > > — [1] [2].
> > > > > >
> > > > > > Both tasks (this one and IGNITE-7647) are ready for review and
> > should
> > > > be
> > > > > merged simultaneously.
> > > > > >
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > > > > [2] https://github.com/apache/ignite-release/pull/1
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com
> > > >
> > > > > wrote:
> > > > > >>
> > > > > >> Hello!
> > > > > >>
> > > > > >> Let me share my idea of how this shoud work. Splitting package
> > into
> > > > > >> sub-packages should be dependency-driven.
> > > > > >>
> > > > > >> It means that all Ignite modules without dependencies or with
> > small
> > > > > >> dependencies (such as ignite-log4j) should be included in
> > > ignite-core.
> > > > > It
> > > > > >> doesn't make sense to make a zillion RPM packages.
> > > > > >>
> > > > > >> Critical things like ignite-spring and ignite-indexing should be
> > in
> > > > > >> ignite-core of course, even if they have dependencies.
> Ignite-core
> > > > > should
> > > > > >> be fully self-sufficient and feature-complete.
> > > > > >>
> > > > > >> However, e.g. .net API should probably be in a separate package,
> > > > > because it
> > > > > >> should depend on mono | net-core. We may also have ignite-devel
> > > > package
> > > > > >> which should include all modules which only make sense for
> > > developers
> > > > > who
> > > > > >> write code. Such as hibernate integration.
> > > > > >>
> > > > > >> I'm not sure about MR modules. The main question should be, does
> > it
> > > > have
> > > > > >> dependencies? Can it run stand-alone without writing code?
> > > > > >>
> > > > > >> Hope this helps,
> > > > > >>
> > > > > >> --
> > > > > >> Ilya Kasnacheev
> > > > > >>
> > > > > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > > > >>
> > > > > >>> Hi, Igniters!
> > > > > >>>
> > > > > >>>
> > > > > >>> Here are some news on our RPM packages initiative.
> > > > > >>>
> > > > > >>> 1. I’ve finished preliminary developing of Stage II version of
> > RPM
> > > > > >>> packages [1]. Main “new feature” is — split design. Also I’ve
> > added
> > > > > >>> package.sh script for automating package building process which
> > > will
> > > > > help
> > > > > >>> organise corresponding builds in TC as well as simplify process
> > for
> > > > > >>> developers who wishes to have custom packages.
> > > > > >>> PR#3703 [2] is ready for review. Denis, in order to catch up
> with
> > > > > Apache
> > > > > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > > > > reviewer.
> > > > > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB
> > [4]
> > > > > >>> repositories on Apache Bintray. Though they are already
> prepared
> > > for
> > > > > >>> hosting RPM and DEB packages respectively, and there is a way
> of
> > > > > linking
> > > > > >>> them to apache.org/dist/ignite page, there is possible
> > alternative
> > > > in
> > > > > >>> storing there only plain directory layout corresponding to each
> > > > > repository
> > > > > >>> type (RPM and DEB) and manage this layout (repodata,
> > distributions,
> > > > > >>> versions, etc.) by ourselves, having more control over
> > repositories
> > > > but
> > > > > >>> lacking some simplicity of deploying new releases. WDYT? Should
> > we
> > > > try
> > > > > >>> Cassandra approach? They are storing their DEB packages as I
> > > > described
> > > > > >>> above [5].
> > > > > >>>
> > > > > >>> Also — a question arose while I was working on this issue:
> which
> > > OSes
> > > > > (and
> > > > > >>> which versions of each) are we going to support (if we are
> going)
> > > in
> > > > > terms
> > > > > >>> of step-by-step list? Currently RPM packages are tested only
> with
> > > > > latest
> > > > > >>> CentOS (and, respectively — RHEL), but there are a lot more
> > > RPM-based
> > > > > >>> distributives [6] some of which are more o less popular among
> OS
> > > > > community
> > > > > >>> (ALT, Fedora, openSUSE, etc.).
> > > > > >>>
> > > > > >>>
> > > > > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > > > > >>> [2] https://github.com/apache/ignite/pull/3703
> > > > > >>> [3] https://bintray.com/apache/ignite-rpm
> > > > > >>> [4] https://bintray.com/apache/ignite-deb
> > > > > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > > > > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > > > > distributions
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>
> > > > > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> > > wrote:
> > > > > >>>>
> > > > > >>>> I suppose that most everything if not all from libs/options
> will
> > > go
> > > > to
> > > > > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > > > > >>>> More precise lib selection (if something from optional would
> > > better
> > > > to
> > > > > >>> have in core package) will be discussed right after preliminary
> > > split
> > > > > >>> architecture agreement.
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>
> > > > > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <
> dpavlov.spb@gmail.com
> > >
> > > > > wrote:
> > > > > >>>>>
> > > > > >>>>> I like idea of keeping simple system of modules, so +1 from
> me.
> > > > > >>>>>
> > > > > >>>>> Where optional libs (e.g Direct IO plugin) would be included,
> > > would
> > > > > it
> > > > > >>> be
> > > > > >>>>> core or optional?
> > > > > >>>>>
> > > > > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dmagda@apache.org
> >:
> > > > > >>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>> How big would be a final core module?
> > > > > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> > > create
> > > > > it’s
> > > > > >>> own
> > > > > >>>>>>> package.
> > > > > >>>>>>
> > > > > >>>>>>
> > > > > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree
> with
> > > Petr
> > > > > and
> > > > > >>>>>> propose the simplest modular system:
> > > > > >>>>>>
> > > > > >>>>>> - core module that includes basic Ignite capabilities
> > including
> > > > SQL,
> > > > > >>>>>> compute grid, service grid, k/v
> > > > > >>>>>> - optional module hosts the rest - ML, streamers integration
> > > > (kafka,
> > > > > >>>>>> flink), kubernetes, etc.
> > > > > >>>>>>
> > > > > >>>>>> What do you think?
> > > > > >>>>>>
> > > > > >>>>>> --
> > > > > >>>>>> Denis
> > > > > >>>>>>
> > > > > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> > > > mr.weider@gmail.com>
> > > > > >>> wrote:
> > > > > >>>>>>
> > > > > >>>>>>> *DEB package
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <
> mr.weider@gmail.com>
> > > > > wrote:
> > > > > >>>>>>>>
> > > > > >>>>>>>> Considering that DEV package for now is almost platform
> > > > > independent
> > > > > >>>>>> (its
> > > > > >>>>>>> a java application more or less), that package will work
> > almost
> > > > on
> > > > > any
> > > > > >>>>>>> DEB-based linux, including but not limited to Ubuntu,
> Debian,
> > > > etc.
> > > > > >>>>>>>> The only restriction is existence of systemctl (systemd)
> > > service
> > > > > >>>>>> manager
> > > > > >>>>>>> — we are dependent on it.
> > > > > >>>>>>>>
> > > > > >>>>>>>> Thats why, for instance, our RPM repository is called
> simply
> > > > ‘rpm’
> > > > > >>> and
> > > > > >>>>>>> package has no arch or dist suffix — it will work on
> CentOS,
> > > > RHEL,
> > > > > >>>>>> Fedora,
> > > > > >>>>>>> etc. with presence of aforementioned systemd.
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > > > > dsetrakyan@apache.org>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> Will Debian package work for Ubuntu?
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> D.
> > > > > >>>>>>>>>
> > > > > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > > > > mr.weider@gmail.com>
> > > > > >>>>>>> wrote:
> > > > > >>>>>>>>>
> > > > > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move
> to
> > > > > official
> > > > > >>>>>>>>>> repositories, there can be a problem from OS community.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for
> > DEB
> > > > > package
> > > > > >>>>>>> build
> > > > > >>>>>>>>>> (package layout / install scripts) for speeding up
> things
> > > and
> > > > > >>>>>> excluding
> > > > > >>>>>>>>>> possible duplication and desynchronisation, so its a
> > matter
> > > of
> > > > > ’sit
> > > > > >>>>>>> and do’
> > > > > >>>>>>>>>> rather then some technical research. Thats why I rose
> > > > discussion
> > > > > >>>>>> about
> > > > > >>>>>>>>>> future package architecture, so that after agreement I'm
> > be
> > > > > able to
> > > > > >>>>>>> pack
> > > > > >>>>>>>>>> both RPM and DEB identically.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>> Yet, if you insist, I can create DEB package according
> to
> > > > > current
> > > > > >>> RPM
> > > > > >>>>>>>>>> layout in no time.
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > > > > >>> dsetrakyan@apache.org>
> > > > > >>>>>>>>>> wrote:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Peter,
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> I don't think the package size of 280M is going to be a
> > > > > problem at
> > > > > >>>>>>> all,
> > > > > >>>>>>>>>> but
> > > > > >>>>>>>>>>> what you are suggesting can be an improvement down the
> > > road.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> In the mean time, I think our top priority should be to
> > > > provide
> > > > > >>>>>>> packages
> > > > > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> > > enough.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> Agree?
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> D.
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> > > > mr.weider@gmail.com>
> > > > > >>>>>> wrote:
> > > > > >>>>>>>>>>>
> > > > > >>>>>>>>>>>> Hi, Igniters!
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of
> it,
> > > so
> > > > > I'd
> > > > > >>>>>> like
> > > > > >>>>>>> to
> > > > > >>>>>>>>>>>> move
> > > > > >>>>>>>>>>>> forward to further improve and widen AI delivery
> through
> > > > > >>> packages.
> > > > > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing
> > > about
> > > > > >>> 280M+
> > > > > >>>>>>> and,
> > > > > >>>>>>>>>> to
> > > > > >>>>>>>>>>>> improve usability and significantly reduce required
> > > download
> > > > > >>>>>> sizes, I
> > > > > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> > delivery
> > > > as
> > > > > >>>>>>> follows:
> > > > > >>>>>>>>>>>> - CORE
> > > > > >>>>>>>>>>>> - bin
> > > > > >>>>>>>>>>>> - config
> > > > > >>>>>>>>>>>> - libs (!optional)
> > > > > >>>>>>>>>>>> - OPTIONAL LIBS
> > > > > >>>>>>>>>>>> - BENCHMARKS
> > > > > >>>>>>>>>>>> - DOCS (?)
> > > > > >>>>>>>>>>>> - EXAMPLES
> > > > > >>>>>>>>>>>> - .NET PLATFORM FILES
> > > > > >>>>>>>>>>>> - C++ PLATFORM FILES
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> This architecture, as I assume, will add flexibility
> (no
> > > > > reason
> > > > > >>> to
> > > > > >>>>>>>>>> download
> > > > > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core
> > node
> > > > > >>>>>>> functionality)
> > > > > >>>>>>>>>>>> and
> > > > > >>>>>>>>>>>> maintainability (you are in full control of what is
> > > > installed
> > > > > on
> > > > > >>>>>> your
> > > > > >>>>>>>>>>>> system).
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> After successful architecture choice, same scheme are
> > > > planned
> > > > > to
> > > > > >>> be
> > > > > >>>>>>>>>> used in
> > > > > >>>>>>>>>>>> DEB packages as well.
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> WDYT?
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>>> --
> > > > > >>>>>>>>>>>> Sent from:
> > > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > > >>> com/
> > > > > >>>>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>>>
> > > > > >>>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>>
> > > > > >>>>>>
> > > > > >>>>
> > > > > >>>
> > > > > >>>
> > > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Denis Magda <dm...@apache.org>.
Ilya,
Thanks for your inputs. The reason why we decided to split Ignite into
several packages mimics the reason why Java community introduced modular
subsystem for JDK. That's all about size. Ignite distribution is too big,
and we're trying to separate it into several components so that people can
install only the features they need.
The point of a package is to ship something into root file system that can
> be used from root file system. If cpp files require compilation we should
> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> there's no mechanism to add optional libs to Ignite classpath, we should
> not ship optional libs. Moreover, some of 'optional' modules such as yarn
> don't make sense here because they're not supposed to be used with
> standalone Ignite.
Agree that we need to ship the code that is ready to be run. As for the
classpath thing, if an optional package is installed into the root (core)
package directory, then its jars have to be added to "ignite/libs" folder.
After that, the one needs to restart a cluster node, nd it will add the
just installed optional libs to the classpath. *Petr*, does it work this
way or can be implemented this way to address Ilya's concerns?
--
Denis
On Fri, Apr 13, 2018 at 7:00 AM, Ilya Kasnacheev <il...@gmail.com>
wrote:
> 2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
>
> > On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >
> > wrote:
> >
> > >
> > > Moreover I did not find a way to start service if default installed JVM
> > is
> > > Java 7 :( I understand it's EOL, still this is something that hit me.
> >
> >
> > Apache Ignite >=2.4 does not support Java 7 - it is said in
> documentation,
> > DEVNOTES and even in startup scripts.
> >
> > I have Java 8 too, but I could not get service from package to start
> Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
> matter). Is it possible to specify it while running packaged Ignite?
>
>
> >
> > >
> > > apache-ignite-libs is a totally unexpected package name. apache-ignite
> > core
> > > doesn't depend on it. It doesn't enable anything out of the box. The
> > > package is huge.
> >
> > ‘apache-ignite-libs’ is an aggregation package (for now) for all optional
> > libs we are delivering. Possibly later they will be split more granular
> or
> > even package per lib (like php, perl, python, etc. do for their libs).
> > This package dependency on ‘apache-ignite-core’ may seem confusing
> though,
> > I will try to explain it in IEP at least for current iteration.
> >
>
> Okay, but how do you add optional libs to be included into Ignite classpath
> while being launched by service? Is it even possible? If it isn't, I think
> it doesn't make sense to ship apache-ignite-libs at all.
>
>
> >
> > Further naming may become clear when we’ll start initiative on including
> > packages to popular Linux distributions and theirs community will join
> > naming discussions.
> >
> Renaming packages once they're deployed widely will be a pain point to out
> users. Some things should probably be thought out in advance.
>
>
> >
> >
> >
> > >
> > > Frankly speaking, I'm not sure that improvements over Stage I are
> enough
> > as
> > > of now. For demo-like activity, we can probably go with one package
> fits
> > > all.
> > >
> >
> > The process of finding the best package architecture is iterative, but
> > previously community agreed in split design proposed for 2.5 release.
> >
> > Also, split architecture is half of proposed improvements. The other
> half -
> > new process for deploying packages to Bintray (with virtually indefinite
> > storage capabilities).
> >
> I think we could drop the split for now, or at least drop
> apache-ignite-libs package at all. Probably also drop apache-ignite-cpp
> package and maybe apache-ignite-benchmarks.
>
> The point of a package is to ship something into root file system that can
> be used from root file system. If cpp files require compilation we should
> not ship them, or ship them to 'examples'. Ditto with benchmarks. If
> there's no mechanism to add optional libs to Ignite classpath, we should
> not ship optional libs. Moreover, some of 'optional' modules such as yarn
> don't make sense here because they're not supposed to be used with
> standalone Ignite.
>
> IMO it is not right to try and shove every file from Ignite distribution
> into some package. We should only put in packages things that can be used.
> If something can't be used without copying it to a different FS location,
> it should be in examples or not packaged at all.
>
> In my opinion, it doesn't make sense to implement an underwhelming package
> split right now just because we have agreed to have *some* package split in
> 2.5. Let's aim for happiness.
>
>
> >
> >
> >
> > >
> > > --
> > > Ilya Kasnacheev
> >
>
>
>
> > >
> > > 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > >
> > > > If someone from PMCы or Committers still sees necessity about
> including
> > > > these tasks into Apache Ignite 2.5 release, this is the last chance
> to
> > do
> > > > so.
> > > > Otherwise this task will be moved to at 2.6 release at least, or even
> > > > moved to backlog indefinitely.
> > > >
> > > >
> > > >
> > > > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> > > > >
> > > > > To top new RPM architecture off, update to release process is
> > > introduced
> > > > — [1] [2].
> > > > >
> > > > > Both tasks (this one and IGNITE-7647) are ready for review and
> should
> > > be
> > > > merged simultaneously.
> > > > >
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > > > [2] https://github.com/apache/ignite-release/pull/1
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com
> > >
> > > > wrote:
> > > > >>
> > > > >> Hello!
> > > > >>
> > > > >> Let me share my idea of how this shoud work. Splitting package
> into
> > > > >> sub-packages should be dependency-driven.
> > > > >>
> > > > >> It means that all Ignite modules without dependencies or with
> small
> > > > >> dependencies (such as ignite-log4j) should be included in
> > ignite-core.
> > > > It
> > > > >> doesn't make sense to make a zillion RPM packages.
> > > > >>
> > > > >> Critical things like ignite-spring and ignite-indexing should be
> in
> > > > >> ignite-core of course, even if they have dependencies. Ignite-core
> > > > should
> > > > >> be fully self-sufficient and feature-complete.
> > > > >>
> > > > >> However, e.g. .net API should probably be in a separate package,
> > > > because it
> > > > >> should depend on mono | net-core. We may also have ignite-devel
> > > package
> > > > >> which should include all modules which only make sense for
> > developers
> > > > who
> > > > >> write code. Such as hibernate integration.
> > > > >>
> > > > >> I'm not sure about MR modules. The main question should be, does
> it
> > > have
> > > > >> dependencies? Can it run stand-alone without writing code?
> > > > >>
> > > > >> Hope this helps,
> > > > >>
> > > > >> --
> > > > >> Ilya Kasnacheev
> > > > >>
> > > > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > > >>
> > > > >>> Hi, Igniters!
> > > > >>>
> > > > >>>
> > > > >>> Here are some news on our RPM packages initiative.
> > > > >>>
> > > > >>> 1. I’ve finished preliminary developing of Stage II version of
> RPM
> > > > >>> packages [1]. Main “new feature” is — split design. Also I’ve
> added
> > > > >>> package.sh script for automating package building process which
> > will
> > > > help
> > > > >>> organise corresponding builds in TC as well as simplify process
> for
> > > > >>> developers who wishes to have custom packages.
> > > > >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> > > > Apache
> > > > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > > > reviewer.
> > > > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB
> [4]
> > > > >>> repositories on Apache Bintray. Though they are already prepared
> > for
> > > > >>> hosting RPM and DEB packages respectively, and there is a way of
> > > > linking
> > > > >>> them to apache.org/dist/ignite page, there is possible
> alternative
> > > in
> > > > >>> storing there only plain directory layout corresponding to each
> > > > repository
> > > > >>> type (RPM and DEB) and manage this layout (repodata,
> distributions,
> > > > >>> versions, etc.) by ourselves, having more control over
> repositories
> > > but
> > > > >>> lacking some simplicity of deploying new releases. WDYT? Should
> we
> > > try
> > > > >>> Cassandra approach? They are storing their DEB packages as I
> > > described
> > > > >>> above [5].
> > > > >>>
> > > > >>> Also — a question arose while I was working on this issue: which
> > OSes
> > > > (and
> > > > >>> which versions of each) are we going to support (if we are going)
> > in
> > > > terms
> > > > >>> of step-by-step list? Currently RPM packages are tested only with
> > > > latest
> > > > >>> CentOS (and, respectively — RHEL), but there are a lot more
> > RPM-based
> > > > >>> distributives [6] some of which are more o less popular among OS
> > > > community
> > > > >>> (ALT, Fedora, openSUSE, etc.).
> > > > >>>
> > > > >>>
> > > > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > > > >>> [2] https://github.com/apache/ignite/pull/3703
> > > > >>> [3] https://bintray.com/apache/ignite-rpm
> > > > >>> [4] https://bintray.com/apache/ignite-deb
> > > > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > > > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > > > distributions
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> > wrote:
> > > > >>>>
> > > > >>>> I suppose that most everything if not all from libs/options will
> > go
> > > to
> > > > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > > > >>>> More precise lib selection (if something from optional would
> > better
> > > to
> > > > >>> have in core package) will be discussed right after preliminary
> > split
> > > > >>> architecture agreement.
> > > > >>>>
> > > > >>>>
> > > > >>>>
> > > > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dpavlov.spb@gmail.com
> >
> > > > wrote:
> > > > >>>>>
> > > > >>>>> I like idea of keeping simple system of modules, so +1 from me.
> > > > >>>>>
> > > > >>>>> Where optional libs (e.g Direct IO plugin) would be included,
> > would
> > > > it
> > > > >>> be
> > > > >>>>> core or optional?
> > > > >>>>>
> > > > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> > > > >>>>>
> > > > >>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>> How big would be a final core module?
> > > > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> > create
> > > > it’s
> > > > >>> own
> > > > >>>>>>> package.
> > > > >>>>>>
> > > > >>>>>>
> > > > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with
> > Petr
> > > > and
> > > > >>>>>> propose the simplest modular system:
> > > > >>>>>>
> > > > >>>>>> - core module that includes basic Ignite capabilities
> including
> > > SQL,
> > > > >>>>>> compute grid, service grid, k/v
> > > > >>>>>> - optional module hosts the rest - ML, streamers integration
> > > (kafka,
> > > > >>>>>> flink), kubernetes, etc.
> > > > >>>>>>
> > > > >>>>>> What do you think?
> > > > >>>>>>
> > > > >>>>>> --
> > > > >>>>>> Denis
> > > > >>>>>>
> > > > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> > > mr.weider@gmail.com>
> > > > >>> wrote:
> > > > >>>>>>
> > > > >>>>>>> *DEB package
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> > > > wrote:
> > > > >>>>>>>>
> > > > >>>>>>>> Considering that DEV package for now is almost platform
> > > > independent
> > > > >>>>>> (its
> > > > >>>>>>> a java application more or less), that package will work
> almost
> > > on
> > > > any
> > > > >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
> > > etc.
> > > > >>>>>>>> The only restriction is existence of systemctl (systemd)
> > service
> > > > >>>>>> manager
> > > > >>>>>>> — we are dependent on it.
> > > > >>>>>>>>
> > > > >>>>>>>> Thats why, for instance, our RPM repository is called simply
> > > ‘rpm’
> > > > >>> and
> > > > >>>>>>> package has no arch or dist suffix — it will work on CentOS,
> > > RHEL,
> > > > >>>>>> Fedora,
> > > > >>>>>>> etc. with presence of aforementioned systemd.
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > > > dsetrakyan@apache.org>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>> Will Debian package work for Ubuntu?
> > > > >>>>>>>>>
> > > > >>>>>>>>> D.
> > > > >>>>>>>>>
> > > > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > > > mr.weider@gmail.com>
> > > > >>>>>>> wrote:
> > > > >>>>>>>>>
> > > > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> > > > official
> > > > >>>>>>>>>> repositories, there can be a problem from OS community.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for
> DEB
> > > > package
> > > > >>>>>>> build
> > > > >>>>>>>>>> (package layout / install scripts) for speeding up things
> > and
> > > > >>>>>> excluding
> > > > >>>>>>>>>> possible duplication and desynchronisation, so its a
> matter
> > of
> > > > ’sit
> > > > >>>>>>> and do’
> > > > >>>>>>>>>> rather then some technical research. Thats why I rose
> > > discussion
> > > > >>>>>> about
> > > > >>>>>>>>>> future package architecture, so that after agreement I'm
> be
> > > > able to
> > > > >>>>>>> pack
> > > > >>>>>>>>>> both RPM and DEB identically.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> > > > current
> > > > >>> RPM
> > > > >>>>>>>>>> layout in no time.
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > > > >>> dsetrakyan@apache.org>
> > > > >>>>>>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Peter,
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> I don't think the package size of 280M is going to be a
> > > > problem at
> > > > >>>>>>> all,
> > > > >>>>>>>>>> but
> > > > >>>>>>>>>>> what you are suggesting can be an improvement down the
> > road.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> In the mean time, I think our top priority should be to
> > > provide
> > > > >>>>>>> packages
> > > > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> > enough.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> Agree?
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> D.
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> > > mr.weider@gmail.com>
> > > > >>>>>> wrote:
> > > > >>>>>>>>>>>
> > > > >>>>>>>>>>>> Hi, Igniters!
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it,
> > so
> > > > I'd
> > > > >>>>>> like
> > > > >>>>>>> to
> > > > >>>>>>>>>>>> move
> > > > >>>>>>>>>>>> forward to further improve and widen AI delivery through
> > > > >>> packages.
> > > > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing
> > about
> > > > >>> 280M+
> > > > >>>>>>> and,
> > > > >>>>>>>>>> to
> > > > >>>>>>>>>>>> improve usability and significantly reduce required
> > download
> > > > >>>>>> sizes, I
> > > > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted
> delivery
> > > as
> > > > >>>>>>> follows:
> > > > >>>>>>>>>>>> - CORE
> > > > >>>>>>>>>>>> - bin
> > > > >>>>>>>>>>>> - config
> > > > >>>>>>>>>>>> - libs (!optional)
> > > > >>>>>>>>>>>> - OPTIONAL LIBS
> > > > >>>>>>>>>>>> - BENCHMARKS
> > > > >>>>>>>>>>>> - DOCS (?)
> > > > >>>>>>>>>>>> - EXAMPLES
> > > > >>>>>>>>>>>> - .NET PLATFORM FILES
> > > > >>>>>>>>>>>> - C++ PLATFORM FILES
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> > > > reason
> > > > >>> to
> > > > >>>>>>>>>> download
> > > > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core
> node
> > > > >>>>>>> functionality)
> > > > >>>>>>>>>>>> and
> > > > >>>>>>>>>>>> maintainability (you are in full control of what is
> > > installed
> > > > on
> > > > >>>>>> your
> > > > >>>>>>>>>>>> system).
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> After successful architecture choice, same scheme are
> > > planned
> > > > to
> > > > >>> be
> > > > >>>>>>>>>> used in
> > > > >>>>>>>>>>>> DEB packages as well.
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> WDYT?
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>>> --
> > > > >>>>>>>>>>>> Sent from:
> > > http://apache-ignite-developers.2346864.n4.nabble.
> > > > >>> com/
> > > > >>>>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>>>
> > > > >>>>>>>>
> > > > >>>>>>>
> > > > >>>>>>>
> > > > >>>>>>
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >
> > > >
> > > >
> > >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
2018-04-13 7:42 GMT+03:00 Peter Ivanov <mr...@gmail.com>:
> On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
> >
> > Moreover I did not find a way to start service if default installed JVM
> is
> > Java 7 :( I understand it's EOL, still this is something that hit me.
>
>
> Apache Ignite >=2.4 does not support Java 7 - it is said in documentation,
> DEVNOTES and even in startup scripts.
>
> I have Java 8 too, but I could not get service from package to start
Ignite since there's nowhere to put JAVA_HOME (or JVM_ARGS for that
matter). Is it possible to specify it while running packaged Ignite?
>
> >
> > apache-ignite-libs is a totally unexpected package name. apache-ignite
> core
> > doesn't depend on it. It doesn't enable anything out of the box. The
> > package is huge.
>
> ‘apache-ignite-libs’ is an aggregation package (for now) for all optional
> libs we are delivering. Possibly later they will be split more granular or
> even package per lib (like php, perl, python, etc. do for their libs).
> This package dependency on ‘apache-ignite-core’ may seem confusing though,
> I will try to explain it in IEP at least for current iteration.
>
Okay, but how do you add optional libs to be included into Ignite classpath
while being launched by service? Is it even possible? If it isn't, I think
it doesn't make sense to ship apache-ignite-libs at all.
>
> Further naming may become clear when we’ll start initiative on including
> packages to popular Linux distributions and theirs community will join
> naming discussions.
>
Renaming packages once they're deployed widely will be a pain point to out
users. Some things should probably be thought out in advance.
>
>
>
> >
> > Frankly speaking, I'm not sure that improvements over Stage I are enough
> as
> > of now. For demo-like activity, we can probably go with one package fits
> > all.
> >
>
> The process of finding the best package architecture is iterative, but
> previously community agreed in split design proposed for 2.5 release.
>
> Also, split architecture is half of proposed improvements. The other half -
> new process for deploying packages to Bintray (with virtually indefinite
> storage capabilities).
>
I think we could drop the split for now, or at least drop
apache-ignite-libs package at all. Probably also drop apache-ignite-cpp
package and maybe apache-ignite-benchmarks.
The point of a package is to ship something into root file system that can
be used from root file system. If cpp files require compilation we should
not ship them, or ship them to 'examples'. Ditto with benchmarks. If
there's no mechanism to add optional libs to Ignite classpath, we should
not ship optional libs. Moreover, some of 'optional' modules such as yarn
don't make sense here because they're not supposed to be used with
standalone Ignite.
IMO it is not right to try and shove every file from Ignite distribution
into some package. We should only put in packages things that can be used.
If something can't be used without copying it to a different FS location,
it should be in examples or not packaged at all.
In my opinion, it doesn't make sense to implement an underwhelming package
split right now just because we have agreed to have *some* package split in
2.5. Let's aim for happiness.
>
>
>
> >
> > --
> > Ilya Kasnacheev
>
> >
> > 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >
> > > If someone from PMCы or Committers still sees necessity about including
> > > these tasks into Apache Ignite 2.5 release, this is the last chance to
> do
> > > so.
> > > Otherwise this task will be moved to at 2.6 release at least, or even
> > > moved to backlog indefinitely.
> > >
> > >
> > >
> > > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> > > >
> > > > To top new RPM architecture off, update to release process is
> > introduced
> > > — [1] [2].
> > > >
> > > > Both tasks (this one and IGNITE-7647) are ready for review and should
> > be
> > > merged simultaneously.
> > > >
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > > [2] https://github.com/apache/ignite-release/pull/1
> > > >
> > > >
> > > >
> > > >
> > > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >
> > > wrote:
> > > >>
> > > >> Hello!
> > > >>
> > > >> Let me share my idea of how this shoud work. Splitting package into
> > > >> sub-packages should be dependency-driven.
> > > >>
> > > >> It means that all Ignite modules without dependencies or with small
> > > >> dependencies (such as ignite-log4j) should be included in
> ignite-core.
> > > It
> > > >> doesn't make sense to make a zillion RPM packages.
> > > >>
> > > >> Critical things like ignite-spring and ignite-indexing should be in
> > > >> ignite-core of course, even if they have dependencies. Ignite-core
> > > should
> > > >> be fully self-sufficient and feature-complete.
> > > >>
> > > >> However, e.g. .net API should probably be in a separate package,
> > > because it
> > > >> should depend on mono | net-core. We may also have ignite-devel
> > package
> > > >> which should include all modules which only make sense for
> developers
> > > who
> > > >> write code. Such as hibernate integration.
> > > >>
> > > >> I'm not sure about MR modules. The main question should be, does it
> > have
> > > >> dependencies? Can it run stand-alone without writing code?
> > > >>
> > > >> Hope this helps,
> > > >>
> > > >> --
> > > >> Ilya Kasnacheev
> > > >>
> > > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > > >>
> > > >>> Hi, Igniters!
> > > >>>
> > > >>>
> > > >>> Here are some news on our RPM packages initiative.
> > > >>>
> > > >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> > > >>> packages [1]. Main “new feature” is — split design. Also I’ve added
> > > >>> package.sh script for automating package building process which
> will
> > > help
> > > >>> organise corresponding builds in TC as well as simplify process for
> > > >>> developers who wishes to have custom packages.
> > > >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> > > Apache
> > > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > > reviewer.
> > > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> > > >>> repositories on Apache Bintray. Though they are already prepared
> for
> > > >>> hosting RPM and DEB packages respectively, and there is a way of
> > > linking
> > > >>> them to apache.org/dist/ignite page, there is possible alternative
> > in
> > > >>> storing there only plain directory layout corresponding to each
> > > repository
> > > >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> > > >>> versions, etc.) by ourselves, having more control over repositories
> > but
> > > >>> lacking some simplicity of deploying new releases. WDYT? Should we
> > try
> > > >>> Cassandra approach? They are storing their DEB packages as I
> > described
> > > >>> above [5].
> > > >>>
> > > >>> Also — a question arose while I was working on this issue: which
> OSes
> > > (and
> > > >>> which versions of each) are we going to support (if we are going)
> in
> > > terms
> > > >>> of step-by-step list? Currently RPM packages are tested only with
> > > latest
> > > >>> CentOS (and, respectively — RHEL), but there are a lot more
> RPM-based
> > > >>> distributives [6] some of which are more o less popular among OS
> > > community
> > > >>> (ALT, Fedora, openSUSE, etc.).
> > > >>>
> > > >>>
> > > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > > >>> [2] https://github.com/apache/ignite/pull/3703
> > > >>> [3] https://bintray.com/apache/ignite-rpm
> > > >>> [4] https://bintray.com/apache/ignite-deb
> > > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > > distributions
> > > >>>
> > > >>>
> > > >>>
> > > >>>
> > > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com>
> wrote:
> > > >>>>
> > > >>>> I suppose that most everything if not all from libs/options will
> go
> > to
> > > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > > >>>> More precise lib selection (if something from optional would
> better
> > to
> > > >>> have in core package) will be discussed right after preliminary
> split
> > > >>> architecture agreement.
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> > > wrote:
> > > >>>>>
> > > >>>>> I like idea of keeping simple system of modules, so +1 from me.
> > > >>>>>
> > > >>>>> Where optional libs (e.g Direct IO plugin) would be included,
> would
> > > it
> > > >>> be
> > > >>>>> core or optional?
> > > >>>>>
> > > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> > > >>>>>
> > > >>>>>>>
> > > >>>>>>>>
> > > >>>>>>>> How big would be a final core module?
> > > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and
> create
> > > it’s
> > > >>> own
> > > >>>>>>> package.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with
> Petr
> > > and
> > > >>>>>> propose the simplest modular system:
> > > >>>>>>
> > > >>>>>> - core module that includes basic Ignite capabilities including
> > SQL,
> > > >>>>>> compute grid, service grid, k/v
> > > >>>>>> - optional module hosts the rest - ML, streamers integration
> > (kafka,
> > > >>>>>> flink), kubernetes, etc.
> > > >>>>>>
> > > >>>>>> What do you think?
> > > >>>>>>
> > > >>>>>> --
> > > >>>>>> Denis
> > > >>>>>>
> > > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> > mr.weider@gmail.com>
> > > >>> wrote:
> > > >>>>>>
> > > >>>>>>> *DEB package
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> > > wrote:
> > > >>>>>>>>
> > > >>>>>>>> Considering that DEV package for now is almost platform
> > > independent
> > > >>>>>> (its
> > > >>>>>>> a java application more or less), that package will work almost
> > on
> > > any
> > > >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
> > etc.
> > > >>>>>>>> The only restriction is existence of systemctl (systemd)
> service
> > > >>>>>> manager
> > > >>>>>>> — we are dependent on it.
> > > >>>>>>>>
> > > >>>>>>>> Thats why, for instance, our RPM repository is called simply
> > ‘rpm’
> > > >>> and
> > > >>>>>>> package has no arch or dist suffix — it will work on CentOS,
> > RHEL,
> > > >>>>>> Fedora,
> > > >>>>>>> etc. with presence of aforementioned systemd.
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > > dsetrakyan@apache.org>
> > > >>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>> Will Debian package work for Ubuntu?
> > > >>>>>>>>>
> > > >>>>>>>>> D.
> > > >>>>>>>>>
> > > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > > mr.weider@gmail.com>
> > > >>>>>>> wrote:
> > > >>>>>>>>>
> > > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> > > official
> > > >>>>>>>>>> repositories, there can be a problem from OS community.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> > > package
> > > >>>>>>> build
> > > >>>>>>>>>> (package layout / install scripts) for speeding up things
> and
> > > >>>>>> excluding
> > > >>>>>>>>>> possible duplication and desynchronisation, so its a matter
> of
> > > ’sit
> > > >>>>>>> and do’
> > > >>>>>>>>>> rather then some technical research. Thats why I rose
> > discussion
> > > >>>>>> about
> > > >>>>>>>>>> future package architecture, so that after agreement I'm be
> > > able to
> > > >>>>>>> pack
> > > >>>>>>>>>> both RPM and DEB identically.
> > > >>>>>>>>>>
> > > >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> > > current
> > > >>> RPM
> > > >>>>>>>>>> layout in no time.
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > > >>> dsetrakyan@apache.org>
> > > >>>>>>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Peter,
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> I don't think the package size of 280M is going to be a
> > > problem at
> > > >>>>>>> all,
> > > >>>>>>>>>> but
> > > >>>>>>>>>>> what you are suggesting can be an improvement down the
> road.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> In the mean time, I think our top priority should be to
> > provide
> > > >>>>>>> packages
> > > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly
> enough.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> Agree?
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> D.
> > > >>>>>>>>>>>
> > > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> > mr.weider@gmail.com>
> > > >>>>>> wrote:
> > > >>>>>>>>>>>
> > > >>>>>>>>>>>> Hi, Igniters!
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it,
> so
> > > I'd
> > > >>>>>> like
> > > >>>>>>> to
> > > >>>>>>>>>>>> move
> > > >>>>>>>>>>>> forward to further improve and widen AI delivery through
> > > >>> packages.
> > > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing
> about
> > > >>> 280M+
> > > >>>>>>> and,
> > > >>>>>>>>>> to
> > > >>>>>>>>>>>> improve usability and significantly reduce required
> download
> > > >>>>>> sizes, I
> > > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery
> > as
> > > >>>>>>> follows:
> > > >>>>>>>>>>>> - CORE
> > > >>>>>>>>>>>> - bin
> > > >>>>>>>>>>>> - config
> > > >>>>>>>>>>>> - libs (!optional)
> > > >>>>>>>>>>>> - OPTIONAL LIBS
> > > >>>>>>>>>>>> - BENCHMARKS
> > > >>>>>>>>>>>> - DOCS (?)
> > > >>>>>>>>>>>> - EXAMPLES
> > > >>>>>>>>>>>> - .NET PLATFORM FILES
> > > >>>>>>>>>>>> - C++ PLATFORM FILES
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> > > reason
> > > >>> to
> > > >>>>>>>>>> download
> > > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> > > >>>>>>> functionality)
> > > >>>>>>>>>>>> and
> > > >>>>>>>>>>>> maintainability (you are in full control of what is
> > installed
> > > on
> > > >>>>>> your
> > > >>>>>>>>>>>> system).
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> After successful architecture choice, same scheme are
> > planned
> > > to
> > > >>> be
> > > >>>>>>>>>> used in
> > > >>>>>>>>>>>> DEB packages as well.
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> WDYT?
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>>> --
> > > >>>>>>>>>>>> Sent from:
> > http://apache-ignite-developers.2346864.n4.nabble.
> > > >>> com/
> > > >>>>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>>>
> > > >>>>>>>>
> > > >>>>>>>
> > > >>>>>>>
> > > >>>>>>
> > > >>>>
> > > >>>
> > > >>>
> > > >
> > >
> > >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Peter Ivanov <mr...@gmail.com>.
On Thu, 12 Apr 2018 at 20:04, Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello! I have tried your RPM scripts.
>
> First of all, I'm not sure that apache-ignite-core is a good name for
> package which contains the actual server node kit, and that apache-ignite
> is a good name for a package that will install everything (including cpp
> bindings). How does other packages solve this naming? I would go with
> apache-ignite and apache-ignite-full.
Previous package was named ‘apache-ignite’ and installed everything.
Current package is named ‘apache-ignite’ and installs everything (via
dependencies). I see convenience in this scheme.
‘apache-ignite-core’ name was selected because it is a core - minimum
required files to start the service. But I am open to suggestions
(-service, -node, etc.)
>
> Moreover I did not find a way to start service if default installed JVM is
> Java 7 :( I understand it's EOL, still this is something that hit me.
Apache Ignite >=2.4 does not support Java 7 - it is said in documentation,
DEVNOTES and even in startup scripts.
>
> apache-ignite-docs only contains scaladoc for scalar. Who would need a
> separate package for that? Why no javadocs? Maybe it's my build problem?
‘apache-ignite-docs’ contains both Javadoc and Scalardoc if sources were
built correctly
>
> apache-ignite-libs is a totally unexpected package name. apache-ignite core
> doesn't depend on it. It doesn't enable anything out of the box. The
> package is huge.
‘apache-ignite-libs’ is an aggregation package (for now) for all optional
libs we are delivering. Possibly later they will be split more granular or
even package per lib (like php, perl, python, etc. do for their libs).
This package dependency on ‘apache-ignite-core’ may seem confusing though,
I will try to explain it in IEP at least for current iteration.
Further naming may become clear when we’ll start initiative on including
packages to popular Linux distributions and theirs community will join
naming discussions.
>
> Frankly speaking, I'm not sure that improvements over Stage I are enough as
> of now. For demo-like activity, we can probably go with one package fits
> all.
>
The process of finding the best package architecture is iterative, but
previously community agreed in split design proposed for 2.5 release.
Also, split architecture is half of proposed improvements. The other half -
new process for deploying packages to Bintray (with virtually indefinite
storage capabilities).
>
> --
> Ilya Kasnacheev
>
> 2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
> > If someone from PMCы or Committers still sees necessity about including
> > these tasks into Apache Ignite 2.5 release, this is the last chance to do
> > so.
> > Otherwise this task will be moved to at 2.6 release at least, or even
> > moved to backlog indefinitely.
> >
> >
> >
> > > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> > >
> > > To top new RPM architecture off, update to release process is
> introduced
> > — [1] [2].
> > >
> > > Both tasks (this one and IGNITE-7647) are ready for review and should
> be
> > merged simultaneously.
> > >
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > > [2] https://github.com/apache/ignite-release/pull/1
> > >
> > >
> > >
> > >
> > >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com>
> > wrote:
> > >>
> > >> Hello!
> > >>
> > >> Let me share my idea of how this shoud work. Splitting package into
> > >> sub-packages should be dependency-driven.
> > >>
> > >> It means that all Ignite modules without dependencies or with small
> > >> dependencies (such as ignite-log4j) should be included in ignite-core.
> > It
> > >> doesn't make sense to make a zillion RPM packages.
> > >>
> > >> Critical things like ignite-spring and ignite-indexing should be in
> > >> ignite-core of course, even if they have dependencies. Ignite-core
> > should
> > >> be fully self-sufficient and feature-complete.
> > >>
> > >> However, e.g. .net API should probably be in a separate package,
> > because it
> > >> should depend on mono | net-core. We may also have ignite-devel
> package
> > >> which should include all modules which only make sense for developers
> > who
> > >> write code. Such as hibernate integration.
> > >>
> > >> I'm not sure about MR modules. The main question should be, does it
> have
> > >> dependencies? Can it run stand-alone without writing code?
> > >>
> > >> Hope this helps,
> > >>
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> > >>
> > >>> Hi, Igniters!
> > >>>
> > >>>
> > >>> Here are some news on our RPM packages initiative.
> > >>>
> > >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> > >>> packages [1]. Main “new feature” is — split design. Also I’ve added
> > >>> package.sh script for automating package building process which will
> > help
> > >>> organise corresponding builds in TC as well as simplify process for
> > >>> developers who wishes to have custom packages.
> > >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> > Apache
> > >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> > reviewer.
> > >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> > >>> repositories on Apache Bintray. Though they are already prepared for
> > >>> hosting RPM and DEB packages respectively, and there is a way of
> > linking
> > >>> them to apache.org/dist/ignite page, there is possible alternative
> in
> > >>> storing there only plain directory layout corresponding to each
> > repository
> > >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> > >>> versions, etc.) by ourselves, having more control over repositories
> but
> > >>> lacking some simplicity of deploying new releases. WDYT? Should we
> try
> > >>> Cassandra approach? They are storing their DEB packages as I
> described
> > >>> above [5].
> > >>>
> > >>> Also — a question arose while I was working on this issue: which OSes
> > (and
> > >>> which versions of each) are we going to support (if we are going) in
> > terms
> > >>> of step-by-step list? Currently RPM packages are tested only with
> > latest
> > >>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> > >>> distributives [6] some of which are more o less popular among OS
> > community
> > >>> (ALT, Fedora, openSUSE, etc.).
> > >>>
> > >>>
> > >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> > >>> [2] https://github.com/apache/ignite/pull/3703
> > >>> [3] https://bintray.com/apache/ignite-rpm
> > >>> [4] https://bintray.com/apache/ignite-deb
> > >>> [5] https://bintray.com/apache/cassandra/debian#files/
> > >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> > distributions
> > >>>
> > >>>
> > >>>
> > >>>
> > >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> > >>>>
> > >>>> I suppose that most everything if not all from libs/options will go
> to
> > >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > >>>> More precise lib selection (if something from optional would better
> to
> > >>> have in core package) will be discussed right after preliminary split
> > >>> architecture agreement.
> > >>>>
> > >>>>
> > >>>>
> > >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> > wrote:
> > >>>>>
> > >>>>> I like idea of keeping simple system of modules, so +1 from me.
> > >>>>>
> > >>>>> Where optional libs (e.g Direct IO plugin) would be included, would
> > it
> > >>> be
> > >>>>> core or optional?
> > >>>>>
> > >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> > >>>>>
> > >>>>>>>
> > >>>>>>>>
> > >>>>>>>> How big would be a final core module?
> > >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
> > it’s
> > >>> own
> > >>>>>>> package.
> > >>>>>>
> > >>>>>>
> > >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
> > and
> > >>>>>> propose the simplest modular system:
> > >>>>>>
> > >>>>>> - core module that includes basic Ignite capabilities including
> SQL,
> > >>>>>> compute grid, service grid, k/v
> > >>>>>> - optional module hosts the rest - ML, streamers integration
> (kafka,
> > >>>>>> flink), kubernetes, etc.
> > >>>>>>
> > >>>>>> What do you think?
> > >>>>>>
> > >>>>>> --
> > >>>>>> Denis
> > >>>>>>
> > >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <
> mr.weider@gmail.com>
> > >>> wrote:
> > >>>>>>
> > >>>>>>> *DEB package
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> > wrote:
> > >>>>>>>>
> > >>>>>>>> Considering that DEV package for now is almost platform
> > independent
> > >>>>>> (its
> > >>>>>>> a java application more or less), that package will work almost
> on
> > any
> > >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian,
> etc.
> > >>>>>>>> The only restriction is existence of systemctl (systemd) service
> > >>>>>> manager
> > >>>>>>> — we are dependent on it.
> > >>>>>>>>
> > >>>>>>>> Thats why, for instance, our RPM repository is called simply
> ‘rpm’
> > >>> and
> > >>>>>>> package has no arch or dist suffix — it will work on CentOS,
> RHEL,
> > >>>>>> Fedora,
> > >>>>>>> etc. with presence of aforementioned systemd.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>> Will Debian package work for Ubuntu?
> > >>>>>>>>>
> > >>>>>>>>> D.
> > >>>>>>>>>
> > >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> > mr.weider@gmail.com>
> > >>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> > official
> > >>>>>>>>>> repositories, there can be a problem from OS community.
> > >>>>>>>>>>
> > >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> > package
> > >>>>>>> build
> > >>>>>>>>>> (package layout / install scripts) for speeding up things and
> > >>>>>> excluding
> > >>>>>>>>>> possible duplication and desynchronisation, so its a matter of
> > ’sit
> > >>>>>>> and do’
> > >>>>>>>>>> rather then some technical research. Thats why I rose
> discussion
> > >>>>>> about
> > >>>>>>>>>> future package architecture, so that after agreement I'm be
> > able to
> > >>>>>>> pack
> > >>>>>>>>>> both RPM and DEB identically.
> > >>>>>>>>>>
> > >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> > current
> > >>> RPM
> > >>>>>>>>>> layout in no time.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> > >>> dsetrakyan@apache.org>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Peter,
> > >>>>>>>>>>>
> > >>>>>>>>>>> I don't think the package size of 280M is going to be a
> > problem at
> > >>>>>>> all,
> > >>>>>>>>>> but
> > >>>>>>>>>>> what you are suggesting can be an improvement down the road.
> > >>>>>>>>>>>
> > >>>>>>>>>>> In the mean time, I think our top priority should be to
> provide
> > >>>>>>> packages
> > >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Agree?
> > >>>>>>>>>>>
> > >>>>>>>>>>> D.
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <
> mr.weider@gmail.com>
> > >>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Hi, Igniters!
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so
> > I'd
> > >>>>>> like
> > >>>>>>> to
> > >>>>>>>>>>>> move
> > >>>>>>>>>>>> forward to further improve and widen AI delivery through
> > >>> packages.
> > >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> > >>> 280M+
> > >>>>>>> and,
> > >>>>>>>>>> to
> > >>>>>>>>>>>> improve usability and significantly reduce required download
> > >>>>>> sizes, I
> > >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery
> as
> > >>>>>>> follows:
> > >>>>>>>>>>>> - CORE
> > >>>>>>>>>>>> - bin
> > >>>>>>>>>>>> - config
> > >>>>>>>>>>>> - libs (!optional)
> > >>>>>>>>>>>> - OPTIONAL LIBS
> > >>>>>>>>>>>> - BENCHMARKS
> > >>>>>>>>>>>> - DOCS (?)
> > >>>>>>>>>>>> - EXAMPLES
> > >>>>>>>>>>>> - .NET PLATFORM FILES
> > >>>>>>>>>>>> - C++ PLATFORM FILES
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> > reason
> > >>> to
> > >>>>>>>>>> download
> > >>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> > >>>>>>> functionality)
> > >>>>>>>>>>>> and
> > >>>>>>>>>>>> maintainability (you are in full control of what is
> installed
> > on
> > >>>>>> your
> > >>>>>>>>>>>> system).
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> After successful architecture choice, same scheme are
> planned
> > to
> > >>> be
> > >>>>>>>>>> used in
> > >>>>>>>>>>>> DEB packages as well.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> --
> > >>>>>>>>>>>> Sent from:
> http://apache-ignite-developers.2346864.n4.nabble.
> > >>> com/
> > >>>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>>
> > >>>
> > >
> >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello! I have tried your RPM scripts.
First of all, I'm not sure that apache-ignite-core is a good name for
package which contains the actual server node kit, and that apache-ignite
is a good name for a package that will install everything (including cpp
bindings). How does other packages solve this naming? I would go with
apache-ignite and apache-ignite-full.
Moreover I did not find a way to start service if default installed JVM is
Java 7 :( I understand it's EOL, still this is something that hit me.
apache-ignite-docs only contains scaladoc for scalar. Who would need a
separate package for that? Why no javadocs? Maybe it's my build problem?
apache-ignite-libs is a totally unexpected package name. apache-ignite core
doesn't depend on it. It doesn't enable anything out of the box. The
package is huge.
Frankly speaking, I'm not sure that improvements over Stage I are enough as
of now. For demo-like activity, we can probably go with one package fits
all.
--
Ilya Kasnacheev
2018-04-12 19:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> If someone from PMCы or Committers still sees necessity about including
> these tasks into Apache Ignite 2.5 release, this is the last chance to do
> so.
> Otherwise this task will be moved to at 2.6 release at least, or even
> moved to backlog indefinitely.
>
>
>
> > On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
> >
> > To top new RPM architecture off, update to release process is introduced
> — [1] [2].
> >
> > Both tasks (this one and IGNITE-7647) are ready for review and should be
> merged simultaneously.
> >
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-8172
> > [2] https://github.com/apache/ignite-release/pull/1
> >
> >
> >
> >
> >> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com>
> wrote:
> >>
> >> Hello!
> >>
> >> Let me share my idea of how this shoud work. Splitting package into
> >> sub-packages should be dependency-driven.
> >>
> >> It means that all Ignite modules without dependencies or with small
> >> dependencies (such as ignite-log4j) should be included in ignite-core.
> It
> >> doesn't make sense to make a zillion RPM packages.
> >>
> >> Critical things like ignite-spring and ignite-indexing should be in
> >> ignite-core of course, even if they have dependencies. Ignite-core
> should
> >> be fully self-sufficient and feature-complete.
> >>
> >> However, e.g. .net API should probably be in a separate package,
> because it
> >> should depend on mono | net-core. We may also have ignite-devel package
> >> which should include all modules which only make sense for developers
> who
> >> write code. Such as hibernate integration.
> >>
> >> I'm not sure about MR modules. The main question should be, does it have
> >> dependencies? Can it run stand-alone without writing code?
> >>
> >> Hope this helps,
> >>
> >> --
> >> Ilya Kasnacheev
> >>
> >> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>
> >>> Hi, Igniters!
> >>>
> >>>
> >>> Here are some news on our RPM packages initiative.
> >>>
> >>> 1. I’ve finished preliminary developing of Stage II version of RPM
> >>> packages [1]. Main “new feature” is — split design. Also I’ve added
> >>> package.sh script for automating package building process which will
> help
> >>> organise corresponding builds in TC as well as simplify process for
> >>> developers who wishes to have custom packages.
> >>> PR#3703 [2] is ready for review. Denis, in order to catch up with
> Apache
> >>> Ignite 2.5 release, I’d greatly appreciate your help in finding
> reviewer.
> >>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> >>> repositories on Apache Bintray. Though they are already prepared for
> >>> hosting RPM and DEB packages respectively, and there is a way of
> linking
> >>> them to apache.org/dist/ignite page, there is possible alternative in
> >>> storing there only plain directory layout corresponding to each
> repository
> >>> type (RPM and DEB) and manage this layout (repodata, distributions,
> >>> versions, etc.) by ourselves, having more control over repositories but
> >>> lacking some simplicity of deploying new releases. WDYT? Should we try
> >>> Cassandra approach? They are storing their DEB packages as I described
> >>> above [5].
> >>>
> >>> Also — a question arose while I was working on this issue: which OSes
> (and
> >>> which versions of each) are we going to support (if we are going) in
> terms
> >>> of step-by-step list? Currently RPM packages are tested only with
> latest
> >>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> >>> distributives [6] some of which are more o less popular among OS
> community
> >>> (ALT, Fedora, openSUSE, etc.).
> >>>
> >>>
> >>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> >>> [2] https://github.com/apache/ignite/pull/3703
> >>> [3] https://bintray.com/apache/ignite-rpm
> >>> [4] https://bintray.com/apache/ignite-deb
> >>> [5] https://bintray.com/apache/cassandra/debian#files/
> >>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> distributions
> >>>
> >>>
> >>>
> >>>
> >>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>
> >>>> I suppose that most everything if not all from libs/options will go to
> >>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> >>>> More precise lib selection (if something from optional would better to
> >>> have in core package) will be discussed right after preliminary split
> >>> architecture agreement.
> >>>>
> >>>>
> >>>>
> >>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> wrote:
> >>>>>
> >>>>> I like idea of keeping simple system of modules, so +1 from me.
> >>>>>
> >>>>> Where optional libs (e.g Direct IO plugin) would be included, would
> it
> >>> be
> >>>>> core or optional?
> >>>>>
> >>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >>>>>
> >>>>>>>
> >>>>>>>>
> >>>>>>>> How big would be a final core module?
> >>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
> it’s
> >>> own
> >>>>>>> package.
> >>>>>>
> >>>>>>
> >>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
> and
> >>>>>> propose the simplest modular system:
> >>>>>>
> >>>>>> - core module that includes basic Ignite capabilities including SQL,
> >>>>>> compute grid, service grid, k/v
> >>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
> >>>>>> flink), kubernetes, etc.
> >>>>>>
> >>>>>> What do you think?
> >>>>>>
> >>>>>> --
> >>>>>> Denis
> >>>>>>
> >>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
> >>> wrote:
> >>>>>>
> >>>>>>> *DEB package
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com>
> wrote:
> >>>>>>>>
> >>>>>>>> Considering that DEV package for now is almost platform
> independent
> >>>>>> (its
> >>>>>>> a java application more or less), that package will work almost on
> any
> >>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>>>>> The only restriction is existence of systemctl (systemd) service
> >>>>>> manager
> >>>>>>> — we are dependent on it.
> >>>>>>>>
> >>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> >>> and
> >>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>>>>> Fedora,
> >>>>>>> etc. with presence of aforementioned systemd.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Will Debian package work for Ubuntu?
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <
> mr.weider@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> official
> >>>>>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>>>>
> >>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> package
> >>>>>>> build
> >>>>>>>>>> (package layout / install scripts) for speeding up things and
> >>>>>> excluding
> >>>>>>>>>> possible duplication and desynchronisation, so its a matter of
> ’sit
> >>>>>>> and do’
> >>>>>>>>>> rather then some technical research. Thats why I rose discussion
> >>>>>> about
> >>>>>>>>>> future package architecture, so that after agreement I'm be
> able to
> >>>>>>> pack
> >>>>>>>>>> both RPM and DEB identically.
> >>>>>>>>>>
> >>>>>>>>>> Yet, if you insist, I can create DEB package according to
> current
> >>> RPM
> >>>>>>>>>> layout in no time.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Peter,
> >>>>>>>>>>>
> >>>>>>>>>>> I don't think the package size of 280M is going to be a
> problem at
> >>>>>>> all,
> >>>>>>>>>> but
> >>>>>>>>>>> what you are suggesting can be an improvement down the road.
> >>>>>>>>>>>
> >>>>>>>>>>> In the mean time, I think our top priority should be to provide
> >>>>>>> packages
> >>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>>>>>>>>
> >>>>>>>>>>> Agree?
> >>>>>>>>>>>
> >>>>>>>>>>> D.
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> >>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so
> I'd
> >>>>>> like
> >>>>>>> to
> >>>>>>>>>>>> move
> >>>>>>>>>>>> forward to further improve and widen AI delivery through
> >>> packages.
> >>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> >>> 280M+
> >>>>>>> and,
> >>>>>>>>>> to
> >>>>>>>>>>>> improve usability and significantly reduce required download
> >>>>>> sizes, I
> >>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
> >>>>>>> follows:
> >>>>>>>>>>>> - CORE
> >>>>>>>>>>>> - bin
> >>>>>>>>>>>> - config
> >>>>>>>>>>>> - libs (!optional)
> >>>>>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>>>>> - BENCHMARKS
> >>>>>>>>>>>> - DOCS (?)
> >>>>>>>>>>>> - EXAMPLES
> >>>>>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>>>>
> >>>>>>>>>>>> This architecture, as I assume, will add flexibility (no
> reason
> >>> to
> >>>>>>>>>> download
> >>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> >>>>>>> functionality)
> >>>>>>>>>>>> and
> >>>>>>>>>>>> maintainability (you are in full control of what is installed
> on
> >>>>>> your
> >>>>>>>>>>>> system).
> >>>>>>>>>>>>
> >>>>>>>>>>>> After successful architecture choice, same scheme are planned
> to
> >>> be
> >>>>>>>>>> used in
> >>>>>>>>>>>> DEB packages as well.
> >>>>>>>>>>>>
> >>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> >>> com/
> >>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>
> >>>
> >
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
If someone from PMCы or Committers still sees necessity about including these tasks into Apache Ignite 2.5 release, this is the last chance to do so.
Otherwise this task will be moved to at 2.6 release at least, or even moved to backlog indefinitely.
> On 9 Apr 2018, at 19:08, Petr Ivanov <mr...@gmail.com> wrote:
>
> To top new RPM architecture off, update to release process is introduced — [1] [2].
>
> Both tasks (this one and IGNITE-7647) are ready for review and should be merged simultaneously.
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-8172
> [2] https://github.com/apache/ignite-release/pull/1
>
>
>
>
>> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com> wrote:
>>
>> Hello!
>>
>> Let me share my idea of how this shoud work. Splitting package into
>> sub-packages should be dependency-driven.
>>
>> It means that all Ignite modules without dependencies or with small
>> dependencies (such as ignite-log4j) should be included in ignite-core. It
>> doesn't make sense to make a zillion RPM packages.
>>
>> Critical things like ignite-spring and ignite-indexing should be in
>> ignite-core of course, even if they have dependencies. Ignite-core should
>> be fully self-sufficient and feature-complete.
>>
>> However, e.g. .net API should probably be in a separate package, because it
>> should depend on mono | net-core. We may also have ignite-devel package
>> which should include all modules which only make sense for developers who
>> write code. Such as hibernate integration.
>>
>> I'm not sure about MR modules. The main question should be, does it have
>> dependencies? Can it run stand-alone without writing code?
>>
>> Hope this helps,
>>
>> --
>> Ilya Kasnacheev
>>
>> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>
>>> Hi, Igniters!
>>>
>>>
>>> Here are some news on our RPM packages initiative.
>>>
>>> 1. I’ve finished preliminary developing of Stage II version of RPM
>>> packages [1]. Main “new feature” is — split design. Also I’ve added
>>> package.sh script for automating package building process which will help
>>> organise corresponding builds in TC as well as simplify process for
>>> developers who wishes to have custom packages.
>>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>>> repositories on Apache Bintray. Though they are already prepared for
>>> hosting RPM and DEB packages respectively, and there is a way of linking
>>> them to apache.org/dist/ignite page, there is possible alternative in
>>> storing there only plain directory layout corresponding to each repository
>>> type (RPM and DEB) and manage this layout (repodata, distributions,
>>> versions, etc.) by ourselves, having more control over repositories but
>>> lacking some simplicity of deploying new releases. WDYT? Should we try
>>> Cassandra approach? They are storing their DEB packages as I described
>>> above [5].
>>>
>>> Also — a question arose while I was working on this issue: which OSes (and
>>> which versions of each) are we going to support (if we are going) in terms
>>> of step-by-step list? Currently RPM packages are tested only with latest
>>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>>> distributives [6] some of which are more o less popular among OS community
>>> (ALT, Fedora, openSUSE, etc.).
>>>
>>>
>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>> [2] https://github.com/apache/ignite/pull/3703
>>> [3] https://bintray.com/apache/ignite-rpm
>>> [4] https://bintray.com/apache/ignite-deb
>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>>
>>>
>>>
>>>
>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>>>>
>>>> I suppose that most everything if not all from libs/options will go to
>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>> More precise lib selection (if something from optional would better to
>>> have in core package) will be discussed right after preliminary split
>>> architecture agreement.
>>>>
>>>>
>>>>
>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>>>>>
>>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>>
>>>>> Where optional libs (e.g Direct IO plugin) would be included, would it
>>> be
>>>>> core or optional?
>>>>>
>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> How big would be a final core module?
>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
>>> own
>>>>>>> package.
>>>>>>
>>>>>>
>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>>>>>> propose the simplest modular system:
>>>>>>
>>>>>> - core module that includes basic Ignite capabilities including SQL,
>>>>>> compute grid, service grid, k/v
>>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>>>>> flink), kubernetes, etc.
>>>>>>
>>>>>> What do you think?
>>>>>>
>>>>>> --
>>>>>> Denis
>>>>>>
>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
>>> wrote:
>>>>>>
>>>>>>> *DEB package
>>>>>>>
>>>>>>>
>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>>>
>>>>>>>> Considering that DEV package for now is almost platform independent
>>>>>> (its
>>>>>>> a java application more or less), that package will work almost on any
>>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>>>>>> The only restriction is existence of systemctl (systemd) service
>>>>>> manager
>>>>>>> — we are dependent on it.
>>>>>>>>
>>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
>>> and
>>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>>>>>> Fedora,
>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to official
>>>>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>>>>
>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>>>>>> build
>>>>>>>>>> (package layout / install scripts) for speeding up things and
>>>>>> excluding
>>>>>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
>>>>>>> and do’
>>>>>>>>>> rather then some technical research. Thats why I rose discussion
>>>>>> about
>>>>>>>>>> future package architecture, so that after agreement I'm be able to
>>>>>>> pack
>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>>
>>>>>>>>>> Yet, if you insist, I can create DEB package according to current
>>> RPM
>>>>>>>>>> layout in no time.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>>> dsetrakyan@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Peter,
>>>>>>>>>>>
>>>>>>>>>>> I don't think the package size of 280M is going to be a problem at
>>>>>>> all,
>>>>>>>>>> but
>>>>>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>>>>>
>>>>>>>>>>> In the mean time, I think our top priority should be to provide
>>>>>>> packages
>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>>>>>
>>>>>>>>>>> Agree?
>>>>>>>>>>>
>>>>>>>>>>> D.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>>>>>> like
>>>>>>> to
>>>>>>>>>>>> move
>>>>>>>>>>>> forward to further improve and widen AI delivery through
>>> packages.
>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
>>> 280M+
>>>>>>> and,
>>>>>>>>>> to
>>>>>>>>>>>> improve usability and significantly reduce required download
>>>>>> sizes, I
>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>>>>>> follows:
>>>>>>>>>>>> - CORE
>>>>>>>>>>>> - bin
>>>>>>>>>>>> - config
>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>>
>>>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason
>>> to
>>>>>>>>>> download
>>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
>>>>>>> functionality)
>>>>>>>>>>>> and
>>>>>>>>>>>> maintainability (you are in full control of what is installed on
>>>>>> your
>>>>>>>>>>>> system).
>>>>>>>>>>>>
>>>>>>>>>>>> After successful architecture choice, same scheme are planned to
>>> be
>>>>>>>>>> used in
>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>>
>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>> com/
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
To top new RPM architecture off, update to release process is introduced — [1] [2].
Both tasks (this one and IGNITE-7647) are ready for review and should be merged simultaneously.
[1] https://issues.apache.org/jira/browse/IGNITE-8172
[2] https://github.com/apache/ignite-release/pull/1
> On 2 Apr 2018, at 18:22, Ilya Kasnacheev <il...@gmail.com> wrote:
>
> Hello!
>
> Let me share my idea of how this shoud work. Splitting package into
> sub-packages should be dependency-driven.
>
> It means that all Ignite modules without dependencies or with small
> dependencies (such as ignite-log4j) should be included in ignite-core. It
> doesn't make sense to make a zillion RPM packages.
>
> Critical things like ignite-spring and ignite-indexing should be in
> ignite-core of course, even if they have dependencies. Ignite-core should
> be fully self-sufficient and feature-complete.
>
> However, e.g. .net API should probably be in a separate package, because it
> should depend on mono | net-core. We may also have ignite-devel package
> which should include all modules which only make sense for developers who
> write code. Such as hibernate integration.
>
> I'm not sure about MR modules. The main question should be, does it have
> dependencies? Can it run stand-alone without writing code?
>
> Hope this helps,
>
> --
> Ilya Kasnacheev
>
> 2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
>> Hi, Igniters!
>>
>>
>> Here are some news on our RPM packages initiative.
>>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>> repositories on Apache Bintray. Though they are already prepared for
>> hosting RPM and DEB packages respectively, and there is a way of linking
>> them to apache.org/dist/ignite page, there is possible alternative in
>> storing there only plain directory layout corresponding to each repository
>> type (RPM and DEB) and manage this layout (repodata, distributions,
>> versions, etc.) by ourselves, having more control over repositories but
>> lacking some simplicity of deploying new releases. WDYT? Should we try
>> Cassandra approach? They are storing their DEB packages as I described
>> above [5].
>>
>> Also — a question arose while I was working on this issue: which OSes (and
>> which versions of each) are we going to support (if we are going) in terms
>> of step-by-step list? Currently RPM packages are tested only with latest
>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>> distributives [6] some of which are more o less popular among OS community
>> (ALT, Fedora, openSUSE, etc.).
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>> [2] https://github.com/apache/ignite/pull/3703
>> [3] https://bintray.com/apache/ignite-rpm
>> [4] https://bintray.com/apache/ignite-deb
>> [5] https://bintray.com/apache/cassandra/debian#files/
>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>
>>
>>
>>
>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>> I suppose that most everything if not all from libs/options will go to
>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>> More precise lib selection (if something from optional would better to
>> have in core package) will be discussed right after preliminary split
>> architecture agreement.
>>>
>>>
>>>
>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>>>>
>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>
>>>> Where optional libs (e.g Direct IO plugin) would be included, would it
>> be
>>>> core or optional?
>>>>
>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>>>>
>>>>>>
>>>>>>>
>>>>>>> How big would be a final core module?
>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
>> own
>>>>>> package.
>>>>>
>>>>>
>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>>>>> propose the simplest modular system:
>>>>>
>>>>> - core module that includes basic Ignite capabilities including SQL,
>>>>> compute grid, service grid, k/v
>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>>>> flink), kubernetes, etc.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
>> wrote:
>>>>>
>>>>>> *DEB package
>>>>>>
>>>>>>
>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>>
>>>>>>> Considering that DEV package for now is almost platform independent
>>>>> (its
>>>>>> a java application more or less), that package will work almost on any
>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>>>>> The only restriction is existence of systemctl (systemd) service
>>>>> manager
>>>>>> — we are dependent on it.
>>>>>>>
>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
>> and
>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>>>>> Fedora,
>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to official
>>>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>>>
>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>>>>> build
>>>>>>>>> (package layout / install scripts) for speeding up things and
>>>>> excluding
>>>>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
>>>>>> and do’
>>>>>>>>> rather then some technical research. Thats why I rose discussion
>>>>> about
>>>>>>>>> future package architecture, so that after agreement I'm be able to
>>>>>> pack
>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>
>>>>>>>>> Yet, if you insist, I can create DEB package according to current
>> RPM
>>>>>>>>> layout in no time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Peter,
>>>>>>>>>>
>>>>>>>>>> I don't think the package size of 280M is going to be a problem at
>>>>>> all,
>>>>>>>>> but
>>>>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>>>>
>>>>>>>>>> In the mean time, I think our top priority should be to provide
>>>>>> packages
>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>>>>
>>>>>>>>>> Agree?
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>>>>> like
>>>>>> to
>>>>>>>>>>> move
>>>>>>>>>>> forward to further improve and widen AI delivery through
>> packages.
>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
>> 280M+
>>>>>> and,
>>>>>>>>> to
>>>>>>>>>>> improve usability and significantly reduce required download
>>>>> sizes, I
>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>>>>> follows:
>>>>>>>>>>> - CORE
>>>>>>>>>>> - bin
>>>>>>>>>>> - config
>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>
>>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason
>> to
>>>>>>>>> download
>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
>>>>>> functionality)
>>>>>>>>>>> and
>>>>>>>>>>> maintainability (you are in full control of what is installed on
>>>>> your
>>>>>>>>>>> system).
>>>>>>>>>>>
>>>>>>>>>>> After successful architecture choice, same scheme are planned to
>> be
>>>>>>>>> used in
>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>> com/
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Let me share my idea of how this shoud work. Splitting package into
sub-packages should be dependency-driven.
It means that all Ignite modules without dependencies or with small
dependencies (such as ignite-log4j) should be included in ignite-core. It
doesn't make sense to make a zillion RPM packages.
Critical things like ignite-spring and ignite-indexing should be in
ignite-core of course, even if they have dependencies. Ignite-core should
be fully self-sufficient and feature-complete.
However, e.g. .net API should probably be in a separate package, because it
should depend on mono | net-core. We may also have ignite-devel package
which should include all modules which only make sense for developers who
write code. Such as hibernate integration.
I'm not sure about MR modules. The main question should be, does it have
dependencies? Can it run stand-alone without writing code?
Hope this helps,
--
Ilya Kasnacheev
2018-03-27 15:10 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> Hi, Igniters!
>
>
> Here are some news on our RPM packages initiative.
>
> 1. I’ve finished preliminary developing of Stage II version of RPM
> packages [1]. Main “new feature” is — split design. Also I’ve added
> package.sh script for automating package building process which will help
> organise corresponding builds in TC as well as simplify process for
> developers who wishes to have custom packages.
> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> repositories on Apache Bintray. Though they are already prepared for
> hosting RPM and DEB packages respectively, and there is a way of linking
> them to apache.org/dist/ignite page, there is possible alternative in
> storing there only plain directory layout corresponding to each repository
> type (RPM and DEB) and manage this layout (repodata, distributions,
> versions, etc.) by ourselves, having more control over repositories but
> lacking some simplicity of deploying new releases. WDYT? Should we try
> Cassandra approach? They are storing their DEB packages as I described
> above [5].
>
> Also — a question arose while I was working on this issue: which OSes (and
> which versions of each) are we going to support (if we are going) in terms
> of step-by-step list? Currently RPM packages are tested only with latest
> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> distributives [6] some of which are more o less popular among OS community
> (ALT, Fedora, openSUSE, etc.).
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> [2] https://github.com/apache/ignite/pull/3703
> [3] https://bintray.com/apache/ignite-rpm
> [4] https://bintray.com/apache/ignite-deb
> [5] https://bintray.com/apache/cassandra/debian#files/
> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>
>
>
>
> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> >
> > I suppose that most everything if not all from libs/options will go to
> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > More precise lib selection (if something from optional would better to
> have in core package) will be discussed right after preliminary split
> architecture agreement.
> >
> >
> >
> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
> >>
> >> I like idea of keeping simple system of modules, so +1 from me.
> >>
> >> Where optional libs (e.g Direct IO plugin) would be included, would it
> be
> >> core or optional?
> >>
> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >>
> >>>>
> >>>>>
> >>>>> How big would be a final core module?
> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
> own
> >>>> package.
> >>>
> >>>
> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
> >>> propose the simplest modular system:
> >>>
> >>> - core module that includes basic Ignite capabilities including SQL,
> >>> compute grid, service grid, k/v
> >>> - optional module hosts the rest - ML, streamers integration (kafka,
> >>> flink), kubernetes, etc.
> >>>
> >>> What do you think?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
> wrote:
> >>>
> >>>> *DEB package
> >>>>
> >>>>
> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>>
> >>>>> Considering that DEV package for now is almost platform independent
> >>> (its
> >>>> a java application more or less), that package will work almost on any
> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>> The only restriction is existence of systemctl (systemd) service
> >>> manager
> >>>> — we are dependent on it.
> >>>>>
> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> and
> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>> Fedora,
> >>>> etc. with presence of aforementioned systemd.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> Will Debian package work for Ubuntu?
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Not a problem, rather nuisance. Also, when we will move to official
> >>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>
> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
> >>>> build
> >>>>>>> (package layout / install scripts) for speeding up things and
> >>> excluding
> >>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
> >>>> and do’
> >>>>>>> rather then some technical research. Thats why I rose discussion
> >>> about
> >>>>>>> future package architecture, so that after agreement I'm be able to
> >>>> pack
> >>>>>>> both RPM and DEB identically.
> >>>>>>>
> >>>>>>> Yet, if you insist, I can create DEB package according to current
> RPM
> >>>>>>> layout in no time.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Peter,
> >>>>>>>>
> >>>>>>>> I don't think the package size of 280M is going to be a problem at
> >>>> all,
> >>>>>>> but
> >>>>>>>> what you are suggesting can be an improvement down the road.
> >>>>>>>>
> >>>>>>>> In the mean time, I think our top priority should be to provide
> >>>> packages
> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>>>>>
> >>>>>>>> Agree?
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi, Igniters!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
> >>> like
> >>>> to
> >>>>>>>>> move
> >>>>>>>>> forward to further improve and widen AI delivery through
> packages.
> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> 280M+
> >>>> and,
> >>>>>>> to
> >>>>>>>>> improve usability and significantly reduce required download
> >>> sizes, I
> >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
> >>>> follows:
> >>>>>>>>> - CORE
> >>>>>>>>> - bin
> >>>>>>>>> - config
> >>>>>>>>> - libs (!optional)
> >>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>> - BENCHMARKS
> >>>>>>>>> - DOCS (?)
> >>>>>>>>> - EXAMPLES
> >>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>
> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason
> to
> >>>>>>> download
> >>>>>>>>> all 280M+ of binaries where you are to run only core node
> >>>> functionality)
> >>>>>>>>> and
> >>>>>>>>> maintainability (you are in full control of what is installed on
> >>> your
> >>>>>>>>> system).
> >>>>>>>>>
> >>>>>>>>> After successful architecture choice, same scheme are planned to
> be
> >>>>>>> used in
> >>>>>>>>> DEB packages as well.
> >>>>>>>>>
> >>>>>>>>> WDYT?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> com/
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Max Shonichev <ms...@yandex.ru>.
Peter, great work!
I'll check up PR in the near time.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
> On 29 Mar 2018, at 12:32, Max Shonichev <ms...@yandex.ru> wrote:
>
>
>
> 1. About packaging/package.sh
>
> 1.1. I personally dislike the use of bashism, e.g. like reserved keywords,
> double '[[', double '((', but that's ok
> for targeted distro (RHEL/Centos). The only requirement is please add more
> comments in tough places
> to improve code readability and maintanbility.
Will try to get rid of them as much as possible, but SH is really outdated scripting language and most all of the systems now support BASH, even Windows :)
>
>
> 1.2. few code style comments:
>
>>> name=$(cat /etc/*release | grep ^NAME | sed -r 's|.*"(.*)".*|\1|')
>
> add
>
> | head -n 1
>
> or else it will fail in case more than one file with *release mask in /etc
I disagree. Script will not fail, just cat every file matched by mask and then pass everything as one single stdout.
>
>
> 1.3. rework prepEnv
>
>>> if [ ! -z installCmd ]; then
>
> this will fail miserably, replace with
>
> if [ ! -z $installCmd ]; then
Fixed
>
> 1.3.1. however running package.sh under Ubuntu still would fail due to
> $installCmd is unpacked to 'apt --no-install-recommends'
> and -z 'apt --no-install-recommends' would always return false.
The main purpose of checking that variable ${installCmd} is not empty — because it is instantiated empty and will stay empty if OS detection won’t be able to define OS and pre-build software installation and environment preparation will be skipped.
>
> 1.3.2. and still, when apt/yum is installed and rpm/rpm-build is not, the
> whole prepEnv would fail to ensure all
> build-time requirements are installed, because it only checks for apt/yum.
I suppose that apt / yum is always installed by default because it is default package manager for corresponding OSes.
I did not get where is error here.
>
> 1.3.3. installing requires root privileges. However, no checks for $EUID nor
> running apt/yum with 'sudo' present.
> Please, add this check, otherwise running package.sh under unprivileged user
> with no installed build-time dependencies
> will fail with very undescriptive errors.
Check for ${EUID} will mean that whole build package process will run under root, which is undesirable.
Added INFO message that explains necessity for super user privileges and added sudo command to root-required commands.
>
>
> 1.4. getBin
>
> 1.4.1. I don't like detecting target apache ignite package version by
> parsing embedded changelog. This might fail
> when any package text description would contain '*' in the first char of the
> string. Either move changelog to separate
> Changelog file and %include it in .spec or parse for, e.g. '^Version:
> [0-9\.]+'
>
>>> igniteVersion=$(cat rpm/SPECS/apache-ignite.spec | grep '^*' | head -1 |
>>> sed -r 's|.*\ -\ (.*)-.*|\1|’)
Replaced with parsing of main version
>
> 1.4.2.
>
>>> ls ${binName} && binPreparedFlag=true || true
>
> why not use
>
> if [ ! -f $binName ];
Indeed this variant is much better, fixed.
>
> 1.4.3. 'curl' is used, but not checked as build-time dependency.
Added to build dependencies (via prepEnv function)
>
> 1.4.4. we try to find and download 'apache-ignite-fabric-$igniteVersion',
> that's ok for <2.4 releases, but in future
> we would need to find and download 'apache-ignite-$igniteVersion' too.
> Better add this check as a fallback option, e.g.
> try to use curl with both urls in correct order.
>
> if curl --output /dev/null --silent --head --fail "$url"; then
> echo "URL exists: $url"
> else
> echo "URL does not exist: $url"
> fi
1. Packages did not exist before 2.4.0.
2. Build script does not provide opportunity to build packages of different from .spec versions and is considered to be part of packaging build process (specifications and packaging script are modified together) — a shortcut, thus relying on current (same commit) state for building.
>
> 1.4.5. also strange, but may be ok that we download
> apache-ignite-fabric-2.4.0
Fabric removal task is not yet merged to master.
Will be supported in its reimplementation
>
>
> 2. about apache-ignite.spec
>
> 2.1. fix typo
>
>>> Apache Ignite's optinal libs and integrations
Fixed
>
> 2.2. fix description
>
>>> C++ files necessary for using Apache Ignite
>
> platform files are not necessary for Ignite, only for integration with .NET,
> afaik?
Introduced alternative description
>
> 2.3. Tough place with 'Obsoletes: apache-ignite < 2.5.0'
>
> I understand the current logic behind splitting apache-ignite to several
> .rpm's and setting obsolete for the old
> apache-ignite.
>
> However, for future releases, e.g. if/when we handover this task to another
> engineer, it is not quite obvious that one
> must (or must not?) update version in 5 different places, including
> changelog. Please add a few lines about
> 'How to package next (new) version of Apache Ignite RPMs' to Confluence.
Everything that relies on package version is mapped to %{version} variable and defined only once.
Other versions in dependencies directives will stay the same in future specification updates.
Task for description of RPM specification update process, I guess, is out of scope of current issue. However I agree with necessity of such documentation in future and will create corresponding task in the nearest time.
>
> 2.4. firewalld / firewall-cmd is optionally used if installed, but not set
> in the dependency. It is OK, but may be we
> add this as a post-install note, like we do recommendation messages in node
> logs.
Firewall-CMD rules from package is an override for current installation rather then necessity in some firewall implementations required for service usage.
>
>
> 3. service scripts : we'll should add a test suite for it, overall looks
> pretty solid.
Agree. Will create corresponding task in the nearest time
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Apache Ignite RPM packaging (Stage II)
Posted by Max Shonichev <ms...@yandex.ru>.
1. About packaging/package.sh
1.1. I personally dislike the use of bashism, e.g. like reserved keywords,
double '[[', double '((', but that's ok
for targeted distro (RHEL/Centos). The only requirement is please add more
comments in tough places
to improve code readability and maintanbility.
1.2. few code style comments:
>> name=$(cat /etc/*release | grep ^NAME | sed -r 's|.*"(.*)".*|\1|')
add
| head -n 1
or else it will fail in case more than one file with *release mask in /etc
1.3. rework prepEnv
>> if [ ! -z installCmd ]; then
this will fail miserably, replace with
if [ ! -z $installCmd ]; then
1.3.1. however running package.sh under Ubuntu still would fail due to
$installCmd is unpacked to 'apt --no-install-recommends'
and -z 'apt --no-install-recommends' would always return false.
1.3.2. and still, when apt/yum is installed and rpm/rpm-build is not, the
whole prepEnv would fail to ensure all
build-time requirements are installed, because it only checks for apt/yum.
1.3.3. installing requires root privileges. However, no checks for $EUID nor
running apt/yum with 'sudo' present.
Please, add this check, otherwise running package.sh under unprivileged user
with no installed build-time dependencies
will fail with very undescriptive errors.
1.4. getBin
1.4.1. I don't like detecting target apache ignite package version by
parsing embedded changelog. This might fail
when any package text description would contain '*' in the first char of the
string. Either move changelog to separate
Changelog file and %include it in .spec or parse for, e.g. '^Version:
[0-9\.]+'
>> igniteVersion=$(cat rpm/SPECS/apache-ignite.spec | grep '^*' | head -1 |
>> sed -r 's|.*\ -\ (.*)-.*|\1|')
1.4.2.
>> ls ${binName} && binPreparedFlag=true || true
why not use
if [ ! -f $binName ];
1.4.3. 'curl' is used, but not checked as build-time dependency.
1.4.4. we try to find and download 'apache-ignite-fabric-$igniteVersion',
that's ok for <2.4 releases, but in future
we would need to find and download 'apache-ignite-$igniteVersion' too.
Better add this check as a fallback option, e.g.
try to use curl with both urls in correct order.
if curl --output /dev/null --silent --head --fail "$url"; then
echo "URL exists: $url"
else
echo "URL does not exist: $url"
fi
1.4.5. also strange, but may be ok that we download
apache-ignite-fabric-2.4.0
2. about apache-ignite.spec
2.1. fix typo
>> Apache Ignite's optinal libs and integrations
2.2. fix description
>> C++ files necessary for using Apache Ignite
platform files are not necessary for Ignite, only for integration with .NET,
afaik?
2.3. Tough place with 'Obsoletes: apache-ignite < 2.5.0'
I understand the current logic behind splitting apache-ignite to several
.rpm's and setting obsolete for the old
apache-ignite.
However, for future releases, e.g. if/when we handover this task to another
engineer, it is not quite obvious that one
must (or must not?) update version in 5 different places, including
changelog. Please add a few lines about
'How to package next (new) version of Apache Ignite RPMs' to Confluence.
2.4. firewalld / firewall-cmd is optionally used if installed, but not set
in the dependency. It is OK, but may be we
add this as a post-install note, like we do recommendation messages in node
logs.
3. service scripts : we'll should add a test suite for it, overall looks
pretty solid.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
I can start preparing DEB packages right after adding RPM build to nightly release build (as an experiment / experience for future addition of packages build into release process) basing on current RPM architecture.
I will create branch from IGNITE-7647, then.
> On 28 Mar 2018, at 10:06, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> Thanks, Petr!
>
> I would love to test the package installation, but I can only do it on
> Ubuntu. Do you know when will we be able to get the Debian instructions,
> similar to this:
>
> https://ignite.apache.org/download.cgi#rpm-package
>
> D.
>
> On Wed, Mar 28, 2018 at 12:01 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
>> No, not yet.
>>
>>
>> Currently we are discussing RPM packages only.
>> I want to get all feedback and possible errors working on RPM packages, so
>> that when we have stable agreed architecture and etc. I can recreate it in
>> DEB packages without necessity to fix bugs in both RPM and DEB packages
>> simultaneously.
>>
>>
>>
>>> On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>>
>>> Petr,
>>>
>>> I am confused. Do we already have Debian packages?
>>>
>>> D.
>>>
>>> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr...@gmail.com>
>> wrote:
>>>
>>>> Hi, Igniters!
>>>>
>>>>
>>>> Here are some news on our RPM packages initiative.
>>>>
>>>> 1. I’ve finished preliminary developing of Stage II version of RPM
>>>> packages [1]. Main “new feature” is — split design. Also I’ve added
>>>> package.sh script for automating package building process which will
>> help
>>>> organise corresponding builds in TC as well as simplify process for
>>>> developers who wishes to have custom packages.
>>>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>>>> Ignite 2.5 release, I’d greatly appreciate your help in finding
>> reviewer.
>>>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>>>> repositories on Apache Bintray. Though they are already prepared for
>>>> hosting RPM and DEB packages respectively, and there is a way of linking
>>>> them to apache.org/dist/ignite page, there is possible alternative in
>>>> storing there only plain directory layout corresponding to each
>> repository
>>>> type (RPM and DEB) and manage this layout (repodata, distributions,
>>>> versions, etc.) by ourselves, having more control over repositories but
>>>> lacking some simplicity of deploying new releases. WDYT? Should we try
>>>> Cassandra approach? They are storing their DEB packages as I described
>>>> above [5].
>>>>
>>>> Also — a question arose while I was working on this issue: which OSes
>> (and
>>>> which versions of each) are we going to support (if we are going) in
>> terms
>>>> of step-by-step list? Currently RPM packages are tested only with latest
>>>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>>>> distributives [6] some of which are more o less popular among OS
>> community
>>>> (ALT, Fedora, openSUSE, etc.).
>>>>
>>>>
>>>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>>>> [2] https://github.com/apache/ignite/pull/3703
>>>> [3] https://bintray.com/apache/ignite-rpm
>>>> [4] https://bintray.com/apache/ignite-deb
>>>> [5] https://bintray.com/apache/cassandra/debian#files/
>>>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
>> distributions
>>>>
>>>>
>>>>
>>>>
>>>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>
>>>>> I suppose that most everything if not all from libs/options will go to
>>>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>>>> More precise lib selection (if something from optional would better to
>>>> have in core package) will be discussed right after preliminary split
>>>> architecture agreement.
>>>>>
>>>>>
>>>>>
>>>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
>> wrote:
>>>>>>
>>>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>>>
>>>>>> Where optional libs (e.g Direct IO plugin) would be included, would it
>>>> be
>>>>>> core or optional?
>>>>>>
>>>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> How big would be a final core module?
>>>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
>> it’s
>>>> own
>>>>>>>> package.
>>>>>>>
>>>>>>>
>>>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
>> and
>>>>>>> propose the simplest modular system:
>>>>>>>
>>>>>>> - core module that includes basic Ignite capabilities including SQL,
>>>>>>> compute grid, service grid, k/v
>>>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>>>>>> flink), kubernetes, etc.
>>>>>>>
>>>>>>> What do you think?
>>>>>>>
>>>>>>> --
>>>>>>> Denis
>>>>>>>
>>>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
>>>> wrote:
>>>>>>>
>>>>>>>> *DEB package
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> Considering that DEV package for now is almost platform independent
>>>>>>> (its
>>>>>>>> a java application more or less), that package will work almost on
>> any
>>>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>>>>>>> The only restriction is existence of systemctl (systemd) service
>>>>>>> manager
>>>>>>>> — we are dependent on it.
>>>>>>>>>
>>>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
>>>> and
>>>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>>>>>>> Fedora,
>>>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com
>>>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
>> official
>>>>>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>>>>>
>>>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
>> package
>>>>>>>> build
>>>>>>>>>>> (package layout / install scripts) for speeding up things and
>>>>>>> excluding
>>>>>>>>>>> possible duplication and desynchronisation, so its a matter of
>> ’sit
>>>>>>>> and do’
>>>>>>>>>>> rather then some technical research. Thats why I rose discussion
>>>>>>> about
>>>>>>>>>>> future package architecture, so that after agreement I'm be able
>> to
>>>>>>>> pack
>>>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>>>
>>>>>>>>>>> Yet, if you insist, I can create DEB package according to current
>>>> RPM
>>>>>>>>>>> layout in no time.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>>>> dsetrakyan@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Peter,
>>>>>>>>>>>>
>>>>>>>>>>>> I don't think the package size of 280M is going to be a problem
>> at
>>>>>>>> all,
>>>>>>>>>>> but
>>>>>>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>>>>>>
>>>>>>>>>>>> In the mean time, I think our top priority should be to provide
>>>>>>>> packages
>>>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>>>>>>
>>>>>>>>>>>> Agree?
>>>>>>>>>>>>
>>>>>>>>>>>> D.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>>>>>>> like
>>>>>>>> to
>>>>>>>>>>>>> move
>>>>>>>>>>>>> forward to further improve and widen AI delivery through
>>>> packages.
>>>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
>>>> 280M+
>>>>>>>> and,
>>>>>>>>>>> to
>>>>>>>>>>>>> improve usability and significantly reduce required download
>>>>>>> sizes, I
>>>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>>>>>>> follows:
>>>>>>>>>>>>> - CORE
>>>>>>>>>>>>> - bin
>>>>>>>>>>>>> - config
>>>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>>>
>>>>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason
>>>> to
>>>>>>>>>>> download
>>>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
>>>>>>>> functionality)
>>>>>>>>>>>>> and
>>>>>>>>>>>>> maintainability (you are in full control of what is installed
>> on
>>>>>>> your
>>>>>>>>>>>>> system).
>>>>>>>>>>>>>
>>>>>>>>>>>>> After successful architecture choice, same scheme are planned
>> to
>>>> be
>>>>>>>>>>> used in
>>>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>> com/
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Thanks, Petr!
I would love to test the package installation, but I can only do it on
Ubuntu. Do you know when will we be able to get the Debian instructions,
similar to this:
https://ignite.apache.org/download.cgi#rpm-package
D.
On Wed, Mar 28, 2018 at 12:01 AM, Petr Ivanov <mr...@gmail.com> wrote:
> No, not yet.
>
>
> Currently we are discussing RPM packages only.
> I want to get all feedback and possible errors working on RPM packages, so
> that when we have stable agreed architecture and etc. I can recreate it in
> DEB packages without necessity to fix bugs in both RPM and DEB packages
> simultaneously.
>
>
>
> > On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > Petr,
> >
> > I am confused. Do we already have Debian packages?
> >
> > D.
> >
> > On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr...@gmail.com>
> wrote:
> >
> >> Hi, Igniters!
> >>
> >>
> >> Here are some news on our RPM packages initiative.
> >>
> >> 1. I’ve finished preliminary developing of Stage II version of RPM
> >> packages [1]. Main “new feature” is — split design. Also I’ve added
> >> package.sh script for automating package building process which will
> help
> >> organise corresponding builds in TC as well as simplify process for
> >> developers who wishes to have custom packages.
> >> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
> >> Ignite 2.5 release, I’d greatly appreciate your help in finding
> reviewer.
> >> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> >> repositories on Apache Bintray. Though they are already prepared for
> >> hosting RPM and DEB packages respectively, and there is a way of linking
> >> them to apache.org/dist/ignite page, there is possible alternative in
> >> storing there only plain directory layout corresponding to each
> repository
> >> type (RPM and DEB) and manage this layout (repodata, distributions,
> >> versions, etc.) by ourselves, having more control over repositories but
> >> lacking some simplicity of deploying new releases. WDYT? Should we try
> >> Cassandra approach? They are storing their DEB packages as I described
> >> above [5].
> >>
> >> Also — a question arose while I was working on this issue: which OSes
> (and
> >> which versions of each) are we going to support (if we are going) in
> terms
> >> of step-by-step list? Currently RPM packages are tested only with latest
> >> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> >> distributives [6] some of which are more o less popular among OS
> community
> >> (ALT, Fedora, openSUSE, etc.).
> >>
> >>
> >> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> >> [2] https://github.com/apache/ignite/pull/3703
> >> [3] https://bintray.com/apache/ignite-rpm
> >> [4] https://bintray.com/apache/ignite-deb
> >> [5] https://bintray.com/apache/cassandra/debian#files/
> >> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_
> distributions
> >>
> >>
> >>
> >>
> >>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> >>>
> >>> I suppose that most everything if not all from libs/options will go to
> >> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> >>> More precise lib selection (if something from optional would better to
> >> have in core package) will be discussed right after preliminary split
> >> architecture agreement.
> >>>
> >>>
> >>>
> >>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com>
> wrote:
> >>>>
> >>>> I like idea of keeping simple system of modules, so +1 from me.
> >>>>
> >>>> Where optional libs (e.g Direct IO plugin) would be included, would it
> >> be
> >>>> core or optional?
> >>>>
> >>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >>>>
> >>>>>>
> >>>>>>>
> >>>>>>> How big would be a final core module?
> >>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
> it’s
> >> own
> >>>>>> package.
> >>>>>
> >>>>>
> >>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr
> and
> >>>>> propose the simplest modular system:
> >>>>>
> >>>>> - core module that includes basic Ignite capabilities including SQL,
> >>>>> compute grid, service grid, k/v
> >>>>> - optional module hosts the rest - ML, streamers integration (kafka,
> >>>>> flink), kubernetes, etc.
> >>>>>
> >>>>> What do you think?
> >>>>>
> >>>>> --
> >>>>> Denis
> >>>>>
> >>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> *DEB package
> >>>>>>
> >>>>>>
> >>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Considering that DEV package for now is almost platform independent
> >>>>> (its
> >>>>>> a java application more or less), that package will work almost on
> any
> >>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>>>> The only restriction is existence of systemctl (systemd) service
> >>>>> manager
> >>>>>> — we are dependent on it.
> >>>>>>>
> >>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> >> and
> >>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>>>> Fedora,
> >>>>>> etc. with presence of aforementioned systemd.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Will Debian package work for Ubuntu?
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr.weider@gmail.com
> >
> >>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Not a problem, rather nuisance. Also, when we will move to
> official
> >>>>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>>>
> >>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
> package
> >>>>>> build
> >>>>>>>>> (package layout / install scripts) for speeding up things and
> >>>>> excluding
> >>>>>>>>> possible duplication and desynchronisation, so its a matter of
> ’sit
> >>>>>> and do’
> >>>>>>>>> rather then some technical research. Thats why I rose discussion
> >>>>> about
> >>>>>>>>> future package architecture, so that after agreement I'm be able
> to
> >>>>>> pack
> >>>>>>>>> both RPM and DEB identically.
> >>>>>>>>>
> >>>>>>>>> Yet, if you insist, I can create DEB package according to current
> >> RPM
> >>>>>>>>> layout in no time.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org>
> >>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Peter,
> >>>>>>>>>>
> >>>>>>>>>> I don't think the package size of 280M is going to be a problem
> at
> >>>>>> all,
> >>>>>>>>> but
> >>>>>>>>>> what you are suggesting can be an improvement down the road.
> >>>>>>>>>>
> >>>>>>>>>> In the mean time, I think our top priority should be to provide
> >>>>>> packages
> >>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>>>>>>>
> >>>>>>>>>> Agree?
> >>>>>>>>>>
> >>>>>>>>>> D.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> >>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Hi, Igniters!
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
> >>>>> like
> >>>>>> to
> >>>>>>>>>>> move
> >>>>>>>>>>> forward to further improve and widen AI delivery through
> >> packages.
> >>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> >> 280M+
> >>>>>> and,
> >>>>>>>>> to
> >>>>>>>>>>> improve usability and significantly reduce required download
> >>>>> sizes, I
> >>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
> >>>>>> follows:
> >>>>>>>>>>> - CORE
> >>>>>>>>>>> - bin
> >>>>>>>>>>> - config
> >>>>>>>>>>> - libs (!optional)
> >>>>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>>>> - BENCHMARKS
> >>>>>>>>>>> - DOCS (?)
> >>>>>>>>>>> - EXAMPLES
> >>>>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>>>
> >>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason
> >> to
> >>>>>>>>> download
> >>>>>>>>>>> all 280M+ of binaries where you are to run only core node
> >>>>>> functionality)
> >>>>>>>>>>> and
> >>>>>>>>>>> maintainability (you are in full control of what is installed
> on
> >>>>> your
> >>>>>>>>>>> system).
> >>>>>>>>>>>
> >>>>>>>>>>> After successful architecture choice, same scheme are planned
> to
> >> be
> >>>>>>>>> used in
> >>>>>>>>>>> DEB packages as well.
> >>>>>>>>>>>
> >>>>>>>>>>> WDYT?
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> >> com/
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
> >>
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
No, not yet.
Currently we are discussing RPM packages only.
I want to get all feedback and possible errors working on RPM packages, so that when we have stable agreed architecture and etc. I can recreate it in DEB packages without necessity to fix bugs in both RPM and DEB packages simultaneously.
> On 28 Mar 2018, at 03:17, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> Petr,
>
> I am confused. Do we already have Debian packages?
>
> D.
>
> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
>> Hi, Igniters!
>>
>>
>> Here are some news on our RPM packages initiative.
>>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>> repositories on Apache Bintray. Though they are already prepared for
>> hosting RPM and DEB packages respectively, and there is a way of linking
>> them to apache.org/dist/ignite page, there is possible alternative in
>> storing there only plain directory layout corresponding to each repository
>> type (RPM and DEB) and manage this layout (repodata, distributions,
>> versions, etc.) by ourselves, having more control over repositories but
>> lacking some simplicity of deploying new releases. WDYT? Should we try
>> Cassandra approach? They are storing their DEB packages as I described
>> above [5].
>>
>> Also — a question arose while I was working on this issue: which OSes (and
>> which versions of each) are we going to support (if we are going) in terms
>> of step-by-step list? Currently RPM packages are tested only with latest
>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>> distributives [6] some of which are more o less popular among OS community
>> (ALT, Fedora, openSUSE, etc.).
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>> [2] https://github.com/apache/ignite/pull/3703
>> [3] https://bintray.com/apache/ignite-rpm
>> [4] https://bintray.com/apache/ignite-deb
>> [5] https://bintray.com/apache/cassandra/debian#files/
>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>
>>
>>
>>
>>> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>> I suppose that most everything if not all from libs/options will go to
>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>>> More precise lib selection (if something from optional would better to
>> have in core package) will be discussed right after preliminary split
>> architecture agreement.
>>>
>>>
>>>
>>>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>>>>
>>>> I like idea of keeping simple system of modules, so +1 from me.
>>>>
>>>> Where optional libs (e.g Direct IO plugin) would be included, would it
>> be
>>>> core or optional?
>>>>
>>>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>>>>
>>>>>>
>>>>>>>
>>>>>>> How big would be a final core module?
>>>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
>> own
>>>>>> package.
>>>>>
>>>>>
>>>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>>>>> propose the simplest modular system:
>>>>>
>>>>> - core module that includes basic Ignite capabilities including SQL,
>>>>> compute grid, service grid, k/v
>>>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>>>> flink), kubernetes, etc.
>>>>>
>>>>> What do you think?
>>>>>
>>>>> --
>>>>> Denis
>>>>>
>>>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
>> wrote:
>>>>>
>>>>>> *DEB package
>>>>>>
>>>>>>
>>>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>>
>>>>>>> Considering that DEV package for now is almost platform independent
>>>>> (its
>>>>>> a java application more or less), that package will work almost on any
>>>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>>>>> The only restriction is existence of systemctl (systemd) service
>>>>> manager
>>>>>> — we are dependent on it.
>>>>>>>
>>>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
>> and
>>>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>>>>> Fedora,
>>>>>> etc. with presence of aforementioned systemd.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Will Debian package work for Ubuntu?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Not a problem, rather nuisance. Also, when we will move to official
>>>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>>>
>>>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>>>>> build
>>>>>>>>> (package layout / install scripts) for speeding up things and
>>>>> excluding
>>>>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
>>>>>> and do’
>>>>>>>>> rather then some technical research. Thats why I rose discussion
>>>>> about
>>>>>>>>> future package architecture, so that after agreement I'm be able to
>>>>>> pack
>>>>>>>>> both RPM and DEB identically.
>>>>>>>>>
>>>>>>>>> Yet, if you insist, I can create DEB package according to current
>> RPM
>>>>>>>>> layout in no time.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Peter,
>>>>>>>>>>
>>>>>>>>>> I don't think the package size of 280M is going to be a problem at
>>>>>> all,
>>>>>>>>> but
>>>>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>>>>
>>>>>>>>>> In the mean time, I think our top priority should be to provide
>>>>>> packages
>>>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>>>>
>>>>>>>>>> Agree?
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi, Igniters!
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>>>>> like
>>>>>> to
>>>>>>>>>>> move
>>>>>>>>>>> forward to further improve and widen AI delivery through
>> packages.
>>>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
>> 280M+
>>>>>> and,
>>>>>>>>> to
>>>>>>>>>>> improve usability and significantly reduce required download
>>>>> sizes, I
>>>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>>>>> follows:
>>>>>>>>>>> - CORE
>>>>>>>>>>> - bin
>>>>>>>>>>> - config
>>>>>>>>>>> - libs (!optional)
>>>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>>>> - BENCHMARKS
>>>>>>>>>>> - DOCS (?)
>>>>>>>>>>> - EXAMPLES
>>>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>>>
>>>>>>>>>>> This architecture, as I assume, will add flexibility (no reason
>> to
>>>>>>>>> download
>>>>>>>>>>> all 280M+ of binaries where you are to run only core node
>>>>>> functionality)
>>>>>>>>>>> and
>>>>>>>>>>> maintainability (you are in full control of what is installed on
>>>>> your
>>>>>>>>>>> system).
>>>>>>>>>>>
>>>>>>>>>>> After successful architecture choice, same scheme are planned to
>> be
>>>>>>>>> used in
>>>>>>>>>>> DEB packages as well.
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>> com/
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Petr,
I am confused. Do we already have Debian packages?
D.
On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr...@gmail.com> wrote:
> Hi, Igniters!
>
>
> Here are some news on our RPM packages initiative.
>
> 1. I’ve finished preliminary developing of Stage II version of RPM
> packages [1]. Main “new feature” is — split design. Also I’ve added
> package.sh script for automating package building process which will help
> organise corresponding builds in TC as well as simplify process for
> developers who wishes to have custom packages.
> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
> repositories on Apache Bintray. Though they are already prepared for
> hosting RPM and DEB packages respectively, and there is a way of linking
> them to apache.org/dist/ignite page, there is possible alternative in
> storing there only plain directory layout corresponding to each repository
> type (RPM and DEB) and manage this layout (repodata, distributions,
> versions, etc.) by ourselves, having more control over repositories but
> lacking some simplicity of deploying new releases. WDYT? Should we try
> Cassandra approach? They are storing their DEB packages as I described
> above [5].
>
> Also — a question arose while I was working on this issue: which OSes (and
> which versions of each) are we going to support (if we are going) in terms
> of step-by-step list? Currently RPM packages are tested only with latest
> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
> distributives [6] some of which are more o less popular among OS community
> (ALT, Fedora, openSUSE, etc.).
>
>
> [1] https://issues.apache.org/jira/browse/IGNITE-7647
> [2] https://github.com/apache/ignite/pull/3703
> [3] https://bintray.com/apache/ignite-rpm
> [4] https://bintray.com/apache/ignite-deb
> [5] https://bintray.com/apache/cassandra/debian#files/
> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>
>
>
>
> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
> >
> > I suppose that most everything if not all from libs/options will go to
> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> > More precise lib selection (if something from optional would better to
> have in core package) will be discussed right after preliminary split
> architecture agreement.
> >
> >
> >
> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
> >>
> >> I like idea of keeping simple system of modules, so +1 from me.
> >>
> >> Where optional libs (e.g Direct IO plugin) would be included, would it
> be
> >> core or optional?
> >>
> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >>
> >>>>
> >>>>>
> >>>>> How big would be a final core module?
> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s
> own
> >>>> package.
> >>>
> >>>
> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
> >>> propose the simplest modular system:
> >>>
> >>> - core module that includes basic Ignite capabilities including SQL,
> >>> compute grid, service grid, k/v
> >>> - optional module hosts the rest - ML, streamers integration (kafka,
> >>> flink), kubernetes, etc.
> >>>
> >>> What do you think?
> >>>
> >>> --
> >>> Denis
> >>>
> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
> wrote:
> >>>
> >>>> *DEB package
> >>>>
> >>>>
> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>>
> >>>>> Considering that DEV package for now is almost platform independent
> >>> (its
> >>>> a java application more or less), that package will work almost on any
> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> >>>>> The only restriction is existence of systemctl (systemd) service
> >>> manager
> >>>> — we are dependent on it.
> >>>>>
> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
> and
> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
> >>> Fedora,
> >>>> etc. with presence of aforementioned systemd.
> >>>>>
> >>>>>
> >>>>>
> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
> >>>> wrote:
> >>>>>>
> >>>>>> Will Debian package work for Ubuntu?
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
> >>>> wrote:
> >>>>>>
> >>>>>>> Not a problem, rather nuisance. Also, when we will move to official
> >>>>>>> repositories, there can be a problem from OS community.
> >>>>>>>
> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
> >>>> build
> >>>>>>> (package layout / install scripts) for speeding up things and
> >>> excluding
> >>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
> >>>> and do’
> >>>>>>> rather then some technical research. Thats why I rose discussion
> >>> about
> >>>>>>> future package architecture, so that after agreement I'm be able to
> >>>> pack
> >>>>>>> both RPM and DEB identically.
> >>>>>>>
> >>>>>>> Yet, if you insist, I can create DEB package according to current
> RPM
> >>>>>>> layout in no time.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> Peter,
> >>>>>>>>
> >>>>>>>> I don't think the package size of 280M is going to be a problem at
> >>>> all,
> >>>>>>> but
> >>>>>>>> what you are suggesting can be an improvement down the road.
> >>>>>>>>
> >>>>>>>> In the mean time, I think our top priority should be to provide
> >>>> packages
> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>>>>>
> >>>>>>>> Agree?
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> >>> wrote:
> >>>>>>>>
> >>>>>>>>> Hi, Igniters!
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
> >>> like
> >>>> to
> >>>>>>>>> move
> >>>>>>>>> forward to further improve and widen AI delivery through
> packages.
> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
> 280M+
> >>>> and,
> >>>>>>> to
> >>>>>>>>> improve usability and significantly reduce required download
> >>> sizes, I
> >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
> >>>> follows:
> >>>>>>>>> - CORE
> >>>>>>>>> - bin
> >>>>>>>>> - config
> >>>>>>>>> - libs (!optional)
> >>>>>>>>> - OPTIONAL LIBS
> >>>>>>>>> - BENCHMARKS
> >>>>>>>>> - DOCS (?)
> >>>>>>>>> - EXAMPLES
> >>>>>>>>> - .NET PLATFORM FILES
> >>>>>>>>> - C++ PLATFORM FILES
> >>>>>>>>>
> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason
> to
> >>>>>>> download
> >>>>>>>>> all 280M+ of binaries where you are to run only core node
> >>>> functionality)
> >>>>>>>>> and
> >>>>>>>>> maintainability (you are in full control of what is installed on
> >>> your
> >>>>>>>>> system).
> >>>>>>>>>
> >>>>>>>>> After successful architecture choice, same scheme are planned to
> be
> >>>>>>> used in
> >>>>>>>>> DEB packages as well.
> >>>>>>>>>
> >>>>>>>>> WDYT?
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> com/
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Denis Magda <dm...@apache.org>.
>
> We can either
> store files as is (as Cassandra does, see link below) in something
> that is called Generic Repository — this way we manage directory layouts
> ourselves and have more control over what is published and how
> or
> store files in prepared RPM / DEB repositories where we only upload
> packages and everything else repository does itself — this way we rely on
> Bintray’s implementations of corresponding repositories and have lower
> control over directory layout, but the overall process seems to be simpler.
Petr, he I would lean towards your experience and opinion. Personally, I
can't see any significant advantages or disadvantages of either approach.
This suggests me that we should go for the simplest (2nd) way.
--
Denis
On Tue, Mar 27, 2018 at 11:59 PM, Petr Ivanov <mr...@gmail.com> wrote:
>
>
> On 28 Mar 2018, at 00:18, Denis Magda <dm...@gridgain.com> wrote:
>
> Petr, thanks for the update.
>
>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>
>
> *Anton*, would you be able to check up Petr's pull-request?
>
>
>> corresponding to each repository type (RPM and DEB) and manage this
>> layout (repodata, distributions, versions, etc.) by ourselves, having more
>> control over repositories but lacking some simplicity of deploying new
>> releases. WDYT? Should we try Cassandra approach? They are storing their
>> DEB packages as I described above [5].
>
>
> Petr, do bintray repos come for free for ASF project? Are there any
> limitations in regards total repository or package size?
>
>
> As I see, ASF has premium free account at Bintray with, I guess, virutally
> limitless posiiblilities (the only I stumbled across is limit to uploading
> files more that 250Mb through UI — upload is possible through API). Yet,
> I’ll ask INFRA specifically about this.
>
>
>
> WDYT? Should we try Cassandra approach? They are storing their DEB
>> packages as I described above [5].
>
>
> I'm not sure I got the question. Could you show examples of both
> approaches?
>
>
> We can either
> store files as is (as Cassandra does, see link below) in something
> that is called Generic Repository — this way we manage directory layouts
> ourselves and have more control over what is published and how
> or
> store files in prepared RPM / DEB repositories where we only upload
> packages and everything else repository does itself — this way we rely on
> Bintray’s implementations of corresponding repositories and have lower
> control over directory layout, but the overall process seems to be simpler.
>
>
>
> Also — a question arose while I was working on this issue: which OSes (and
>> which versions of each) are we going to support (if we are going) in terms
>> of step-by-step list? Currently RPM packages are tested only with latest
>> CentOS (and, respectively — RHEL), but there are a lot more RPM-based
>> distributives [6] some of which are more o less popular among OS community
>> (ALT, Fedora, openSUSE, etc.).
>
>
> I would test RHEL and Ubuntu for sure, of the last versions. As for the
> other distributions, I'd like to hear community opinion.
>
>
> There is minor to none necessity of testing packages on RHEL — CentOS is
> binary clone.
> Ubuntu is out of scope because it uses DEB packages and currently we are
> discussing RPM only.
>
>
>
> --
> Denis
>
>
> On Tue, Mar 27, 2018 at 5:10 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
>> Hi, Igniters!
>>
>>
>> Here are some news on our RPM packages initiative.
>>
>> 1. I’ve finished preliminary developing of Stage II version of RPM
>> packages [1]. Main “new feature” is — split design. Also I’ve added
>> package.sh script for automating package building process which will help
>> organise corresponding builds in TC as well as simplify process for
>> developers who wishes to have custom packages.
>> PR#3703 [2] is ready for review. Denis, in order to catch up with Apache
>> Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
>> 2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4]
>> repositories on Apache Bintray. Though they are already prepared for
>> hosting RPM and DEB packages respectively, and there is a way of linking
>> them to apache.org/dist/ignite page, there is possible alternative in
>> storing there only plain directory layout corresponding to each repository
>> type (RPM and DEB) and manage this layout (repodata, distributions,
>> versions, etc.) by ourselves, having more control over repositories but
>> lacking some simplicity of deploying new releases. WDYT? Should we try
>> Cassandra approach? They are storing their DEB packages as I described
>> above [5].
>>
>> Also — a question arose while I was working on this issue: which OSes
>> (and which versions of each) are we going to support (if we are going) in
>> terms of step-by-step list? Currently RPM packages are tested only with
>> latest CentOS (and, respectively — RHEL), but there are a lot more
>> RPM-based distributives [6] some of which are more o less popular among OS
>> community (ALT, Fedora, openSUSE, etc.).
>>
>>
>> [1] https://issues.apache.org/jira/browse/IGNITE-7647
>> [2] https://github.com/apache/ignite/pull/3703
>> [3] https://bintray.com/apache/ignite-rpm
>> [4] https://bintray.com/apache/ignite-deb
>> [5] https://bintray.com/apache/cassandra/debian#files/
>> [6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
>>
>>
>>
>>
>> > On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>> >
>> > I suppose that most everything if not all from libs/options will go to
>> OPTIONAL (I’d call it simply ‘apache-ignite-libs').
>> > More precise lib selection (if something from optional would better to
>> have in core package) will be discussed right after preliminary split
>> architecture agreement.
>> >
>> >
>> >
>> >> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>> >>
>> >> I like idea of keeping simple system of modules, so +1 from me.
>> >>
>> >> Where optional libs (e.g Direct IO plugin) would be included, would it
>> be
>> >> core or optional?
>> >>
>> >> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>> >>
>> >>>>
>> >>>>>
>> >>>>> How big would be a final core module?
>> >>>> Around 30M. Can be shrinked to ~15M if separate Visor and create
>> it’s own
>> >>>> package.
>> >>>
>> >>>
>> >>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>> >>> propose the simplest modular system:
>> >>>
>> >>> - core module that includes basic Ignite capabilities including SQL,
>> >>> compute grid, service grid, k/v
>> >>> - optional module hosts the rest - ML, streamers integration (kafka,
>> >>> flink), kubernetes, etc.
>> >>>
>> >>> What do you think?
>> >>>
>> >>> --
>> >>> Denis
>> >>>
>> >>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com>
>> wrote:
>> >>>
>> >>>> *DEB package
>> >>>>
>> >>>>
>> >>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>> >>>>>
>> >>>>> Considering that DEV package for now is almost platform independent
>> >>> (its
>> >>>> a java application more or less), that package will work almost on
>> any
>> >>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>> >>>>> The only restriction is existence of systemctl (systemd) service
>> >>> manager
>> >>>> — we are dependent on it.
>> >>>>>
>> >>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’
>> and
>> >>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>> >>> Fedora,
>> >>>> etc. with presence of aforementioned systemd.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <dsetrakyan@apache.org
>> >
>> >>>> wrote:
>> >>>>>>
>> >>>>>> Will Debian package work for Ubuntu?
>> >>>>>>
>> >>>>>> D.
>> >>>>>>
>> >>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>> >>>> wrote:
>> >>>>>>
>> >>>>>>> Not a problem, rather nuisance. Also, when we will move to
>> official
>> >>>>>>> repositories, there can be a problem from OS community.
>> >>>>>>>
>> >>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB
>> package
>> >>>> build
>> >>>>>>> (package layout / install scripts) for speeding up things and
>> >>> excluding
>> >>>>>>> possible duplication and desynchronisation, so its a matter of
>> ’sit
>> >>>> and do’
>> >>>>>>> rather then some technical research. Thats why I rose discussion
>> >>> about
>> >>>>>>> future package architecture, so that after agreement I'm be able
>> to
>> >>>> pack
>> >>>>>>> both RPM and DEB identically.
>> >>>>>>>
>> >>>>>>> Yet, if you insist, I can create DEB package according to current
>> RPM
>> >>>>>>> layout in no time.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <
>> dsetrakyan@apache.org>
>> >>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>> Peter,
>> >>>>>>>>
>> >>>>>>>> I don't think the package size of 280M is going to be a problem
>> at
>> >>>> all,
>> >>>>>>> but
>> >>>>>>>> what you are suggesting can be an improvement down the road.
>> >>>>>>>>
>> >>>>>>>> In the mean time, I think our top priority should be to provide
>> >>>> packages
>> >>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>> >>>>>>>>
>> >>>>>>>> Agree?
>> >>>>>>>>
>> >>>>>>>> D.
>> >>>>>>>>
>> >>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>> >>> wrote:
>> >>>>>>>>
>> >>>>>>>>> Hi, Igniters!
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>> >>> like
>> >>>> to
>> >>>>>>>>> move
>> >>>>>>>>> forward to further improve and widen AI delivery through
>> packages.
>> >>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about
>> 280M+
>> >>>> and,
>> >>>>>>> to
>> >>>>>>>>> improve usability and significantly reduce required download
>> >>> sizes, I
>> >>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>> >>>> follows:
>> >>>>>>>>> - CORE
>> >>>>>>>>> - bin
>> >>>>>>>>> - config
>> >>>>>>>>> - libs (!optional)
>> >>>>>>>>> - OPTIONAL LIBS
>> >>>>>>>>> - BENCHMARKS
>> >>>>>>>>> - DOCS (?)
>> >>>>>>>>> - EXAMPLES
>> >>>>>>>>> - .NET PLATFORM FILES
>> >>>>>>>>> - C++ PLATFORM FILES
>> >>>>>>>>>
>> >>>>>>>>> This architecture, as I assume, will add flexibility (no reason
>> to
>> >>>>>>> download
>> >>>>>>>>> all 280M+ of binaries where you are to run only core node
>> >>>> functionality)
>> >>>>>>>>> and
>> >>>>>>>>> maintainability (you are in full control of what is installed on
>> >>> your
>> >>>>>>>>> system).
>> >>>>>>>>>
>> >>>>>>>>> After successful architecture choice, same scheme are planned
>> to be
>> >>>>>>> used in
>> >>>>>>>>> DEB packages as well.
>> >>>>>>>>>
>> >>>>>>>>> WDYT?
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> Sent from: http://apache-ignite-developer
>> s.2346864.n4.nabble.com/
>> >>>>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>
>> >
>>
>>
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
Hi, Igniters!
Here are some news on our RPM packages initiative.
1. I’ve finished preliminary developing of Stage II version of RPM packages [1]. Main “new feature” is — split design. Also I’ve added package.sh script for automating package building process which will help organise corresponding builds in TC as well as simplify process for developers who wishes to have custom packages.
PR#3703 [2] is ready for review. Denis, in order to catch up with Apache Ignite 2.5 release, I’d greatly appreciate your help in finding reviewer.
2. With the help of ASF INFRA team, we now have RPM [3] and DEB [4] repositories on Apache Bintray. Though they are already prepared for hosting RPM and DEB packages respectively, and there is a way of linking them to apache.org/dist/ignite page, there is possible alternative in storing there only plain directory layout corresponding to each repository type (RPM and DEB) and manage this layout (repodata, distributions, versions, etc.) by ourselves, having more control over repositories but lacking some simplicity of deploying new releases. WDYT? Should we try Cassandra approach? They are storing their DEB packages as I described above [5].
Also — a question arose while I was working on this issue: which OSes (and which versions of each) are we going to support (if we are going) in terms of step-by-step list? Currently RPM packages are tested only with latest CentOS (and, respectively — RHEL), but there are a lot more RPM-based distributives [6] some of which are more o less popular among OS community (ALT, Fedora, openSUSE, etc.).
[1] https://issues.apache.org/jira/browse/IGNITE-7647
[2] https://github.com/apache/ignite/pull/3703
[3] https://bintray.com/apache/ignite-rpm
[4] https://bintray.com/apache/ignite-deb
[5] https://bintray.com/apache/cassandra/debian#files/
[6] https://en.wikipedia.org/wiki/Category:RPM-based_Linux_distributions
> On 15 Mar 2018, at 22:15, Petr Ivanov <mr...@gmail.com> wrote:
>
> I suppose that most everything if not all from libs/options will go to OPTIONAL (I’d call it simply ‘apache-ignite-libs').
> More precise lib selection (if something from optional would better to have in core package) will be discussed right after preliminary split architecture agreement.
>
>
>
>> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>>
>> I like idea of keeping simple system of modules, so +1 from me.
>>
>> Where optional libs (e.g Direct IO plugin) would be included, would it be
>> core or optional?
>>
>> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>>
>>>>
>>>>>
>>>>> How big would be a final core module?
>>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
>>>> package.
>>>
>>>
>>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>>> propose the simplest modular system:
>>>
>>> - core module that includes basic Ignite capabilities including SQL,
>>> compute grid, service grid, k/v
>>> - optional module hosts the rest - ML, streamers integration (kafka,
>>> flink), kubernetes, etc.
>>>
>>> What do you think?
>>>
>>> --
>>> Denis
>>>
>>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>>> *DEB package
>>>>
>>>>
>>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>
>>>>> Considering that DEV package for now is almost platform independent
>>> (its
>>>> a java application more or less), that package will work almost on any
>>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>>> The only restriction is existence of systemctl (systemd) service
>>> manager
>>>> — we are dependent on it.
>>>>>
>>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ and
>>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>>> Fedora,
>>>> etc. with presence of aforementioned systemd.
>>>>>
>>>>>
>>>>>
>>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
>>>> wrote:
>>>>>>
>>>>>> Will Debian package work for Ubuntu?
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>>> Not a problem, rather nuisance. Also, when we will move to official
>>>>>>> repositories, there can be a problem from OS community.
>>>>>>>
>>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>>> build
>>>>>>> (package layout / install scripts) for speeding up things and
>>> excluding
>>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
>>>> and do’
>>>>>>> rather then some technical research. Thats why I rose discussion
>>> about
>>>>>>> future package architecture, so that after agreement I'm be able to
>>>> pack
>>>>>>> both RPM and DEB identically.
>>>>>>>
>>>>>>> Yet, if you insist, I can create DEB package according to current RPM
>>>>>>> layout in no time.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Peter,
>>>>>>>>
>>>>>>>> I don't think the package size of 280M is going to be a problem at
>>>> all,
>>>>>>> but
>>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>>
>>>>>>>> In the mean time, I think our top priority should be to provide
>>>> packages
>>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>>
>>>>>>>> Agree?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>>> wrote:
>>>>>>>>
>>>>>>>>> Hi, Igniters!
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>>> like
>>>> to
>>>>>>>>> move
>>>>>>>>> forward to further improve and widen AI delivery through packages.
>>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+
>>>> and,
>>>>>>> to
>>>>>>>>> improve usability and significantly reduce required download
>>> sizes, I
>>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>>> follows:
>>>>>>>>> - CORE
>>>>>>>>> - bin
>>>>>>>>> - config
>>>>>>>>> - libs (!optional)
>>>>>>>>> - OPTIONAL LIBS
>>>>>>>>> - BENCHMARKS
>>>>>>>>> - DOCS (?)
>>>>>>>>> - EXAMPLES
>>>>>>>>> - .NET PLATFORM FILES
>>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>>
>>>>>>>>> This architecture, as I assume, will add flexibility (no reason to
>>>>>>> download
>>>>>>>>> all 280M+ of binaries where you are to run only core node
>>>> functionality)
>>>>>>>>> and
>>>>>>>>> maintainability (you are in full control of what is installed on
>>> your
>>>>>>>>> system).
>>>>>>>>>
>>>>>>>>> After successful architecture choice, same scheme are planned to be
>>>>>>> used in
>>>>>>>>> DEB packages as well.
>>>>>>>>>
>>>>>>>>> WDYT?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>>
>>>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
I suppose that most everything if not all from libs/options will go to OPTIONAL (I’d call it simply ‘apache-ignite-libs').
More precise lib selection (if something from optional would better to have in core package) will be discussed right after preliminary split architecture agreement.
> On 15 Mar 2018, at 22:11, Dmitry Pavlov <dp...@gmail.com> wrote:
>
> I like idea of keeping simple system of modules, so +1 from me.
>
> Where optional libs (e.g Direct IO plugin) would be included, would it be
> core or optional?
>
> чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
>
>>>
>>>>
>>>> How big would be a final core module?
>>> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
>>> package.
>>
>>
>> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
>> propose the simplest modular system:
>>
>> - core module that includes basic Ignite capabilities including SQL,
>> compute grid, service grid, k/v
>> - optional module hosts the rest - ML, streamers integration (kafka,
>> flink), kubernetes, etc.
>>
>> What do you think?
>>
>> --
>> Denis
>>
>> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com> wrote:
>>
>>> *DEB package
>>>
>>>
>>>> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>
>>>> Considering that DEV package for now is almost platform independent
>> (its
>>> a java application more or less), that package will work almost on any
>>> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
>>>> The only restriction is existence of systemctl (systemd) service
>> manager
>>> — we are dependent on it.
>>>>
>>>> Thats why, for instance, our RPM repository is called simply ‘rpm’ and
>>> package has no arch or dist suffix — it will work on CentOS, RHEL,
>> Fedora,
>>> etc. with presence of aforementioned systemd.
>>>>
>>>>
>>>>
>>>>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
>>> wrote:
>>>>>
>>>>> Will Debian package work for Ubuntu?
>>>>>
>>>>> D.
>>>>>
>>>>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
>>> wrote:
>>>>>
>>>>>> Not a problem, rather nuisance. Also, when we will move to official
>>>>>> repositories, there can be a problem from OS community.
>>>>>>
>>>>>> Concerning DEB packages — I plan to use RPM as base for DEB package
>>> build
>>>>>> (package layout / install scripts) for speeding up things and
>> excluding
>>>>>> possible duplication and desynchronisation, so its a matter of ’sit
>>> and do’
>>>>>> rather then some technical research. Thats why I rose discussion
>> about
>>>>>> future package architecture, so that after agreement I'm be able to
>>> pack
>>>>>> both RPM and DEB identically.
>>>>>>
>>>>>> Yet, if you insist, I can create DEB package according to current RPM
>>>>>> layout in no time.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> Peter,
>>>>>>>
>>>>>>> I don't think the package size of 280M is going to be a problem at
>>> all,
>>>>>> but
>>>>>>> what you are suggesting can be an improvement down the road.
>>>>>>>
>>>>>>> In the mean time, I think our top priority should be to provide
>>> packages
>>>>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>>>>
>>>>>>> Agree?
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
>> wrote:
>>>>>>>
>>>>>>>> Hi, Igniters!
>>>>>>>>
>>>>>>>>
>>>>>>>> Release 2.4 is almost there, at least binary part of it, so I'd
>> like
>>> to
>>>>>>>> move
>>>>>>>> forward to further improve and widen AI delivery through packages.
>>>>>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+
>>> and,
>>>>>> to
>>>>>>>> improve usability and significantly reduce required download
>> sizes, I
>>>>>>>> purpose that in 2.5 release we introduce splitted delivery as
>>> follows:
>>>>>>>> - CORE
>>>>>>>> - bin
>>>>>>>> - config
>>>>>>>> - libs (!optional)
>>>>>>>> - OPTIONAL LIBS
>>>>>>>> - BENCHMARKS
>>>>>>>> - DOCS (?)
>>>>>>>> - EXAMPLES
>>>>>>>> - .NET PLATFORM FILES
>>>>>>>> - C++ PLATFORM FILES
>>>>>>>>
>>>>>>>> This architecture, as I assume, will add flexibility (no reason to
>>>>>> download
>>>>>>>> all 280M+ of binaries where you are to run only core node
>>> functionality)
>>>>>>>> and
>>>>>>>> maintainability (you are in full control of what is installed on
>> your
>>>>>>>> system).
>>>>>>>>
>>>>>>>> After successful architecture choice, same scheme are planned to be
>>>>>> used in
>>>>>>>> DEB packages as well.
>>>>>>>>
>>>>>>>> WDYT?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>
>>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitry Pavlov <dp...@gmail.com>.
I like idea of keeping simple system of modules, so +1 from me.
Where optional libs (e.g Direct IO plugin) would be included, would it be
core or optional?
чт, 15 мар. 2018 г. в 22:09, Denis Magda <dm...@apache.org>:
> >
> > >
> > > How big would be a final core module?
> > Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
> > package.
>
>
> Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
> propose the simplest modular system:
>
> - core module that includes basic Ignite capabilities including SQL,
> compute grid, service grid, k/v
> - optional module hosts the rest - ML, streamers integration (kafka,
> flink), kubernetes, etc.
>
> What do you think?
>
> --
> Denis
>
> On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
> > *DEB package
> >
> >
> > > On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
> > >
> > > Considering that DEV package for now is almost platform independent
> (its
> > a java application more or less), that package will work almost on any
> > DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> > > The only restriction is existence of systemctl (systemd) service
> manager
> > — we are dependent on it.
> > >
> > > Thats why, for instance, our RPM repository is called simply ‘rpm’ and
> > package has no arch or dist suffix — it will work on CentOS, RHEL,
> Fedora,
> > etc. with presence of aforementioned systemd.
> > >
> > >
> > >
> > >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
> > wrote:
> > >>
> > >> Will Debian package work for Ubuntu?
> > >>
> > >> D.
> > >>
> > >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
> > wrote:
> > >>
> > >>> Not a problem, rather nuisance. Also, when we will move to official
> > >>> repositories, there can be a problem from OS community.
> > >>>
> > >>> Concerning DEB packages — I plan to use RPM as base for DEB package
> > build
> > >>> (package layout / install scripts) for speeding up things and
> excluding
> > >>> possible duplication and desynchronisation, so its a matter of ’sit
> > and do’
> > >>> rather then some technical research. Thats why I rose discussion
> about
> > >>> future package architecture, so that after agreement I'm be able to
> > pack
> > >>> both RPM and DEB identically.
> > >>>
> > >>> Yet, if you insist, I can create DEB package according to current RPM
> > >>> layout in no time.
> > >>>
> > >>>
> > >>>
> > >>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
> > >>> wrote:
> > >>>>
> > >>>> Peter,
> > >>>>
> > >>>> I don't think the package size of 280M is going to be a problem at
> > all,
> > >>> but
> > >>>> what you are suggesting can be an improvement down the road.
> > >>>>
> > >>>> In the mean time, I think our top priority should be to provide
> > packages
> > >>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> > >>>>
> > >>>> Agree?
> > >>>>
> > >>>> D.
> > >>>>
> > >>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com>
> wrote:
> > >>>>
> > >>>>> Hi, Igniters!
> > >>>>>
> > >>>>>
> > >>>>> Release 2.4 is almost there, at least binary part of it, so I'd
> like
> > to
> > >>>>> move
> > >>>>> forward to further improve and widen AI delivery through packages.
> > >>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+
> > and,
> > >>> to
> > >>>>> improve usability and significantly reduce required download
> sizes, I
> > >>>>> purpose that in 2.5 release we introduce splitted delivery as
> > follows:
> > >>>>> - CORE
> > >>>>> - bin
> > >>>>> - config
> > >>>>> - libs (!optional)
> > >>>>> - OPTIONAL LIBS
> > >>>>> - BENCHMARKS
> > >>>>> - DOCS (?)
> > >>>>> - EXAMPLES
> > >>>>> - .NET PLATFORM FILES
> > >>>>> - C++ PLATFORM FILES
> > >>>>>
> > >>>>> This architecture, as I assume, will add flexibility (no reason to
> > >>> download
> > >>>>> all 280M+ of binaries where you are to run only core node
> > functionality)
> > >>>>> and
> > >>>>> maintainability (you are in full control of what is installed on
> your
> > >>>>> system).
> > >>>>>
> > >>>>> After successful architecture choice, same scheme are planned to be
> > >>> used in
> > >>>>> DEB packages as well.
> > >>>>>
> > >>>>> WDYT?
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> --
> > >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >>>>>
> > >>>
> > >>>
> > >
> >
> >
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Denis Magda <dm...@apache.org>.
>
> >
> > How big would be a final core module?
> Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own
> package.
Guys, 30 vs 280M is a huuuuge difference. I would agree with Petr and
propose the simplest modular system:
- core module that includes basic Ignite capabilities including SQL,
compute grid, service grid, k/v
- optional module hosts the rest - ML, streamers integration (kafka,
flink), kubernetes, etc.
What do you think?
--
Denis
On Thu, Mar 15, 2018 at 12:36 AM, Petr Ivanov <mr...@gmail.com> wrote:
> *DEB package
>
>
> > On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
> >
> > Considering that DEV package for now is almost platform independent (its
> a java application more or less), that package will work almost on any
> DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> > The only restriction is existence of systemctl (systemd) service manager
> — we are dependent on it.
> >
> > Thats why, for instance, our RPM repository is called simply ‘rpm’ and
> package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora,
> etc. with presence of aforementioned systemd.
> >
> >
> >
> >> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >>
> >> Will Debian package work for Ubuntu?
> >>
> >> D.
> >>
> >> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com>
> wrote:
> >>
> >>> Not a problem, rather nuisance. Also, when we will move to official
> >>> repositories, there can be a problem from OS community.
> >>>
> >>> Concerning DEB packages — I plan to use RPM as base for DEB package
> build
> >>> (package layout / install scripts) for speeding up things and excluding
> >>> possible duplication and desynchronisation, so its a matter of ’sit
> and do’
> >>> rather then some technical research. Thats why I rose discussion about
> >>> future package architecture, so that after agreement I'm be able to
> pack
> >>> both RPM and DEB identically.
> >>>
> >>> Yet, if you insist, I can create DEB package according to current RPM
> >>> layout in no time.
> >>>
> >>>
> >>>
> >>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
> >>> wrote:
> >>>>
> >>>> Peter,
> >>>>
> >>>> I don't think the package size of 280M is going to be a problem at
> all,
> >>> but
> >>>> what you are suggesting can be an improvement down the road.
> >>>>
> >>>> In the mean time, I think our top priority should be to provide
> packages
> >>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >>>>
> >>>> Agree?
> >>>>
> >>>> D.
> >>>>
> >>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
> >>>>
> >>>>> Hi, Igniters!
> >>>>>
> >>>>>
> >>>>> Release 2.4 is almost there, at least binary part of it, so I'd like
> to
> >>>>> move
> >>>>> forward to further improve and widen AI delivery through packages.
> >>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+
> and,
> >>> to
> >>>>> improve usability and significantly reduce required download sizes, I
> >>>>> purpose that in 2.5 release we introduce splitted delivery as
> follows:
> >>>>> - CORE
> >>>>> - bin
> >>>>> - config
> >>>>> - libs (!optional)
> >>>>> - OPTIONAL LIBS
> >>>>> - BENCHMARKS
> >>>>> - DOCS (?)
> >>>>> - EXAMPLES
> >>>>> - .NET PLATFORM FILES
> >>>>> - C++ PLATFORM FILES
> >>>>>
> >>>>> This architecture, as I assume, will add flexibility (no reason to
> >>> download
> >>>>> all 280M+ of binaries where you are to run only core node
> functionality)
> >>>>> and
> >>>>> maintainability (you are in full control of what is installed on your
> >>>>> system).
> >>>>>
> >>>>> After successful architecture choice, same scheme are planned to be
> >>> used in
> >>>>> DEB packages as well.
> >>>>>
> >>>>> WDYT?
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>>>>
> >>>
> >>>
> >
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
*DEB package
> On 15 Mar 2018, at 10:35, Petr Ivanov <mr...@gmail.com> wrote:
>
> Considering that DEV package for now is almost platform independent (its a java application more or less), that package will work almost on any DEB-based linux, including but not limited to Ubuntu, Debian, etc.
> The only restriction is existence of systemctl (systemd) service manager — we are dependent on it.
>
> Thats why, for instance, our RPM repository is called simply ‘rpm’ and package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, etc. with presence of aforementioned systemd.
>
>
>
>> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org> wrote:
>>
>> Will Debian package work for Ubuntu?
>>
>> D.
>>
>> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>
>>> Not a problem, rather nuisance. Also, when we will move to official
>>> repositories, there can be a problem from OS community.
>>>
>>> Concerning DEB packages — I plan to use RPM as base for DEB package build
>>> (package layout / install scripts) for speeding up things and excluding
>>> possible duplication and desynchronisation, so its a matter of ’sit and do’
>>> rather then some technical research. Thats why I rose discussion about
>>> future package architecture, so that after agreement I'm be able to pack
>>> both RPM and DEB identically.
>>>
>>> Yet, if you insist, I can create DEB package according to current RPM
>>> layout in no time.
>>>
>>>
>>>
>>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
>>> wrote:
>>>>
>>>> Peter,
>>>>
>>>> I don't think the package size of 280M is going to be a problem at all,
>>> but
>>>> what you are suggesting can be an improvement down the road.
>>>>
>>>> In the mean time, I think our top priority should be to provide packages
>>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>>
>>>> Agree?
>>>>
>>>> D.
>>>>
>>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
>>>>
>>>>> Hi, Igniters!
>>>>>
>>>>>
>>>>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>>>>> move
>>>>> forward to further improve and widen AI delivery through packages.
>>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
>>> to
>>>>> improve usability and significantly reduce required download sizes, I
>>>>> purpose that in 2.5 release we introduce splitted delivery as follows:
>>>>> - CORE
>>>>> - bin
>>>>> - config
>>>>> - libs (!optional)
>>>>> - OPTIONAL LIBS
>>>>> - BENCHMARKS
>>>>> - DOCS (?)
>>>>> - EXAMPLES
>>>>> - .NET PLATFORM FILES
>>>>> - C++ PLATFORM FILES
>>>>>
>>>>> This architecture, as I assume, will add flexibility (no reason to
>>> download
>>>>> all 280M+ of binaries where you are to run only core node functionality)
>>>>> and
>>>>> maintainability (you are in full control of what is installed on your
>>>>> system).
>>>>>
>>>>> After successful architecture choice, same scheme are planned to be
>>> used in
>>>>> DEB packages as well.
>>>>>
>>>>> WDYT?
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>
>>>
>>>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
Considering that DEV package for now is almost platform independent (its a java application more or less), that package will work almost on any DEB-based linux, including but not limited to Ubuntu, Debian, etc.
The only restriction is existence of systemctl (systemd) service manager — we are dependent on it.
Thats why, for instance, our RPM repository is called simply ‘rpm’ and package has no arch or dist suffix — it will work on CentOS, RHEL, Fedora, etc. with presence of aforementioned systemd.
> On 15 Mar 2018, at 07:57, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> Will Debian package work for Ubuntu?
>
> D.
>
> On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com> wrote:
>
>> Not a problem, rather nuisance. Also, when we will move to official
>> repositories, there can be a problem from OS community.
>>
>> Concerning DEB packages — I plan to use RPM as base for DEB package build
>> (package layout / install scripts) for speeding up things and excluding
>> possible duplication and desynchronisation, so its a matter of ’sit and do’
>> rather then some technical research. Thats why I rose discussion about
>> future package architecture, so that after agreement I'm be able to pack
>> both RPM and DEB identically.
>>
>> Yet, if you insist, I can create DEB package according to current RPM
>> layout in no time.
>>
>>
>>
>>> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>>
>>> Peter,
>>>
>>> I don't think the package size of 280M is going to be a problem at all,
>> but
>>> what you are suggesting can be an improvement down the road.
>>>
>>> In the mean time, I think our top priority should be to provide packages
>>> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>>>
>>> Agree?
>>>
>>> D.
>>>
>>> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
>>>
>>>> Hi, Igniters!
>>>>
>>>>
>>>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>>>> move
>>>> forward to further improve and widen AI delivery through packages.
>>>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
>> to
>>>> improve usability and significantly reduce required download sizes, I
>>>> purpose that in 2.5 release we introduce splitted delivery as follows:
>>>> - CORE
>>>> - bin
>>>> - config
>>>> - libs (!optional)
>>>> - OPTIONAL LIBS
>>>> - BENCHMARKS
>>>> - DOCS (?)
>>>> - EXAMPLES
>>>> - .NET PLATFORM FILES
>>>> - C++ PLATFORM FILES
>>>>
>>>> This architecture, as I assume, will add flexibility (no reason to
>> download
>>>> all 280M+ of binaries where you are to run only core node functionality)
>>>> and
>>>> maintainability (you are in full control of what is installed on your
>>>> system).
>>>>
>>>> After successful architecture choice, same scheme are planned to be
>> used in
>>>> DEB packages as well.
>>>>
>>>> WDYT?
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>
>>
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Will Debian package work for Ubuntu?
D.
On Wed, Mar 14, 2018 at 9:52 PM, Petr Ivanov <mr...@gmail.com> wrote:
> Not a problem, rather nuisance. Also, when we will move to official
> repositories, there can be a problem from OS community.
>
> Concerning DEB packages — I plan to use RPM as base for DEB package build
> (package layout / install scripts) for speeding up things and excluding
> possible duplication and desynchronisation, so its a matter of ’sit and do’
> rather then some technical research. Thats why I rose discussion about
> future package architecture, so that after agreement I'm be able to pack
> both RPM and DEB identically.
>
> Yet, if you insist, I can create DEB package according to current RPM
> layout in no time.
>
>
>
> > On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > Peter,
> >
> > I don't think the package size of 280M is going to be a problem at all,
> but
> > what you are suggesting can be an improvement down the road.
> >
> > In the mean time, I think our top priority should be to provide packages
> > for Debian and Ubuntu. Having only RPMs is not nearly enough.
> >
> > Agree?
> >
> > D.
> >
> > On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
> >
> >> Hi, Igniters!
> >>
> >>
> >> Release 2.4 is almost there, at least binary part of it, so I'd like to
> >> move
> >> forward to further improve and widen AI delivery through packages.
> >> As of now, Apache Ignite ships in RPM package weighing about 280M+ and,
> to
> >> improve usability and significantly reduce required download sizes, I
> >> purpose that in 2.5 release we introduce splitted delivery as follows:
> >> - CORE
> >> - bin
> >> - config
> >> - libs (!optional)
> >> - OPTIONAL LIBS
> >> - BENCHMARKS
> >> - DOCS (?)
> >> - EXAMPLES
> >> - .NET PLATFORM FILES
> >> - C++ PLATFORM FILES
> >>
> >> This architecture, as I assume, will add flexibility (no reason to
> download
> >> all 280M+ of binaries where you are to run only core node functionality)
> >> and
> >> maintainability (you are in full control of what is installed on your
> >> system).
> >>
> >> After successful architecture choice, same scheme are planned to be
> used in
> >> DEB packages as well.
> >>
> >> WDYT?
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
Not a problem, rather nuisance. Also, when we will move to official repositories, there can be a problem from OS community.
Concerning DEB packages — I plan to use RPM as base for DEB package build (package layout / install scripts) for speeding up things and excluding possible duplication and desynchronisation, so its a matter of ’sit and do’ rather then some technical research. Thats why I rose discussion about future package architecture, so that after agreement I'm be able to pack both RPM and DEB identically.
Yet, if you insist, I can create DEB package according to current RPM layout in no time.
> On 15 Mar 2018, at 04:53, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> Peter,
>
> I don't think the package size of 280M is going to be a problem at all, but
> what you are suggesting can be an improvement down the road.
>
> In the mean time, I think our top priority should be to provide packages
> for Debian and Ubuntu. Having only RPMs is not nearly enough.
>
> Agree?
>
> D.
>
> On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
>
>> Hi, Igniters!
>>
>>
>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>> move
>> forward to further improve and widen AI delivery through packages.
>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
>> improve usability and significantly reduce required download sizes, I
>> purpose that in 2.5 release we introduce splitted delivery as follows:
>> - CORE
>> - bin
>> - config
>> - libs (!optional)
>> - OPTIONAL LIBS
>> - BENCHMARKS
>> - DOCS (?)
>> - EXAMPLES
>> - .NET PLATFORM FILES
>> - C++ PLATFORM FILES
>>
>> This architecture, as I assume, will add flexibility (no reason to download
>> all 280M+ of binaries where you are to run only core node functionality)
>> and
>> maintainability (you are in full control of what is installed on your
>> system).
>>
>> After successful architecture choice, same scheme are planned to be used in
>> DEB packages as well.
>>
>> WDYT?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Peter,
I don't think the package size of 280M is going to be a problem at all, but
what you are suggesting can be an improvement down the road.
In the mean time, I think our top priority should be to provide packages
for Debian and Ubuntu. Having only RPMs is not nearly enough.
Agree?
D.
On Wed, Mar 14, 2018 at 5:36 AM, vveider <mr...@gmail.com> wrote:
> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
> - CORE
> - bin
> - config
> - libs (!optional)
> - OPTIONAL LIBS
> - BENCHMARKS
> - DOCS (?)
> - EXAMPLES
> - .NET PLATFORM FILES
> - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
> On 14 Mar 2018, at 22:12, Denis Magda <dm...@apache.org> wrote:
>
> Petr,
>
> How big would be a final core module?
Around 30M. Can be shrinked to ~15M if separate Visor and create it’s own package.
> As for the optional libs do you
> suggest installing them with a single command or each module can be
> installed separately?
Eventually — separately. If user wishes to try some new integration or lib, there is no need to download all of them.
Also it will be clearer what new integrations / libs we add (if we add) each release.
>
> --
> Denis
>
> On Wed, Mar 14, 2018 at 2:36 AM, vveider <mr...@gmail.com> wrote:
>
>> Hi, Igniters!
>>
>>
>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>> move
>> forward to further improve and widen AI delivery through packages.
>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
>> improve usability and significantly reduce required download sizes, I
>> purpose that in 2.5 release we introduce splitted delivery as follows:
>> - CORE
>> - bin
>> - config
>> - libs (!optional)
>> - OPTIONAL LIBS
>> - BENCHMARKS
>> - DOCS (?)
>> - EXAMPLES
>> - .NET PLATFORM FILES
>> - C++ PLATFORM FILES
>>
>> This architecture, as I assume, will add flexibility (no reason to download
>> all 280M+ of binaries where you are to run only core node functionality)
>> and
>> maintainability (you are in full control of what is installed on your
>> system).
>>
>> After successful architecture choice, same scheme are planned to be used in
>> DEB packages as well.
>>
>> WDYT?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Denis Magda <dm...@apache.org>.
Petr,
How big would be a final core module? As for the optional libs do you
suggest installing them with a single command or each module can be
installed separately?
--
Denis
On Wed, Mar 14, 2018 at 2:36 AM, vveider <mr...@gmail.com> wrote:
> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
> - CORE
> - bin
> - config
> - libs (!optional)
> - OPTIONAL LIBS
> - BENCHMARKS
> - DOCS (?)
> - EXAMPLES
> - .NET PLATFORM FILES
> - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Petr Ivanov <mr...@gmail.com>.
> On 14 Mar 2018, at 12:46, Dmitry Pavlov <dp...@gmail.com> wrote:
>
> Hi,
>
> Did not understand fully that splitted delivery means.
> Is it correct,that user will have install
> install ignite:core
> install ignite:opt_libs
> with separated commands?
It will be one command with several arguments, i.e.
yum install ignite-core ignite-optional
Also, I considered splitting optional libs per package (like php / perl / etc. do, for example) but not sure whether these efforts are appropriate at this moment.
I guess this scheme must have huge support from the community.
>
> Or there will be some aggregating package like ignite:full and several
> options like ignite:core.
Nice idea! If community will agree, I can add aggregating virtual package with dependencies on all other for corresponding release. It can even keep current name ‘apache-ignite’ :)
>
> Sincerely,
> Dmitriy Pavlov
>
> ср, 14 мар. 2018 г. в 12:36, vveider <mr...@gmail.com>:
>
>> Hi, Igniters!
>>
>>
>> Release 2.4 is almost there, at least binary part of it, so I'd like to
>> move
>> forward to further improve and widen AI delivery through packages.
>> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
>> improve usability and significantly reduce required download sizes, I
>> purpose that in 2.5 release we introduce splitted delivery as follows:
>> - CORE
>> - bin
>> - config
>> - libs (!optional)
>> - OPTIONAL LIBS
>> - BENCHMARKS
>> - DOCS (?)
>> - EXAMPLES
>> - .NET PLATFORM FILES
>> - C++ PLATFORM FILES
>>
>> This architecture, as I assume, will add flexibility (no reason to download
>> all 280M+ of binaries where you are to run only core node functionality)
>> and
>> maintainability (you are in full control of what is installed on your
>> system).
>>
>> After successful architecture choice, same scheme are planned to be used in
>> DEB packages as well.
>>
>> WDYT?
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
Re: Apache Ignite RPM packaging (Stage II)
Posted by Dmitry Pavlov <dp...@gmail.com>.
Hi,
Did not understand fully that splitted delivery means.
Is it correct,that user will have install
install ignite:core
install ignite:opt_libs
with separated commands?
Or there will be some aggregating package like ignite:full and several
options like ignite:core.
Sincerely,
Dmitriy Pavlov
ср, 14 мар. 2018 г. в 12:36, vveider <mr...@gmail.com>:
> Hi, Igniters!
>
>
> Release 2.4 is almost there, at least binary part of it, so I'd like to
> move
> forward to further improve and widen AI delivery through packages.
> As of now, Apache Ignite ships in RPM package weighing about 280M+ and, to
> improve usability and significantly reduce required download sizes, I
> purpose that in 2.5 release we introduce splitted delivery as follows:
> - CORE
> - bin
> - config
> - libs (!optional)
> - OPTIONAL LIBS
> - BENCHMARKS
> - DOCS (?)
> - EXAMPLES
> - .NET PLATFORM FILES
> - C++ PLATFORM FILES
>
> This architecture, as I assume, will add flexibility (no reason to download
> all 280M+ of binaries where you are to run only core node functionality)
> and
> maintainability (you are in full control of what is installed on your
> system).
>
> After successful architecture choice, same scheme are planned to be used in
> DEB packages as well.
>
> WDYT?
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>