You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Petr Ivanov <mr...@gmail.com> on 2017/12/07 15:02:59 UTC
IGNITE-7107 Apache Ignite RPM packages
HI, all!
Within the task IGNITE-7107 I’ve prepare pull request #3171 [1] with:
- specification to build RPM package out of built binary archive;
- corresponding instruction in DEVNOTES.txt
Also took responsibility to introduced style correction and unification to DEVNOTES.txt
[1] - https://github.com/apache/ignite/pull/3171
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Denis Magda <dm...@apache.org>.
Just as a follow up for the community.
We had a private conversation with Peter and decided that RPMs will be initially hosted in Ignite root. Going forward (Ignite 2.5) we might to store RPMs and DEBs on JFrog Bintray [1] (this is what Cassandra does for its DEB packages [2]).
So, Peter please keep us posted on immediate results and your investigation results on how to proceed with RPMs and DEBs after we release 2.4.
[1] https://bintray.com/
[2] http://apache.org/dist/cassandra/debian/ <http://apache.org/dist/cassandra/debian/>
—
Denis
> On Jan 17, 2018, at 10:58 PM, Petr Ivanov <mr...@gmail.com> wrote:
>
> Hi, Denis.
>
>
> Proposed layout will require changes to release procedure.
> Also, I’d like to add to this layout symlink to latest repository, which we will update every release.
>
> But my concern will be — are we allowed to add and store about half a gigabyte of artifacts every 3 months?
>
>
>
>> On 18 Jan 2018, at 04:16, Denis Magda <dm...@apache.org> wrote:
>>
>> Hi Petr,
>>
>> I would go for the approach implemented for Cassandra. Specifically:
>>
>> Cassandra root repository includes all the most recent versions and packages for RPM and DEB:
>> http://www.apache.org/dist/cassandra/
>>
>> This is a subdirectory for RPMs there:
>> http://www.apache.org/dist/cassandra/redhat/ <http://www.apache.org/dist/cassandra/redhat/>
>>
>> So, technically it’s possible to have more than one version in the root and not worrying about that something is moved to the archives. It suggests to me that we have only one directory with the latest version in Ignite root because we took this path.
>>
>> I propose to lay out Ignite root this way:
>>
>> - latest Ignite version directory
>> - rpm
>> — 2.4
>> — ignite-2.1..rpm
>> — etc
>> — 2.5
>> ---
>> - deb
>> — 2.5
>> — ignite-2.1..deb
>> — etc
>> — 2.6
>> ---
>>
>> Ignite latest version directory is changed over the time while “rpm” and “deb” are never go to archive. We might archive specific RPM/DEB versions over the time, but presently it’s not an issue.
>>
>> —
>> Denis
>>
>>> On Jan 17, 2018, at 4:18 AM, Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>> Hi, Igniters!
>>>
>>>
>>> As part of "Apache Ignite in Packages" initiative, I’ve introduced update to Release procedure, which consists of:
>>> - RPM-build instruction [1].
>>> - new "vote_3_step_1[rpm]create_rpm_repository.sh” script which does everything else to prepare RPM-repository layout along with source and binary archives.
>>> So 2.4 will be uploaded with RPM-repository structure and will be eligible for use as standard third-party repository.
>>>
>>> Yet, I have concerns regarding this method of packages hosting.
>>> Current package-placing architecture assumes, that repository will be available as something like this [2]. But after next major release (2.5 for example) that URL will become unavailable due to archive procedure — users will have to update repository URL for continuous access to that version.
>>> If we are to place repository layout to the root [3], then after next major release (2.5 for example) package of 2.4 version will silently become unavailable for installing. However if we manage somehow to have single repository layout with multiple packages versions in Apache Archive, users with plugged in main and archive repositories will get access to all new and archive version of Apache Ignite transparently. Unfortunately — I do not know how this can be achieved.
>>>
>>> There is also one last approach for package (and artifacts in common) keeping — we can have some kind of third-party mirror, where will be able to host all artifacts, binaries, packages, repositories and other Apache Ignite related stuff with full control and access according to current developer role (reducing impact on Apache’s storages and uploading there only sources).
>>>
>>>
>>> Please, share your thoughts on subject.
>>>
>>>
>>> [1] https://cwiki.apache.org/confluence/display/IGNITE/RPM-packages
>>> [2] http://apache.org/dist/ignite/2.4.0/rpm
>>> [3] http://apache.org/dist/ignite
>>>
>>>
>>>> On 29 Dec 2017, at 07:54, Petr Ivanov <mr...@gmail.com> wrote:
>>>>
>>>> Hi, Denis.
>>>>
>>>>
>>>> I was glad to contribute.
>>>>
>>>> Concerning DEB package — for 2.4 release I’m planning to introduce instructions and spec files for conversion RPM package to DEB, substantially reducing efforts for now because of file structure and install scripts inheritance.
>>>> Later, of course, I’m going to prepare fully synchronised source-based package creation process, packages from which are meant for inclusion into official repositories.
>>>>
>>>>
>>>>
>>>>> On 28 Dec 2017, at 22:53, Denis Magda <dm...@apache.org> wrote:
>>>>>
>>>>> Peter, thanks for this Christmas/New Year gift for the community! :)
>>>>>
>>>>> I’ll be back soon putting together a documentation for this capability.
>>>>>
>>>>> BTW, what’s the plan in regards DEB packages?
>>>>> https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
>>>>>
>>>>> Do you consider fitting them in AI 2.4 release?
>>>>>
>>>>> —
>>>>> Denis
>>>>>
>>>>>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>>>>>
>>>>>> Guys
>>>>>>
>>>>>> Thanks for your efforts to make it available in 2.4!
>>>>>>
>>>>>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>>>>>>
>>>>>>> Guys,
>>>>>>>
>>>>>>> I've merged change to master branch. Thanks a lot!
>>>>>>>
>>>>>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>>> Fixed remarks and nitpicks.
>>>>>>>>
>>>>>>>> Also, I purpose to delay service autostart implementation until we’ll
>>>>>>> get some first-hand usability feedback.
>>>>>>>> Added notes to DEVNOTES.txt about how to start service.
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hello once more!
>>>>>>>>>
>>>>>>>>> Two more nitpicks:
>>>>>>>>>
>>>>>>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>>>>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>>>>>>> examples for that.
>>>>>>>>> Please remove that alternative for now, create a ticket.
>>>>>>>>>
>>>>>>>>> rpm -ql apache-ignite | grep \\.java
>>>>>>>>> some yardstick and ml sources are there. I don't know who is there to
>>>>>>> blame
>>>>>>>>> - Ignite release process likely, not RPM building - but they shouldn't
>>>>>>> be
>>>>>>>>> here I guess?
>>>>>>>>>
>>>>>>>>> Regards!
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>
>>>>>>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hello again!
>>>>>>>>>>
>>>>>>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>>>>>>> were fixed. I have just two minor remarks:
>>>>>>>>>>
>>>>>>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>>>>>>> works just as well from there as it was from our distribution. visor
>>>>>>>>>> doesn't, hence it stays in examples.
>>>>>>>>>> - I'm a bit confused that we no longer auto-start service after
>>>>>>> package is
>>>>>>>>>> installed. You have to do
>>>>>>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>>>>>>> manually. I'm used to packages that start the thing they install
>>>>>>>>>> immediately. Of course we can just document that clearly on downloads
>>>>>>> pages
>>>>>>>>>> so people won't get lost. What do you all think? Should Ignite
>>>>>>> auto-start
>>>>>>>>>> with default config? Is it possible to do so, but make possible to
>>>>>>> turn it
>>>>>>>>>> off.
>>>>>>>>>>
>>>>>>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>>>>>>> done and 2) is discussed.
>>>>>>>>>>
>>>>>>>>>> Also, would be nice if you create additional JIRA tickets for issues
>>>>>>> in my
>>>>>>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>>>>>>> working) to be implemented in future releases.
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>
>>>>>>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Removed replacement for default-config.xml.
>>>>>>>>>>> Also reimplemented service to be able to run multiple instances of
>>>>>>> Ignite.
>>>>>>>>>>>
>>>>>>>>>>> See updated PR [1].
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>>>>>>> Andrey is right, just use existing default config.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Can we change default configuration file then?
>>>>>>>>>>>>> So that both binary and package deliveries contained PDS turned on
>>>>>>> by
>>>>>>>>>>>>> default?
>>>>>>>>>>>>>
>>>>>>>>>>>>> And what about Multicast Discovery?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>>>>>>> how
>>>>>>>>>>>>> you
>>>>>>>>>>>>>> download and install it.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>>>>>>> use
>>>>>>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>>>>>>> way
>>>>>>>>>>> we
>>>>>>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>>>>>>> amended
>>>>>>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>>>>>>> alongside
>>>>>>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>>>>>>> that
>>>>>>>>>>>>>>> usage of
>>>>>>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>>>>>>> and
>>>>>>>>>>>>>>> updated
>>>>>>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>>>>>>> following
>>>>>>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>>>>>>> file
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>>>>>>> with
>>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>>>>>>> over
>>>>>>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>>>>>>> examples
>>>>>>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>>>>>>> default
>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>> that Ignite users start working with PDS from the very
>>>>>>> beginning.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>>>>>>> service
>>>>>>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>>>>>>> with
>>>>>>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>>>>>>> multicast,
>>>>>>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>>>>>>> there
>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>>>>>>> straight
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>>>>>>> Multicast-enabled by
>>>>>>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>>>>>>> try to
>>>>>>>>>>>>>>>>> revise firewall configuration and create separate service
>>>>>>> firewall
>>>>>>>>>>>>> rules
>>>>>>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>>>>>>> sqlline
>>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>>>>>>> to be
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>>>>>>> fixed
>>>>>>>>>>>>>>> (a
>>>>>>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The problem exists mainly because of
>>>>>>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>>>>>>> overcome
>>>>>>>>>>>>>>>>> this limitation.
>>>>>>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>>>>>>> proposed).
>>>>>>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>>>>>>> package
>>>>>>>>>>>>>>> build,
>>>>>>>>>>>>>>>>> so that those executables work normally form the package
>>>>>>>>>>> installation.
>>>>>>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>>>>>>> all
>>>>>>>>>>>>>>> delivery
>>>>>>>>>>>>>>>>> methods.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>>>>>>> belong to
>>>>>>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>>>>>>> guess
>>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> have to think out of a solution to load libs directly from
>>>>>>> /usr/lib
>>>>>>>>>>>>>>> (along
>>>>>>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> We are building from binary package and not from sources. If
>>>>>>> it is
>>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>>>>>>> repository
>>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>>>>>>> guess
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>>>>>>> the
>>>>>>>>>>>>>>> nearest
>>>>>>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>>>>>>> sources
>>>>>>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>>>>>>> cassandra
>>>>>>>>>>> /
>>>>>>>>>>>>>>> hadoop
>>>>>>>>>>>>>>>>> or alike). And later design package build procedure, which
>>>>>>> includes
>>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>>>>>>> Git
>>>>>>>>>>>>>>>>> repository
>>>>>>>>>>>>>>>>> tag.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>>>>>>> for
>>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>>>>>>> version
>>>>>>>>>>>>>>>>> update,
>>>>>>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>>>>>>> it
>>>>>>>>>>> is
>>>>>>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>>>>>>> '-' in
>>>>>>>>>>>>>>> "In
>>>>>>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>>>>>>> whether
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>>>>>>> Discovery
>>>>>>>>>>>>>>> turned
>>>>>>>>>>>>>>>>> on or not?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>>>>>>> package
>>>>>>>>>>>>>>> design.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>>>>> com/
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sergey Kozlov
>>>>>> GridGain Systems
>>>>>> www.gridgain.com
>>>>>
>>>>
>>>
>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Hi, Denis.
Proposed layout will require changes to release procedure.
Also, I’d like to add to this layout symlink to latest repository, which we will update every release.
But my concern will be — are we allowed to add and store about half a gigabyte of artifacts every 3 months?
> On 18 Jan 2018, at 04:16, Denis Magda <dm...@apache.org> wrote:
>
> Hi Petr,
>
> I would go for the approach implemented for Cassandra. Specifically:
>
> Cassandra root repository includes all the most recent versions and packages for RPM and DEB:
> http://www.apache.org/dist/cassandra/
>
> This is a subdirectory for RPMs there:
> http://www.apache.org/dist/cassandra/redhat/ <http://www.apache.org/dist/cassandra/redhat/>
>
> So, technically it’s possible to have more than one version in the root and not worrying about that something is moved to the archives. It suggests to me that we have only one directory with the latest version in Ignite root because we took this path.
>
> I propose to lay out Ignite root this way:
>
> - latest Ignite version directory
> - rpm
> — 2.4
> — ignite-2.1..rpm
> — etc
> — 2.5
> ---
> - deb
> — 2.5
> — ignite-2.1..deb
> — etc
> — 2.6
> ---
>
> Ignite latest version directory is changed over the time while “rpm” and “deb” are never go to archive. We might archive specific RPM/DEB versions over the time, but presently it’s not an issue.
>
> —
> Denis
>
>> On Jan 17, 2018, at 4:18 AM, Petr Ivanov <mr...@gmail.com> wrote:
>>
>> Hi, Igniters!
>>
>>
>> As part of "Apache Ignite in Packages" initiative, I’ve introduced update to Release procedure, which consists of:
>> - RPM-build instruction [1].
>> - new "vote_3_step_1[rpm]create_rpm_repository.sh” script which does everything else to prepare RPM-repository layout along with source and binary archives.
>> So 2.4 will be uploaded with RPM-repository structure and will be eligible for use as standard third-party repository.
>>
>> Yet, I have concerns regarding this method of packages hosting.
>> Current package-placing architecture assumes, that repository will be available as something like this [2]. But after next major release (2.5 for example) that URL will become unavailable due to archive procedure — users will have to update repository URL for continuous access to that version.
>> If we are to place repository layout to the root [3], then after next major release (2.5 for example) package of 2.4 version will silently become unavailable for installing. However if we manage somehow to have single repository layout with multiple packages versions in Apache Archive, users with plugged in main and archive repositories will get access to all new and archive version of Apache Ignite transparently. Unfortunately — I do not know how this can be achieved.
>>
>> There is also one last approach for package (and artifacts in common) keeping — we can have some kind of third-party mirror, where will be able to host all artifacts, binaries, packages, repositories and other Apache Ignite related stuff with full control and access according to current developer role (reducing impact on Apache’s storages and uploading there only sources).
>>
>>
>> Please, share your thoughts on subject.
>>
>>
>> [1] https://cwiki.apache.org/confluence/display/IGNITE/RPM-packages
>> [2] http://apache.org/dist/ignite/2.4.0/rpm
>> [3] http://apache.org/dist/ignite
>>
>>
>>> On 29 Dec 2017, at 07:54, Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>> Hi, Denis.
>>>
>>>
>>> I was glad to contribute.
>>>
>>> Concerning DEB package — for 2.4 release I’m planning to introduce instructions and spec files for conversion RPM package to DEB, substantially reducing efforts for now because of file structure and install scripts inheritance.
>>> Later, of course, I’m going to prepare fully synchronised source-based package creation process, packages from which are meant for inclusion into official repositories.
>>>
>>>
>>>
>>>> On 28 Dec 2017, at 22:53, Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> Peter, thanks for this Christmas/New Year gift for the community! :)
>>>>
>>>> I’ll be back soon putting together a documentation for this capability.
>>>>
>>>> BTW, what’s the plan in regards DEB packages?
>>>> https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
>>>>
>>>> Do you consider fitting them in AI 2.4 release?
>>>>
>>>> —
>>>> Denis
>>>>
>>>>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>>>>
>>>>> Guys
>>>>>
>>>>> Thanks for your efforts to make it available in 2.4!
>>>>>
>>>>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>>>>>
>>>>>> Guys,
>>>>>>
>>>>>> I've merged change to master branch. Thanks a lot!
>>>>>>
>>>>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>> Fixed remarks and nitpicks.
>>>>>>>
>>>>>>> Also, I purpose to delay service autostart implementation until we’ll
>>>>>> get some first-hand usability feedback.
>>>>>>> Added notes to DEVNOTES.txt about how to start service.
>>>>>>>
>>>>>>>
>>>>>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>>>>>> wrote:
>>>>>>>>
>>>>>>>> Hello once more!
>>>>>>>>
>>>>>>>> Two more nitpicks:
>>>>>>>>
>>>>>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>>>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>>>>>> examples for that.
>>>>>>>> Please remove that alternative for now, create a ticket.
>>>>>>>>
>>>>>>>> rpm -ql apache-ignite | grep \\.java
>>>>>>>> some yardstick and ml sources are there. I don't know who is there to
>>>>>> blame
>>>>>>>> - Ignite release process likely, not RPM building - but they shouldn't
>>>>>> be
>>>>>>>> here I guess?
>>>>>>>>
>>>>>>>> Regards!
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ilya Kasnacheev
>>>>>>>>
>>>>>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>>>>>
>>>>>>>>> Hello again!
>>>>>>>>>
>>>>>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>>>>>> were fixed. I have just two minor remarks:
>>>>>>>>>
>>>>>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>>>>>> works just as well from there as it was from our distribution. visor
>>>>>>>>> doesn't, hence it stays in examples.
>>>>>>>>> - I'm a bit confused that we no longer auto-start service after
>>>>>> package is
>>>>>>>>> installed. You have to do
>>>>>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>>>>>> manually. I'm used to packages that start the thing they install
>>>>>>>>> immediately. Of course we can just document that clearly on downloads
>>>>>> pages
>>>>>>>>> so people won't get lost. What do you all think? Should Ignite
>>>>>> auto-start
>>>>>>>>> with default config? Is it possible to do so, but make possible to
>>>>>> turn it
>>>>>>>>> off.
>>>>>>>>>
>>>>>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>>>>>> done and 2) is discussed.
>>>>>>>>>
>>>>>>>>> Also, would be nice if you create additional JIRA tickets for issues
>>>>>> in my
>>>>>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>>>>>> working) to be implemented in future releases.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>
>>>>>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Removed replacement for default-config.xml.
>>>>>>>>>> Also reimplemented service to be able to run multiple instances of
>>>>>> Ignite.
>>>>>>>>>>
>>>>>>>>>> See updated PR [1].
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>>>>>> Andrey is right, just use existing default config.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Can we change default configuration file then?
>>>>>>>>>>>> So that both binary and package deliveries contained PDS turned on
>>>>>> by
>>>>>>>>>>>> default?
>>>>>>>>>>>>
>>>>>>>>>>>> And what about Multicast Discovery?
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>>>>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>>>>>> how
>>>>>>>>>>>> you
>>>>>>>>>>>>> download and install it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>>>>>> use
>>>>>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>>>>>> way
>>>>>>>>>> we
>>>>>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>>>>>> amended
>>>>>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>>>>>> alongside
>>>>>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>>>>>> that
>>>>>>>>>>>>>> usage of
>>>>>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>>>>>> and
>>>>>>>>>>>>>> updated
>>>>>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>>>>>> following
>>>>>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>>>>>> file
>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>>>>>> with
>>>>>>>>>>>>>> more
>>>>>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>>>>>> over
>>>>>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>>>>>> needed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>>>>>> examples
>>>>>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>>>>>> default
>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>> that Ignite users start working with PDS from the very
>>>>>> beginning.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>>>>>> service
>>>>>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>>>>>> with
>>>>>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>>>>>> multicast,
>>>>>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>>>>>> there
>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>>>>>> straight
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>>>>>> Multicast-enabled by
>>>>>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>>>>>> try to
>>>>>>>>>>>>>>>> revise firewall configuration and create separate service
>>>>>> firewall
>>>>>>>>>>>> rules
>>>>>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>>>>>> sqlline
>>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>>>>>> to be
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>>>>>> fixed
>>>>>>>>>>>>>> (a
>>>>>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The problem exists mainly because of
>>>>>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>>>>>> overcome
>>>>>>>>>>>>>>>> this limitation.
>>>>>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>>>>>> proposed).
>>>>>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>>>>>> package
>>>>>>>>>>>>>> build,
>>>>>>>>>>>>>>>> so that those executables work normally form the package
>>>>>>>>>> installation.
>>>>>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>>>>>> all
>>>>>>>>>>>>>> delivery
>>>>>>>>>>>>>>>> methods.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>>>>>> belong to
>>>>>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>>>>>> guess
>>>>>>>>>>>>>> later
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> have to think out of a solution to load libs directly from
>>>>>> /usr/lib
>>>>>>>>>>>>>> (along
>>>>>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> We are building from binary package and not from sources. If
>>>>>> it is
>>>>>>>>>>>>>> used
>>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>>>>>> repository
>>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>>>>>> guess
>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>>>>>> the
>>>>>>>>>>>>>> nearest
>>>>>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>>>>>> sources
>>>>>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>>>>>> cassandra
>>>>>>>>>> /
>>>>>>>>>>>>>> hadoop
>>>>>>>>>>>>>>>> or alike). And later design package build procedure, which
>>>>>> includes
>>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>>>>>> Git
>>>>>>>>>>>>>>>> repository
>>>>>>>>>>>>>>>> tag.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>>>>>> for
>>>>>>>>>>>> every
>>>>>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>>>>>> version
>>>>>>>>>>>>>>>> update,
>>>>>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>>>>>> it
>>>>>>>>>> is
>>>>>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>>>>>> '-' in
>>>>>>>>>>>>>> "In
>>>>>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>>>>>> whether
>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>>>>>> Discovery
>>>>>>>>>>>>>> turned
>>>>>>>>>>>>>>>> on or not?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>>>>>> package
>>>>>>>>>>>>>> design.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>>>> com/
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sergey Kozlov
>>>>> GridGain Systems
>>>>> www.gridgain.com
>>>>
>>>
>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Denis Magda <dm...@apache.org>.
Hi Petr,
I would go for the approach implemented for Cassandra. Specifically:
Cassandra root repository includes all the most recent versions and packages for RPM and DEB:
http://www.apache.org/dist/cassandra/
This is a subdirectory for RPMs there:
http://www.apache.org/dist/cassandra/redhat/ <http://www.apache.org/dist/cassandra/redhat/>
So, technically it’s possible to have more than one version in the root and not worrying about that something is moved to the archives. It suggests to me that we have only one directory with the latest version in Ignite root because we took this path.
I propose to lay out Ignite root this way:
- latest Ignite version directory
- rpm
— 2.4
— ignite-2.1..rpm
— etc
— 2.5
---
- deb
— 2.5
— ignite-2.1..deb
— etc
— 2.6
---
Ignite latest version directory is changed over the time while “rpm” and “deb” are never go to archive. We might archive specific RPM/DEB versions over the time, but presently it’s not an issue.
—
Denis
> On Jan 17, 2018, at 4:18 AM, Petr Ivanov <mr...@gmail.com> wrote:
>
> Hi, Igniters!
>
>
> As part of "Apache Ignite in Packages" initiative, I’ve introduced update to Release procedure, which consists of:
> - RPM-build instruction [1].
> - new "vote_3_step_1[rpm]create_rpm_repository.sh” script which does everything else to prepare RPM-repository layout along with source and binary archives.
> So 2.4 will be uploaded with RPM-repository structure and will be eligible for use as standard third-party repository.
>
> Yet, I have concerns regarding this method of packages hosting.
> Current package-placing architecture assumes, that repository will be available as something like this [2]. But after next major release (2.5 for example) that URL will become unavailable due to archive procedure — users will have to update repository URL for continuous access to that version.
> If we are to place repository layout to the root [3], then after next major release (2.5 for example) package of 2.4 version will silently become unavailable for installing. However if we manage somehow to have single repository layout with multiple packages versions in Apache Archive, users with plugged in main and archive repositories will get access to all new and archive version of Apache Ignite transparently. Unfortunately — I do not know how this can be achieved.
>
> There is also one last approach for package (and artifacts in common) keeping — we can have some kind of third-party mirror, where will be able to host all artifacts, binaries, packages, repositories and other Apache Ignite related stuff with full control and access according to current developer role (reducing impact on Apache’s storages and uploading there only sources).
>
>
> Please, share your thoughts on subject.
>
>
> [1] https://cwiki.apache.org/confluence/display/IGNITE/RPM-packages
> [2] http://apache.org/dist/ignite/2.4.0/rpm
> [3] http://apache.org/dist/ignite
>
>
>> On 29 Dec 2017, at 07:54, Petr Ivanov <mr...@gmail.com> wrote:
>>
>> Hi, Denis.
>>
>>
>> I was glad to contribute.
>>
>> Concerning DEB package — for 2.4 release I’m planning to introduce instructions and spec files for conversion RPM package to DEB, substantially reducing efforts for now because of file structure and install scripts inheritance.
>> Later, of course, I’m going to prepare fully synchronised source-based package creation process, packages from which are meant for inclusion into official repositories.
>>
>>
>>
>>> On 28 Dec 2017, at 22:53, Denis Magda <dm...@apache.org> wrote:
>>>
>>> Peter, thanks for this Christmas/New Year gift for the community! :)
>>>
>>> I’ll be back soon putting together a documentation for this capability.
>>>
>>> BTW, what’s the plan in regards DEB packages?
>>> https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
>>>
>>> Do you consider fitting them in AI 2.4 release?
>>>
>>> —
>>> Denis
>>>
>>>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>>>
>>>> Guys
>>>>
>>>> Thanks for your efforts to make it available in 2.4!
>>>>
>>>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>>>>
>>>>> Guys,
>>>>>
>>>>> I've merged change to master branch. Thanks a lot!
>>>>>
>>>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>> Fixed remarks and nitpicks.
>>>>>>
>>>>>> Also, I purpose to delay service autostart implementation until we’ll
>>>>> get some first-hand usability feedback.
>>>>>> Added notes to DEVNOTES.txt about how to start service.
>>>>>>
>>>>>>
>>>>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>>>>> wrote:
>>>>>>>
>>>>>>> Hello once more!
>>>>>>>
>>>>>>> Two more nitpicks:
>>>>>>>
>>>>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>>>>> examples for that.
>>>>>>> Please remove that alternative for now, create a ticket.
>>>>>>>
>>>>>>> rpm -ql apache-ignite | grep \\.java
>>>>>>> some yardstick and ml sources are there. I don't know who is there to
>>>>> blame
>>>>>>> - Ignite release process likely, not RPM building - but they shouldn't
>>>>> be
>>>>>>> here I guess?
>>>>>>>
>>>>>>> Regards!
>>>>>>>
>>>>>>> --
>>>>>>> Ilya Kasnacheev
>>>>>>>
>>>>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>>>>
>>>>>>>> Hello again!
>>>>>>>>
>>>>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>>>>> were fixed. I have just two minor remarks:
>>>>>>>>
>>>>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>>>>> works just as well from there as it was from our distribution. visor
>>>>>>>> doesn't, hence it stays in examples.
>>>>>>>> - I'm a bit confused that we no longer auto-start service after
>>>>> package is
>>>>>>>> installed. You have to do
>>>>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>>>>> manually. I'm used to packages that start the thing they install
>>>>>>>> immediately. Of course we can just document that clearly on downloads
>>>>> pages
>>>>>>>> so people won't get lost. What do you all think? Should Ignite
>>>>> auto-start
>>>>>>>> with default config? Is it possible to do so, but make possible to
>>>>> turn it
>>>>>>>> off.
>>>>>>>>
>>>>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>>>>> done and 2) is discussed.
>>>>>>>>
>>>>>>>> Also, would be nice if you create additional JIRA tickets for issues
>>>>> in my
>>>>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>>>>> working) to be implemented in future releases.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ilya Kasnacheev
>>>>>>>>
>>>>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>>
>>>>>>>>> Removed replacement for default-config.xml.
>>>>>>>>> Also reimplemented service to be able to run multiple instances of
>>>>> Ignite.
>>>>>>>>>
>>>>>>>>> See updated PR [1].
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>>>>> Andrey is right, just use existing default config.
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Can we change default configuration file then?
>>>>>>>>>>> So that both binary and package deliveries contained PDS turned on
>>>>> by
>>>>>>>>>>> default?
>>>>>>>>>>>
>>>>>>>>>>> And what about Multicast Discovery?
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>>>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>>>>> how
>>>>>>>>>>> you
>>>>>>>>>>>> download and install it.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>>>>> use
>>>>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>>>>> way
>>>>>>>>> we
>>>>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>>>>> amended
>>>>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>>>>> alongside
>>>>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>>>>> that
>>>>>>>>>>>>> usage of
>>>>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>>>>> and
>>>>>>>>>>>>> updated
>>>>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>>>>> following
>>>>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>>>>> file
>>>>>>>>>>> or
>>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>>>>> with
>>>>>>>>>>>>> more
>>>>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>>>>> over
>>>>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>>>>> needed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>>>>> examples
>>>>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>>>>> default
>>>>>>>>>>>>> so
>>>>>>>>>>>>>>> that Ignite users start working with PDS from the very
>>>>> beginning.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>>>>> service
>>>>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>>>>> with
>>>>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>>>>> multicast,
>>>>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>>>>> there
>>>>>>>>>>>>> can
>>>>>>>>>>>>>>> be
>>>>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>>>>> straight
>>>>>>>>>>> to
>>>>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>>>>> Multicast-enabled by
>>>>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>>>>> try to
>>>>>>>>>>>>>>> revise firewall configuration and create separate service
>>>>> firewall
>>>>>>>>>>> rules
>>>>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>>>>> sqlline
>>>>>>>>>>>>> should
>>>>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>>>>> to be
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>>>>> fixed
>>>>>>>>>>>>> (a
>>>>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The problem exists mainly because of
>>>>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>>>>> overcome
>>>>>>>>>>>>>>> this limitation.
>>>>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>>>>> proposed).
>>>>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>>>>> package
>>>>>>>>>>>>> build,
>>>>>>>>>>>>>>> so that those executables work normally form the package
>>>>>>>>> installation.
>>>>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>>>>> all
>>>>>>>>>>>>> delivery
>>>>>>>>>>>>>>> methods.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>>>>> belong to
>>>>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>>>>> guess
>>>>>>>>>>>>> later
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>> have to think out of a solution to load libs directly from
>>>>> /usr/lib
>>>>>>>>>>>>> (along
>>>>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> We are building from binary package and not from sources. If
>>>>> it is
>>>>>>>>>>>>> used
>>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>>>>> repository
>>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>>>>> guess
>>>>>>>>>>> we
>>>>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>>>>> the
>>>>>>>>>>>>> nearest
>>>>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>>>>> sources
>>>>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>>>>> cassandra
>>>>>>>>> /
>>>>>>>>>>>>> hadoop
>>>>>>>>>>>>>>> or alike). And later design package build procedure, which
>>>>> includes
>>>>>>>>>>>>> getting
>>>>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>>>>> Git
>>>>>>>>>>>>>>> repository
>>>>>>>>>>>>>>> tag.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>>>>> for
>>>>>>>>>>> every
>>>>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>>>>> version
>>>>>>>>>>>>>>> update,
>>>>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>>>>> it
>>>>>>>>> is
>>>>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>>>>> '-' in
>>>>>>>>>>>>> "In
>>>>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>>>>> whether
>>>>>>>>>>> to
>>>>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>>>>> Discovery
>>>>>>>>>>>>> turned
>>>>>>>>>>>>>>> on or not?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>>>>> package
>>>>>>>>>>>>> design.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>>> com/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Sergey Kozlov
>>>> GridGain Systems
>>>> www.gridgain.com
>>>
>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Hi, Igniters!
As part of "Apache Ignite in Packages" initiative, I’ve introduced update to Release procedure, which consists of:
- RPM-build instruction [1].
- new "vote_3_step_1[rpm]create_rpm_repository.sh” script which does everything else to prepare RPM-repository layout along with source and binary archives.
So 2.4 will be uploaded with RPM-repository structure and will be eligible for use as standard third-party repository.
Yet, I have concerns regarding this method of packages hosting.
Current package-placing architecture assumes, that repository will be available as something like this [2]. But after next major release (2.5 for example) that URL will become unavailable due to archive procedure — users will have to update repository URL for continuous access to that version.
If we are to place repository layout to the root [3], then after next major release (2.5 for example) package of 2.4 version will silently become unavailable for installing. However if we manage somehow to have single repository layout with multiple packages versions in Apache Archive, users with plugged in main and archive repositories will get access to all new and archive version of Apache Ignite transparently. Unfortunately — I do not know how this can be achieved.
There is also one last approach for package (and artifacts in common) keeping — we can have some kind of third-party mirror, where will be able to host all artifacts, binaries, packages, repositories and other Apache Ignite related stuff with full control and access according to current developer role (reducing impact on Apache’s storages and uploading there only sources).
Please, share your thoughts on subject.
[1] https://cwiki.apache.org/confluence/display/IGNITE/RPM-packages
[2] http://apache.org/dist/ignite/2.4.0/rpm
[3] http://apache.org/dist/ignite
> On 29 Dec 2017, at 07:54, Petr Ivanov <mr...@gmail.com> wrote:
>
> Hi, Denis.
>
>
> I was glad to contribute.
>
> Concerning DEB package — for 2.4 release I’m planning to introduce instructions and spec files for conversion RPM package to DEB, substantially reducing efforts for now because of file structure and install scripts inheritance.
> Later, of course, I’m going to prepare fully synchronised source-based package creation process, packages from which are meant for inclusion into official repositories.
>
>
>
>> On 28 Dec 2017, at 22:53, Denis Magda <dm...@apache.org> wrote:
>>
>> Peter, thanks for this Christmas/New Year gift for the community! :)
>>
>> I’ll be back soon putting together a documentation for this capability.
>>
>> BTW, what’s the plan in regards DEB packages?
>> https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
>>
>> Do you consider fitting them in AI 2.4 release?
>>
>> —
>> Denis
>>
>>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>>
>>> Guys
>>>
>>> Thanks for your efforts to make it available in 2.4!
>>>
>>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>>>
>>>> Guys,
>>>>
>>>> I've merged change to master branch. Thanks a lot!
>>>>
>>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>>>> Fixed remarks and nitpicks.
>>>>>
>>>>> Also, I purpose to delay service autostart implementation until we’ll
>>>> get some first-hand usability feedback.
>>>>> Added notes to DEVNOTES.txt about how to start service.
>>>>>
>>>>>
>>>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>>>> wrote:
>>>>>>
>>>>>> Hello once more!
>>>>>>
>>>>>> Two more nitpicks:
>>>>>>
>>>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>>>> examples for that.
>>>>>> Please remove that alternative for now, create a ticket.
>>>>>>
>>>>>> rpm -ql apache-ignite | grep \\.java
>>>>>> some yardstick and ml sources are there. I don't know who is there to
>>>> blame
>>>>>> - Ignite release process likely, not RPM building - but they shouldn't
>>>> be
>>>>>> here I guess?
>>>>>>
>>>>>> Regards!
>>>>>>
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>>>
>>>>>>> Hello again!
>>>>>>>
>>>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>>>> were fixed. I have just two minor remarks:
>>>>>>>
>>>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>>>> works just as well from there as it was from our distribution. visor
>>>>>>> doesn't, hence it stays in examples.
>>>>>>> - I'm a bit confused that we no longer auto-start service after
>>>> package is
>>>>>>> installed. You have to do
>>>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>>>> manually. I'm used to packages that start the thing they install
>>>>>>> immediately. Of course we can just document that clearly on downloads
>>>> pages
>>>>>>> so people won't get lost. What do you all think? Should Ignite
>>>> auto-start
>>>>>>> with default config? Is it possible to do so, but make possible to
>>>> turn it
>>>>>>> off.
>>>>>>>
>>>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>>>> done and 2) is discussed.
>>>>>>>
>>>>>>> Also, would be nice if you create additional JIRA tickets for issues
>>>> in my
>>>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>>>> working) to be implemented in future releases.
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> --
>>>>>>> Ilya Kasnacheev
>>>>>>>
>>>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>>
>>>>>>>> Removed replacement for default-config.xml.
>>>>>>>> Also reimplemented service to be able to run multiple instances of
>>>> Ignite.
>>>>>>>>
>>>>>>>> See updated PR [1].
>>>>>>>>
>>>>>>>>
>>>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>>>> Andrey is right, just use existing default config.
>>>>>>>>>
>>>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Can we change default configuration file then?
>>>>>>>>>> So that both binary and package deliveries contained PDS turned on
>>>> by
>>>>>>>>>> default?
>>>>>>>>>>
>>>>>>>>>> And what about Multicast Discovery?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>>>> how
>>>>>>>>>> you
>>>>>>>>>>> download and install it.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>>>> use
>>>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>>
>>>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>>>> way
>>>>>>>> we
>>>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>>>> amended
>>>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>>>> alongside
>>>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>>>> that
>>>>>>>>>>>> usage of
>>>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>>>> and
>>>>>>>>>>>> updated
>>>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>>>> following
>>>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>>>> file
>>>>>>>>>> or
>>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>>>> with
>>>>>>>>>>>> more
>>>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>>>> over
>>>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>>>> needed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>>>> examples
>>>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>>>> default
>>>>>>>>>>>> so
>>>>>>>>>>>>>> that Ignite users start working with PDS from the very
>>>> beginning.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>>>> service
>>>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>>>> with
>>>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>>>> multicast,
>>>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>>>> there
>>>>>>>>>>>> can
>>>>>>>>>>>>>> be
>>>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>>>> straight
>>>>>>>>>> to
>>>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>>>> Multicast-enabled by
>>>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>>>> try to
>>>>>>>>>>>>>> revise firewall configuration and create separate service
>>>> firewall
>>>>>>>>>> rules
>>>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>>>> sqlline
>>>>>>>>>>>> should
>>>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>>>> to be
>>>>>>>>>>>> in
>>>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>>>> fixed
>>>>>>>>>>>> (a
>>>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The problem exists mainly because of
>>>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>>>> overcome
>>>>>>>>>>>>>> this limitation.
>>>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>>>> proposed).
>>>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>>>> package
>>>>>>>>>>>> build,
>>>>>>>>>>>>>> so that those executables work normally form the package
>>>>>>>> installation.
>>>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>>>> all
>>>>>>>>>>>> delivery
>>>>>>>>>>>>>> methods.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>>>> belong to
>>>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>>>> guess
>>>>>>>>>>>> later
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>> have to think out of a solution to load libs directly from
>>>> /usr/lib
>>>>>>>>>>>> (along
>>>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> We are building from binary package and not from sources. If
>>>> it is
>>>>>>>>>>>> used
>>>>>>>>>>>>>> by
>>>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>>>> repository
>>>>>>>>>>>>>> will
>>>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>>>> guess
>>>>>>>>>> we
>>>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>>>> the
>>>>>>>>>>>> nearest
>>>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>>>> sources
>>>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>>>> cassandra
>>>>>>>> /
>>>>>>>>>>>> hadoop
>>>>>>>>>>>>>> or alike). And later design package build procedure, which
>>>> includes
>>>>>>>>>>>> getting
>>>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>>>> Git
>>>>>>>>>>>>>> repository
>>>>>>>>>>>>>> tag.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>>>> for
>>>>>>>>>> every
>>>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>>>> version
>>>>>>>>>>>>>> update,
>>>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>>>> it
>>>>>>>> is
>>>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>>>> '-' in
>>>>>>>>>>>> "In
>>>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>>>> whether
>>>>>>>>>> to
>>>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>>>> Discovery
>>>>>>>>>>>> turned
>>>>>>>>>>>>>> on or not?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>>>> package
>>>>>>>>>>>> design.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>>> com/
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Sergey Kozlov
>>> GridGain Systems
>>> www.gridgain.com
>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Hi, Denis.
I was glad to contribute.
Concerning DEB package — for 2.4 release I’m planning to introduce instructions and spec files for conversion RPM package to DEB, substantially reducing efforts for now because of file structure and install scripts inheritance.
Later, of course, I’m going to prepare fully synchronised source-based package creation process, packages from which are meant for inclusion into official repositories.
> On 28 Dec 2017, at 22:53, Denis Magda <dm...@apache.org> wrote:
>
> Peter, thanks for this Christmas/New Year gift for the community! :)
>
> I’ll be back soon putting together a documentation for this capability.
>
> BTW, what’s the plan in regards DEB packages?
> https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
>
> Do you consider fitting them in AI 2.4 release?
>
> —
> Denis
>
>> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>>
>> Guys
>>
>> Thanks for your efforts to make it available in 2.4!
>>
>> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>>
>>> Guys,
>>>
>>> I've merged change to master branch. Thanks a lot!
>>>
>>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>>> Fixed remarks and nitpicks.
>>>>
>>>> Also, I purpose to delay service autostart implementation until we’ll
>>> get some first-hand usability feedback.
>>>> Added notes to DEVNOTES.txt about how to start service.
>>>>
>>>>
>>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>>> wrote:
>>>>>
>>>>> Hello once more!
>>>>>
>>>>> Two more nitpicks:
>>>>>
>>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>>> examples for that.
>>>>> Please remove that alternative for now, create a ticket.
>>>>>
>>>>> rpm -ql apache-ignite | grep \\.java
>>>>> some yardstick and ml sources are there. I don't know who is there to
>>> blame
>>>>> - Ignite release process likely, not RPM building - but they shouldn't
>>> be
>>>>> here I guess?
>>>>>
>>>>> Regards!
>>>>>
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>>
>>>>>> Hello again!
>>>>>>
>>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>>> were fixed. I have just two minor remarks:
>>>>>>
>>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>>> works just as well from there as it was from our distribution. visor
>>>>>> doesn't, hence it stays in examples.
>>>>>> - I'm a bit confused that we no longer auto-start service after
>>> package is
>>>>>> installed. You have to do
>>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>>> manually. I'm used to packages that start the thing they install
>>>>>> immediately. Of course we can just document that clearly on downloads
>>> pages
>>>>>> so people won't get lost. What do you all think? Should Ignite
>>> auto-start
>>>>>> with default config? Is it possible to do so, but make possible to
>>> turn it
>>>>>> off.
>>>>>>
>>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>>> done and 2) is discussed.
>>>>>>
>>>>>> Also, would be nice if you create additional JIRA tickets for issues
>>> in my
>>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>>> working) to be implemented in future releases.
>>>>>>
>>>>>> Regards,
>>>>>>
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>>
>>>>>>> Removed replacement for default-config.xml.
>>>>>>> Also reimplemented service to be able to run multiple instances of
>>> Ignite.
>>>>>>>
>>>>>>> See updated PR [1].
>>>>>>>
>>>>>>>
>>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>>> wrote:
>>>>>>>>
>>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>>> Andrey is right, just use existing default config.
>>>>>>>>
>>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Can we change default configuration file then?
>>>>>>>>> So that both binary and package deliveries contained PDS turned on
>>> by
>>>>>>>>> default?
>>>>>>>>>
>>>>>>>>> And what about Multicast Discovery?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>>
>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>>> how
>>>>>>>>> you
>>>>>>>>>> download and install it.
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>>> use
>>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>>
>>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>>> way
>>>>>>> we
>>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>>
>>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>>> amended
>>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>>> alongside
>>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>>> that
>>>>>>>>>>> usage of
>>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>>
>>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>>> and
>>>>>>>>>>> updated
>>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>>> following
>>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>>> file
>>>>>>>>> or
>>>>>>>>>>>>>> directory
>>>>>>>>>>>>>
>>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>>> with
>>>>>>>>>>> more
>>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>>> over
>>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>>> needed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>>> examples
>>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>>
>>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>>> default
>>>>>>>>>>> so
>>>>>>>>>>>>> that Ignite users start working with PDS from the very
>>> beginning.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>>> service
>>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>>> with
>>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>>
>>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>>> multicast,
>>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>>> there
>>>>>>>>>>> can
>>>>>>>>>>>>> be
>>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>>> straight
>>>>>>>>> to
>>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>>> Multicast-enabled by
>>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>>> try to
>>>>>>>>>>>>> revise firewall configuration and create separate service
>>> firewall
>>>>>>>>> rules
>>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>>> sqlline
>>>>>>>>>>> should
>>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>>> to be
>>>>>>>>>>> in
>>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>>> fixed
>>>>>>>>>>> (a
>>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The problem exists mainly because of
>>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>>> overcome
>>>>>>>>>>>>> this limitation.
>>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>>> proposed).
>>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>>> package
>>>>>>>>>>> build,
>>>>>>>>>>>>> so that those executables work normally form the package
>>>>>>> installation.
>>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>>> all
>>>>>>>>>>> delivery
>>>>>>>>>>>>> methods.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>>> belong to
>>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>>> guess
>>>>>>>>>>> later
>>>>>>>>>>>>> we
>>>>>>>>>>>>> have to think out of a solution to load libs directly from
>>> /usr/lib
>>>>>>>>>>> (along
>>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> We are building from binary package and not from sources. If
>>> it is
>>>>>>>>>>> used
>>>>>>>>>>>>> by
>>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>>> repository
>>>>>>>>>>>>> will
>>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>>> guess
>>>>>>>>> we
>>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>>> the
>>>>>>>>>>> nearest
>>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>>> sources
>>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>>> cassandra
>>>>>>> /
>>>>>>>>>>> hadoop
>>>>>>>>>>>>> or alike). And later design package build procedure, which
>>> includes
>>>>>>>>>>> getting
>>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>>> Git
>>>>>>>>>>>>> repository
>>>>>>>>>>>>> tag.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>>> for
>>>>>>>>> every
>>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>>> version
>>>>>>>>>>>>> update,
>>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>>> it
>>>>>>> is
>>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>>> '-' in
>>>>>>>>>>> "In
>>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>>
>>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>>> whether
>>>>>>>>> to
>>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>>> Discovery
>>>>>>>>>>> turned
>>>>>>>>>>>>> on or not?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>>> package
>>>>>>>>>>> design.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>>> com/
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Sergey Kozlov
>> GridGain Systems
>> www.gridgain.com
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Denis Magda <dm...@apache.org>.
Peter, thanks for this Christmas/New Year gift for the community! :)
I’ll be back soon putting together a documentation for this capability.
BTW, what’s the plan in regards DEB packages?
https://issues.apache.org/jira/browse/IGNITE-7108 <https://issues.apache.org/jira/browse/IGNITE-7108>
Do you consider fitting them in AI 2.4 release?
—
Denis
> On Dec 28, 2017, at 5:23 AM, Sergey Kozlov <sk...@gridgain.com> wrote:
>
> Guys
>
> Thanks for your efforts to make it available in 2.4!
>
> On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
>
>> Guys,
>>
>> I've merged change to master branch. Thanks a lot!
>>
>> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
>>> Fixed remarks and nitpicks.
>>>
>>> Also, I purpose to delay service autostart implementation until we’ll
>> get some first-hand usability feedback.
>>> Added notes to DEVNOTES.txt about how to start service.
>>>
>>>
>>>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
>> wrote:
>>>>
>>>> Hello once more!
>>>>
>>>> Two more nitpicks:
>>>>
>>>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>>>> work since a) visor wants to write to /usr/share, and b) we moved it to
>>>> examples for that.
>>>> Please remove that alternative for now, create a ticket.
>>>>
>>>> rpm -ql apache-ignite | grep \\.java
>>>> some yardstick and ml sources are there. I don't know who is there to
>> blame
>>>> - Ignite release process likely, not RPM building - but they shouldn't
>> be
>>>> here I guess?
>>>>
>>>> Regards!
>>>>
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>>>
>>>>> Hello again!
>>>>>
>>>>> I have reviewed the modified patch. All the things in my critical list
>>>>> were fixed. I have just two minor remarks:
>>>>>
>>>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>>>> works just as well from there as it was from our distribution. visor
>>>>> doesn't, hence it stays in examples.
>>>>> - I'm a bit confused that we no longer auto-start service after
>> package is
>>>>> installed. You have to do
>>>>> sudo systemctl start apache-ignite@default-config.xml
>>>>> manually. I'm used to packages that start the thing they install
>>>>> immediately. Of course we can just document that clearly on downloads
>> pages
>>>>> so people won't get lost. What do you all think? Should Ignite
>> auto-start
>>>>> with default config? Is it possible to do so, but make possible to
>> turn it
>>>>> off.
>>>>>
>>>>> So from my side I recommend this pull request for inclusion after 1) is
>>>>> done and 2) is discussed.
>>>>>
>>>>> Also, would be nice if you create additional JIRA tickets for issues
>> in my
>>>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>>>> working) to be implemented in future releases.
>>>>>
>>>>> Regards,
>>>>>
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>>>
>>>>>> Removed replacement for default-config.xml.
>>>>>> Also reimplemented service to be able to run multiple instances of
>> Ignite.
>>>>>>
>>>>>> See updated PR [1].
>>>>>>
>>>>>>
>>>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>>>> https://github.com/apache/ignite/pull/3171>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
>> wrote:
>>>>>>>
>>>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>>>> Andrey is right, just use existing default config.
>>>>>>>
>>>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>> Can we change default configuration file then?
>>>>>>>> So that both binary and package deliveries contained PDS turned on
>> by
>>>>>>>> default?
>>>>>>>>
>>>>>>>> And what about Multicast Discovery?
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
>>>
>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> I agree with Andrey. The product should be identical regardless of
>> how
>>>>>>>> you
>>>>>>>>> download and install it.
>>>>>>>>>
>>>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
>> use
>>>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>>>
>>>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>>>> Hello Igniters!
>>>>>>>>>>>
>>>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
>> way
>>>>>> we
>>>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>>>
>>>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>>>> amended
>>>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>>>
>>>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>>>> alongside
>>>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
>> that
>>>>>>>>>> usage of
>>>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>>>
>>>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
>> and
>>>>>>>>>> updated
>>>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>>>> following
>>>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>>>> file
>>>>>>>> or
>>>>>>>>>>>>> directory
>>>>>>>>>>>>
>>>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>>>> with
>>>>>>>>>> more
>>>>>>>>>>>> universal variant.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>>>> default-config.xml) into build scripts. We should just copy
>> over
>>>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>>>> needed.
>>>>>>>>>>>>
>>>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>>>> examples
>>>>>>>>>>>>> but disable by default.
>>>>>>>>>>>>
>>>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>>>> default
>>>>>>>>>> so
>>>>>>>>>>>> that Ignite users start working with PDS from the very
>> beginning.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>>>> service
>>>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>>>> with
>>>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>>>> multicast,
>>>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>>>> there
>>>>>>>>>> can
>>>>>>>>>>>> be
>>>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>>>> straight
>>>>>>>> to
>>>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>>>> Multicast-enabled by
>>>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>>>> try to
>>>>>>>>>>>> revise firewall configuration and create separate service
>> firewall
>>>>>>>> rules
>>>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
>> sqlline
>>>>>>>>>> should
>>>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>>>> to be
>>>>>>>>>> in
>>>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>>>> fixed
>>>>>>>>>> (a
>>>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>>>> /etc/apache-ignite,
>>>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>>>
>>>>>>>>>>>> The problem exists mainly because of
>> sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>>>> overcome
>>>>>>>>>>>> this limitation.
>>>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>>>> proposed).
>>>>>>>>>>>> Secondly, I can design a patch, which can be applied during
>> package
>>>>>>>>>> build,
>>>>>>>>>>>> so that those executables work normally form the package
>>>>>> installation.
>>>>>>>>>>>> And lastly, redesign them to be universal and compatible with
>> all
>>>>>>>>>> delivery
>>>>>>>>>>>> methods.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> Not everything should go into /usr/share. classpath libs
>> belong to
>>>>>>>>>>>>> /usr/lib.
>>>>>>>>>>>>
>>>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
>> guess
>>>>>>>>>> later
>>>>>>>>>>>> we
>>>>>>>>>>>> have to think out of a solution to load libs directly from
>> /usr/lib
>>>>>>>>>> (along
>>>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> We are building from binary package and not from sources. If
>> it is
>>>>>>>>>> used
>>>>>>>>>>>> by
>>>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>>>> repository
>>>>>>>>>>>> will
>>>>>>>>>>>>> want to build from source.
>>>>>>>>>>>>
>>>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>>>> guess
>>>>>>>> we
>>>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
>> the
>>>>>>>>>> nearest
>>>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>>>> sources
>>>>>>>>>>>> package (as it is currently done in apache.org/dist for
>> cassandra
>>>>>> /
>>>>>>>>>> hadoop
>>>>>>>>>>>> or alike). And later design package build procedure, which
>> includes
>>>>>>>>>> getting
>>>>>>>>>>>> sources from sources distribution, or even from Apache Ignite
>> Git
>>>>>>>>>>>> repository
>>>>>>>>>>>> tag.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
>> for
>>>>>>>> every
>>>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>>>
>>>>>>>>>>>> Release procedure on TC is already has some task for project
>>>>>> version
>>>>>>>>>>>> update,
>>>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
>> it
>>>>>> is
>>>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>>>> Package description may be improved. service description lacks
>>>>>> '-' in
>>>>>>>>>> "In
>>>>>>>>>>>>> Memory".
>>>>>>>>>>>>
>>>>>>>>>>>> Fixed.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>>>> whether
>>>>>>>> to
>>>>>>>>>>>> supply package with default-config.xml that has Multicast
>> Discovery
>>>>>>>>>> turned
>>>>>>>>>>>> on or not?
>>>>>>>>>>>>
>>>>>>>>>>>> Also I'll appreciate any other comments concerning current
>> package
>>>>>>>>>> design.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
>> com/
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>
>>
>
>
>
> --
> Sergey Kozlov
> GridGain Systems
> www.gridgain.com
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Sergey Kozlov <sk...@gridgain.com>.
Guys
Thanks for your efforts to make it available in 2.4!
On Thu, Dec 28, 2017 at 4:15 PM, Andrey Gura <ag...@apache.org> wrote:
> Guys,
>
> I've merged change to master branch. Thanks a lot!
>
> On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
> > Fixed remarks and nitpicks.
> >
> > Also, I purpose to delay service autostart implementation until we’ll
> get some first-hand usability feedback.
> > Added notes to DEVNOTES.txt about how to start service.
> >
> >
> >> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com>
> wrote:
> >>
> >> Hello once more!
> >>
> >> Two more nitpicks:
> >>
> >> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
> >> work since a) visor wants to write to /usr/share, and b) we moved it to
> >> examples for that.
> >> Please remove that alternative for now, create a ticket.
> >>
> >> rpm -ql apache-ignite | grep \\.java
> >> some yardstick and ml sources are there. I don't know who is there to
> blame
> >> - Ignite release process likely, not RPM building - but they shouldn't
> be
> >> here I guess?
> >>
> >> Regards!
> >>
> >> --
> >> Ilya Kasnacheev
> >>
> >> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
> >>
> >>> Hello again!
> >>>
> >>> I have reviewed the modified patch. All the things in my critical list
> >>> were fixed. I have just two minor remarks:
> >>>
> >>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
> >>> works just as well from there as it was from our distribution. visor
> >>> doesn't, hence it stays in examples.
> >>> - I'm a bit confused that we no longer auto-start service after
> package is
> >>> installed. You have to do
> >>> sudo systemctl start apache-ignite@default-config.xml
> >>> manually. I'm used to packages that start the thing they install
> >>> immediately. Of course we can just document that clearly on downloads
> pages
> >>> so people won't get lost. What do you all think? Should Ignite
> auto-start
> >>> with default config? Is it possible to do so, but make possible to
> turn it
> >>> off.
> >>>
> >>> So from my side I recommend this pull request for inclusion after 1) is
> >>> done and 2) is discussed.
> >>>
> >>> Also, would be nice if you create additional JIRA tickets for issues
> in my
> >>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
> >>> working) to be implemented in future releases.
> >>>
> >>> Regards,
> >>>
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> >>>
> >>>> Removed replacement for default-config.xml.
> >>>> Also reimplemented service to be able to run multiple instances of
> Ignite.
> >>>>
> >>>> See updated PR [1].
> >>>>
> >>>>
> >>>> [1] https://github.com/apache/ignite/pull/3171 <
> >>>> https://github.com/apache/ignite/pull/3171>
> >>>>
> >>>>
> >>>>
> >>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org>
> wrote:
> >>>>>
> >>>>> PDS and discovery settings are unrelated to the packaging task.
> >>>>> Andrey is right, just use existing default config.
> >>>>>
> >>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>> Can we change default configuration file then?
> >>>>>> So that both binary and package deliveries contained PDS turned on
> by
> >>>>>> default?
> >>>>>>
> >>>>>> And what about Multicast Discovery?
> >>>>>>
> >>>>>>
> >>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <dsetrakyan@apache.org
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> I agree with Andrey. The product should be identical regardless of
> how
> >>>>>> you
> >>>>>>> download and install it.
> >>>>>>>
> >>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
> >>>> wrote:
> >>>>>>>
> >>>>>>>> Hi,
> >>>>>>>>
> >>>>>>>> I think we should provide the same default configuration for all
> >>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should
> use
> >>>>>>>> default configuration file like to standard binary release.
> >>>>>>>>
> >>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
> >>>>>>>> <il...@gmail.com> wrote:
> >>>>>>>>> Hello Igniters!
> >>>>>>>>>
> >>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This
> way
> >>>> we
> >>>>>>>>> could enable it in RPM build with easy heart.
> >>>>>>>>>
> >>>>>>>>> The rest of your answers seem spot on. I'll happily review your
> >>>> amended
> >>>>>>>>> patch when it is ready (later this week?)
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Ilya Kasnacheev
> >>>>>>>>>
> >>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
> >>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
> >>>>>>>> alongside
> >>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm
> that
> >>>>>>>> usage of
> >>>>>>>>>>> tabs is desired in this case.
> >>>>>>>>>>
> >>>>>>>>>> My fault - got used to editing such files in default vim. Fixed
> and
> >>>>>>>> updated
> >>>>>>>>>> vim settings to indent with spaces.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
> >>>> following
> >>>>>>>>>>> message, which I had to fix in spec
> >>>>>>>>>>> + cd 'apache-ignite-*'
> >>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
> >>>> file
> >>>>>> or
> >>>>>>>>>>> directory
> >>>>>>>>>>
> >>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
> >>>> with
> >>>>>>>> more
> >>>>>>>>>> universal variant.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> We absolutely should not inline configuration files (such as
> >>>>>>>>>>> default-config.xml) into build scripts. We should just copy
> over
> >>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
> >>>> needed.
> >>>>>>>>>>
> >>>>>>>>>> Extracted corresponding sections into separate files.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
> >>>>>>>> examples
> >>>>>>>>>>> but disable by default.
> >>>>>>>>>>
> >>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
> >>>>>> default
> >>>>>>>> so
> >>>>>>>>>> that Ignite users start working with PDS from the very
> beginning.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
> >>>> service
> >>>>>>>>>>> configurations should be stored in sources, as files, supplied
> >>>> with
> >>>>>>>>>>> release (somewhere in packages/) and installed from there
> >>>>>>>>>>
> >>>>>>>>>> Unfortunately, I did not find any other way to open ports and
> >>>>>> multicast,
> >>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
> >>>> there
> >>>>>>>> can
> >>>>>>>>>> be
> >>>>>>>>>> no separate service firewall rules file (as can be found here:
> >>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
> >>>> straight
> >>>>>> to
> >>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
> >>>> Multicast-enabled by
> >>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
> >>>> try to
> >>>>>>>>>> revise firewall configuration and create separate service
> firewall
> >>>>>> rules
> >>>>>>>>>> file (but, I guess, much later).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
> >>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that
> sqlline
> >>>>>>>> should
> >>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
> >>>> to be
> >>>>>>>> in
> >>>>>>>>>>> $PATH. And figure out how to connect by itself.
> >>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
> >>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
> >>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
> >>>> fixed
> >>>>>>>> (a
> >>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
> >>>> /etc/apache-ignite,
> >>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
> >>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
> >>>>>>>>>>
> >>>>>>>>>> The problem exists mainly because of
> sqlline.sh|ignitevisorcmd.sh
> >>>>>>>>>> unaware-of-package design and I see the following 3-step way to
> >>>>>> overcome
> >>>>>>>>>> this limitation.
> >>>>>>>>>> At first we hide this executables file in examples (As Iliya
> >>>>>> proposed).
> >>>>>>>>>> Secondly, I can design a patch, which can be applied during
> package
> >>>>>>>> build,
> >>>>>>>>>> so that those executables work normally form the package
> >>>> installation.
> >>>>>>>>>> And lastly, redesign them to be universal and compatible with
> all
> >>>>>>>> delivery
> >>>>>>>>>> methods.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> Not everything should go into /usr/share. classpath libs
> belong to
> >>>>>>>>>>> /usr/lib.
> >>>>>>>>>>
> >>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
> >>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I
> guess
> >>>>>>>> later
> >>>>>>>>>> we
> >>>>>>>>>> have to think out of a solution to load libs directly from
> /usr/lib
> >>>>>>>> (along
> >>>>>>>>>> with configurations / logs / work / etc.)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> We are building from binary package and not from sources. If
> it is
> >>>>>>>> used
> >>>>>>>>>> by
> >>>>>>>>>>> internal RPM building this is tolerable, but any distro or
> >>>> repository
> >>>>>>>>>> will
> >>>>>>>>>>> want to build from source.
> >>>>>>>>>>
> >>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
> >>>> guess
> >>>>>> we
> >>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in
> the
> >>>>>>>> nearest
> >>>>>>>>>> release our package built from binary archive and do not supply
> >>>>>> sources
> >>>>>>>>>> package (as it is currently done in apache.org/dist for
> cassandra
> >>>> /
> >>>>>>>> hadoop
> >>>>>>>>>> or alike). And later design package build procedure, which
> includes
> >>>>>>>> getting
> >>>>>>>>>> sources from sources distribution, or even from Apache Ignite
> Git
> >>>>>>>>>> repository
> >>>>>>>>>> tag.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed
> for
> >>>>>> every
> >>>>>>>>>>> release. I'm not sure if it's possible to extract.
> >>>>>>>>>>
> >>>>>>>>>> Release procedure on TC is already has some task for project
> >>>> version
> >>>>>>>>>> update,
> >>>>>>>>>> so I guess it is not so difficult to update it accordingly (as
> it
> >>>> is
> >>>>>>>>>> currently done for C++ / .Net internal versions).
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Ilya Kasnacheev wrote
> >>>>>>>>>>> Package description may be improved. service description lacks
> >>>> '-' in
> >>>>>>>> "In
> >>>>>>>>>>> Memory".
> >>>>>>>>>>
> >>>>>>>>>> Fixed.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> So the only question to discuss (according Iliya's review) is
> >>>> whether
> >>>>>> to
> >>>>>>>>>> supply package with default-config.xml that has Multicast
> Discovery
> >>>>>>>> turned
> >>>>>>>>>> on or not?
> >>>>>>>>>>
> >>>>>>>>>> Also I'll appreciate any other comments concerning current
> package
> >>>>>>>> design.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.
> com/
> >>>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>>>
> >>>
> >
>
--
Sergey Kozlov
GridGain Systems
www.gridgain.com
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Andrey Gura <ag...@apache.org>.
Guys,
I've merged change to master branch. Thanks a lot!
On Thu, Dec 28, 2017 at 3:54 PM, Petr Ivanov <mr...@gmail.com> wrote:
> Fixed remarks and nitpicks.
>
> Also, I purpose to delay service autostart implementation until we’ll get some first-hand usability feedback.
> Added notes to DEVNOTES.txt about how to start service.
>
>
>> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com> wrote:
>>
>> Hello once more!
>>
>> Two more nitpicks:
>>
>> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
>> work since a) visor wants to write to /usr/share, and b) we moved it to
>> examples for that.
>> Please remove that alternative for now, create a ticket.
>>
>> rpm -ql apache-ignite | grep \\.java
>> some yardstick and ml sources are there. I don't know who is there to blame
>> - Ignite release process likely, not RPM building - but they shouldn't be
>> here I guess?
>>
>> Regards!
>>
>> --
>> Ilya Kasnacheev
>>
>> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>>
>>> Hello again!
>>>
>>> I have reviewed the modified patch. All the things in my critical list
>>> were fixed. I have just two minor remarks:
>>>
>>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>>> works just as well from there as it was from our distribution. visor
>>> doesn't, hence it stays in examples.
>>> - I'm a bit confused that we no longer auto-start service after package is
>>> installed. You have to do
>>> sudo systemctl start apache-ignite@default-config.xml
>>> manually. I'm used to packages that start the thing they install
>>> immediately. Of course we can just document that clearly on downloads pages
>>> so people won't get lost. What do you all think? Should Ignite auto-start
>>> with default config? Is it possible to do so, but make possible to turn it
>>> off.
>>>
>>> So from my side I recommend this pull request for inclusion after 1) is
>>> done and 2) is discussed.
>>>
>>> Also, would be nice if you create additional JIRA tickets for issues in my
>>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>>> working) to be implemented in future releases.
>>>
>>> Regards,
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>>
>>>> Removed replacement for default-config.xml.
>>>> Also reimplemented service to be able to run multiple instances of Ignite.
>>>>
>>>> See updated PR [1].
>>>>
>>>>
>>>> [1] https://github.com/apache/ignite/pull/3171 <
>>>> https://github.com/apache/ignite/pull/3171>
>>>>
>>>>
>>>>
>>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org> wrote:
>>>>>
>>>>> PDS and discovery settings are unrelated to the packaging task.
>>>>> Andrey is right, just use existing default config.
>>>>>
>>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>>> wrote:
>>>>>
>>>>>> Can we change default configuration file then?
>>>>>> So that both binary and package deliveries contained PDS turned on by
>>>>>> default?
>>>>>>
>>>>>> And what about Multicast Discovery?
>>>>>>
>>>>>>
>>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> I agree with Andrey. The product should be identical regardless of how
>>>>>> you
>>>>>>> download and install it.
>>>>>>>
>>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>>> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I think we should provide the same default configuration for all
>>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should use
>>>>>>>> default configuration file like to standard binary release.
>>>>>>>>
>>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>>> <il...@gmail.com> wrote:
>>>>>>>>> Hello Igniters!
>>>>>>>>>
>>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This way
>>>> we
>>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>>
>>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>>> amended
>>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ilya Kasnacheev
>>>>>>>>>
>>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>>> alongside
>>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
>>>>>>>> usage of
>>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>>
>>>>>>>>>> My fault - got used to editing such files in default vim. Fixed and
>>>>>>>> updated
>>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>>> following
>>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>>> file
>>>>>> or
>>>>>>>>>>> directory
>>>>>>>>>>
>>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>>> with
>>>>>>>> more
>>>>>>>>>> universal variant.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>>> default-config.xml) into build scripts. We should just copy over
>>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>>> needed.
>>>>>>>>>>
>>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>>> examples
>>>>>>>>>>> but disable by default.
>>>>>>>>>>
>>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>>> default
>>>>>>>> so
>>>>>>>>>> that Ignite users start working with PDS from the very beginning.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>>> service
>>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>>> with
>>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>>
>>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>>> multicast,
>>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>>> there
>>>>>>>> can
>>>>>>>>>> be
>>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>>> straight
>>>>>> to
>>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>>> Multicast-enabled by
>>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>>> try to
>>>>>>>>>> revise firewall configuration and create separate service firewall
>>>>>> rules
>>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
>>>>>>>> should
>>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>>> to be
>>>>>>>> in
>>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>>> fixed
>>>>>>>> (a
>>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>>> /etc/apache-ignite,
>>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>>
>>>>>>>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>>> overcome
>>>>>>>>>> this limitation.
>>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>>> proposed).
>>>>>>>>>> Secondly, I can design a patch, which can be applied during package
>>>>>>>> build,
>>>>>>>>>> so that those executables work normally form the package
>>>> installation.
>>>>>>>>>> And lastly, redesign them to be universal and compatible with all
>>>>>>>> delivery
>>>>>>>>>> methods.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> Not everything should go into /usr/share. classpath libs belong to
>>>>>>>>>>> /usr/lib.
>>>>>>>>>>
>>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
>>>>>>>> later
>>>>>>>>>> we
>>>>>>>>>> have to think out of a solution to load libs directly from /usr/lib
>>>>>>>> (along
>>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> We are building from binary package and not from sources. If it is
>>>>>>>> used
>>>>>>>>>> by
>>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>>> repository
>>>>>>>>>> will
>>>>>>>>>>> want to build from source.
>>>>>>>>>>
>>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>>> guess
>>>>>> we
>>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in the
>>>>>>>> nearest
>>>>>>>>>> release our package built from binary archive and do not supply
>>>>>> sources
>>>>>>>>>> package (as it is currently done in apache.org/dist for cassandra
>>>> /
>>>>>>>> hadoop
>>>>>>>>>> or alike). And later design package build procedure, which includes
>>>>>>>> getting
>>>>>>>>>> sources from sources distribution, or even from Apache Ignite Git
>>>>>>>>>> repository
>>>>>>>>>> tag.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed for
>>>>>> every
>>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>>
>>>>>>>>>> Release procedure on TC is already has some task for project
>>>> version
>>>>>>>>>> update,
>>>>>>>>>> so I guess it is not so difficult to update it accordingly (as it
>>>> is
>>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>>> Package description may be improved. service description lacks
>>>> '-' in
>>>>>>>> "In
>>>>>>>>>>> Memory".
>>>>>>>>>>
>>>>>>>>>> Fixed.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>>> whether
>>>>>> to
>>>>>>>>>> supply package with default-config.xml that has Multicast Discovery
>>>>>>>> turned
>>>>>>>>>> on or not?
>>>>>>>>>>
>>>>>>>>>> Also I'll appreciate any other comments concerning current package
>>>>>>>> design.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Fixed remarks and nitpicks.
Also, I purpose to delay service autostart implementation until we’ll get some first-hand usability feedback.
Added notes to DEVNOTES.txt about how to start service.
> On 28 Dec 2017, at 14:32, Ilya Kasnacheev <il...@gmail.com> wrote:
>
> Hello once more!
>
> Two more nitpicks:
>
> right now we install /usr/bin/ignitevisorcmd alternative, but it does't
> work since a) visor wants to write to /usr/share, and b) we moved it to
> examples for that.
> Please remove that alternative for now, create a ticket.
>
> rpm -ql apache-ignite | grep \\.java
> some yardstick and ml sources are there. I don't know who is there to blame
> - Ignite release process likely, not RPM building - but they shouldn't be
> here I guess?
>
> Regards!
>
> --
> Ilya Kasnacheev
>
> 2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
>
>> Hello again!
>>
>> I have reviewed the modified patch. All the things in my critical list
>> were fixed. I have just two minor remarks:
>>
>> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
>> works just as well from there as it was from our distribution. visor
>> doesn't, hence it stays in examples.
>> - I'm a bit confused that we no longer auto-start service after package is
>> installed. You have to do
>> sudo systemctl start apache-ignite@default-config.xml
>> manually. I'm used to packages that start the thing they install
>> immediately. Of course we can just document that clearly on downloads pages
>> so people won't get lost. What do you all think? Should Ignite auto-start
>> with default config? Is it possible to do so, but make possible to turn it
>> off.
>>
>> So from my side I recommend this pull request for inclusion after 1) is
>> done and 2) is discussed.
>>
>> Also, would be nice if you create additional JIRA tickets for issues in my
>> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
>> working) to be implemented in future releases.
>>
>> Regards,
>>
>> --
>> Ilya Kasnacheev
>>
>> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>>
>>> Removed replacement for default-config.xml.
>>> Also reimplemented service to be able to run multiple instances of Ignite.
>>>
>>> See updated PR [1].
>>>
>>>
>>> [1] https://github.com/apache/ignite/pull/3171 <
>>> https://github.com/apache/ignite/pull/3171>
>>>
>>>
>>>
>>>> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org> wrote:
>>>>
>>>> PDS and discovery settings are unrelated to the packaging task.
>>>> Andrey is right, just use existing default config.
>>>>
>>>> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>>> wrote:
>>>>
>>>>> Can we change default configuration file then?
>>>>> So that both binary and package deliveries contained PDS turned on by
>>>>> default?
>>>>>
>>>>> And what about Multicast Discovery?
>>>>>
>>>>>
>>>>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
>>>>> wrote:
>>>>>>
>>>>>> I agree with Andrey. The product should be identical regardless of how
>>>>> you
>>>>>> download and install it.
>>>>>>
>>>>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I think we should provide the same default configuration for all
>>>>>>> binary builds. From my point of view RPM/DEB/etc packages should use
>>>>>>> default configuration file like to standard binary release.
>>>>>>>
>>>>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>>>>> <il...@gmail.com> wrote:
>>>>>>>> Hello Igniters!
>>>>>>>>
>>>>>>>> What's your take on enabling PDS in 2.4 in default config? This way
>>> we
>>>>>>>> could enable it in RPM build with easy heart.
>>>>>>>>
>>>>>>>> The rest of your answers seem spot on. I'll happily review your
>>> amended
>>>>>>>> patch when it is ready (later this week?)
>>>>>>>>
>>>>>>>> --
>>>>>>>> Ilya Kasnacheev
>>>>>>>>
>>>>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>>>>> alongside
>>>>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
>>>>>>> usage of
>>>>>>>>>> tabs is desired in this case.
>>>>>>>>>
>>>>>>>>> My fault - got used to editing such files in default vim. Fixed and
>>>>>>> updated
>>>>>>>>> vim settings to indent with spaces.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> Maybe it's my CentOS but initially build will fail with the
>>> following
>>>>>>>>>> message, which I had to fix in spec
>>>>>>>>>> + cd 'apache-ignite-*'
>>>>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>>> file
>>>>> or
>>>>>>>>>> directory
>>>>>>>>>
>>>>>>>>> Strange error - shell expansion is not working. Anyway, replaced
>>> with
>>>>>>> more
>>>>>>>>> universal variant.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>>>>> default-config.xml) into build scripts. We should just copy over
>>>>>>>>>> default-config.xml from config/ to /etc, with dependencies if
>>> needed.
>>>>>>>>>
>>>>>>>>> Extracted corresponding sections into separate files.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>>>>> examples
>>>>>>>>>> but disable by default.
>>>>>>>>>
>>>>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>>>>> default
>>>>>>> so
>>>>>>>>> that Ignite users start working with PDS from the very beginning.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> I cannot comment on firewall rules validity, but firewall and
>>> service
>>>>>>>>>> configurations should be stored in sources, as files, supplied
>>> with
>>>>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>>>>
>>>>>>>>> Unfortunately, I did not find any other way to open ports and
>>>>> multicast,
>>>>>>>>> except applying direct iptables rules though firewall-cmd -- so
>>> there
>>>>>>> can
>>>>>>>>> be
>>>>>>>>> no separate service firewall rules file (as can be found here:
>>>>>>>>> /usr/lib/firewalld/services). Applied rules from package go
>>> straight
>>>>> to
>>>>>>>>> /etc/firewalld/direct.xml. If we agree not to put
>>> Multicast-enabled by
>>>>>>>>> default configuration file and will stick to IP Discovery, I'll
>>> try to
>>>>>>>>> revise firewall configuration and create separate service firewall
>>>>> rules
>>>>>>>>> file (but, I guess, much later).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
>>>>>>> should
>>>>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>>> to be
>>>>>>> in
>>>>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>>> fixed
>>>>>>> (a
>>>>>>>>>> symlink from /usr/share/apache-ignite/config to
>>> /etc/apache-ignite,
>>>>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>>>>
>>>>>>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>>>>>>>>> unaware-of-package design and I see the following 3-step way to
>>>>> overcome
>>>>>>>>> this limitation.
>>>>>>>>> At first we hide this executables file in examples (As Iliya
>>>>> proposed).
>>>>>>>>> Secondly, I can design a patch, which can be applied during package
>>>>>>> build,
>>>>>>>>> so that those executables work normally form the package
>>> installation.
>>>>>>>>> And lastly, redesign them to be universal and compatible with all
>>>>>>> delivery
>>>>>>>>> methods.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> Not everything should go into /usr/share. classpath libs belong to
>>>>>>>>>> /usr/lib.
>>>>>>>>>
>>>>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
>>>>>>> later
>>>>>>>>> we
>>>>>>>>> have to think out of a solution to load libs directly from /usr/lib
>>>>>>> (along
>>>>>>>>> with configurations / logs / work / etc.)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> We are building from binary package and not from sources. If it is
>>>>>>> used
>>>>>>>>> by
>>>>>>>>>> internal RPM building this is tolerable, but any distro or
>>> repository
>>>>>>>>> will
>>>>>>>>>> want to build from source.
>>>>>>>>>
>>>>>>>>> It's not an ordinary task to create "100% honest" package, so I
>>> guess
>>>>> we
>>>>>>>>> definitely won't make to 2.4 release. So I purpose include in the
>>>>>>> nearest
>>>>>>>>> release our package built from binary archive and do not supply
>>>>> sources
>>>>>>>>> package (as it is currently done in apache.org/dist for cassandra
>>> /
>>>>>>> hadoop
>>>>>>>>> or alike). And later design package build procedure, which includes
>>>>>>> getting
>>>>>>>>> sources from sources distribution, or even from Apache Ignite Git
>>>>>>>>> repository
>>>>>>>>> tag.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> spec contains version (2.4.0) - will be needed to be changed for
>>>>> every
>>>>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>>>>
>>>>>>>>> Release procedure on TC is already has some task for project
>>> version
>>>>>>>>> update,
>>>>>>>>> so I guess it is not so difficult to update it accordingly (as it
>>> is
>>>>>>>>> currently done for C++ / .Net internal versions).
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Ilya Kasnacheev wrote
>>>>>>>>>> Package description may be improved. service description lacks
>>> '-' in
>>>>>>> "In
>>>>>>>>>> Memory".
>>>>>>>>>
>>>>>>>>> Fixed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> So the only question to discuss (according Iliya's review) is
>>> whether
>>>>> to
>>>>>>>>> supply package with default-config.xml that has Multicast Discovery
>>>>>>> turned
>>>>>>>>> on or not?
>>>>>>>>>
>>>>>>>>> Also I'll appreciate any other comments concerning current package
>>>>>>> design.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>>
>>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello once more!
Two more nitpicks:
right now we install /usr/bin/ignitevisorcmd alternative, but it does't
work since a) visor wants to write to /usr/share, and b) we moved it to
examples for that.
Please remove that alternative for now, create a ticket.
rpm -ql apache-ignite | grep \\.java
some yardstick and ml sources are there. I don't know who is there to blame
- Ignite release process likely, not RPM building - but they shouldn't be
here I guess?
Regards!
--
Ilya Kasnacheev
2017-12-28 14:21 GMT+03:00 Ilya Kasnacheev <il...@gmail.com>:
> Hello again!
>
> I have reviewed the modified patch. All the things in my critical list
> were fixed. I have just two minor remarks:
>
> - Please move sqlline.sh back from examples to /usr/share/a-i/bin. It
> works just as well from there as it was from our distribution. visor
> doesn't, hence it stays in examples.
> - I'm a bit confused that we no longer auto-start service after package is
> installed. You have to do
> sudo systemctl start apache-ignite@default-config.xml
> manually. I'm used to packages that start the thing they install
> immediately. Of course we can just document that clearly on downloads pages
> so people won't get lost. What do you all think? Should Ignite auto-start
> with default config? Is it possible to do so, but make possible to turn it
> off.
>
> So from my side I recommend this pull request for inclusion after 1) is
> done and 2) is discussed.
>
> Also, would be nice if you create additional JIRA tickets for issues in my
> minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
> working) to be implemented in future releases.
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
>
>> Removed replacement for default-config.xml.
>> Also reimplemented service to be able to run multiple instances of Ignite.
>>
>> See updated PR [1].
>>
>>
>> [1] https://github.com/apache/ignite/pull/3171 <
>> https://github.com/apache/ignite/pull/3171>
>>
>>
>>
>> > On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org> wrote:
>> >
>> > PDS and discovery settings are unrelated to the packaging task.
>> > Andrey is right, just use existing default config.
>> >
>> > On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
>> wrote:
>> >
>> >> Can we change default configuration file then?
>> >> So that both binary and package deliveries contained PDS turned on by
>> >> default?
>> >>
>> >> And what about Multicast Discovery?
>> >>
>> >>
>> >>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
>> >> wrote:
>> >>>
>> >>> I agree with Andrey. The product should be identical regardless of how
>> >> you
>> >>> download and install it.
>> >>>
>> >>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org>
>> wrote:
>> >>>
>> >>>> Hi,
>> >>>>
>> >>>> I think we should provide the same default configuration for all
>> >>>> binary builds. From my point of view RPM/DEB/etc packages should use
>> >>>> default configuration file like to standard binary release.
>> >>>>
>> >>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>> >>>> <il...@gmail.com> wrote:
>> >>>>> Hello Igniters!
>> >>>>>
>> >>>>> What's your take on enabling PDS in 2.4 in default config? This way
>> we
>> >>>>> could enable it in RPM build with easy heart.
>> >>>>>
>> >>>>> The rest of your answers seem spot on. I'll happily review your
>> amended
>> >>>>> patch when it is ready (later this week?)
>> >>>>>
>> >>>>> --
>> >>>>> Ilya Kasnacheev
>> >>>>>
>> >>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>> >>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>> >>>> alongside
>> >>>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
>> >>>> usage of
>> >>>>>>> tabs is desired in this case.
>> >>>>>>
>> >>>>>> My fault - got used to editing such files in default vim. Fixed and
>> >>>> updated
>> >>>>>> vim settings to indent with spaces.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> Maybe it's my CentOS but initially build will fail with the
>> following
>> >>>>>>> message, which I had to fix in spec
>> >>>>>>> + cd 'apache-ignite-*'
>> >>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such
>> file
>> >> or
>> >>>>>>> directory
>> >>>>>>
>> >>>>>> Strange error - shell expansion is not working. Anyway, replaced
>> with
>> >>>> more
>> >>>>>> universal variant.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> We absolutely should not inline configuration files (such as
>> >>>>>>> default-config.xml) into build scripts. We should just copy over
>> >>>>>>> default-config.xml from config/ to /etc, with dependencies if
>> needed.
>> >>>>>>
>> >>>>>> Extracted corresponding sections into separate files.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>> >>>> examples
>> >>>>>>> but disable by default.
>> >>>>>>
>> >>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>> >> default
>> >>>> so
>> >>>>>> that Ignite users start working with PDS from the very beginning.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> I cannot comment on firewall rules validity, but firewall and
>> service
>> >>>>>>> configurations should be stored in sources, as files, supplied
>> with
>> >>>>>>> release (somewhere in packages/) and installed from there
>> >>>>>>
>> >>>>>> Unfortunately, I did not find any other way to open ports and
>> >> multicast,
>> >>>>>> except applying direct iptables rules though firewall-cmd -- so
>> there
>> >>>> can
>> >>>>>> be
>> >>>>>> no separate service firewall rules file (as can be found here:
>> >>>>>> /usr/lib/firewalld/services). Applied rules from package go
>> straight
>> >> to
>> >>>>>> /etc/firewalld/direct.xml. If we agree not to put
>> Multicast-enabled by
>> >>>>>> default configuration file and will stick to IP Discovery, I'll
>> try to
>> >>>>>> revise firewall configuration and create separate service firewall
>> >> rules
>> >>>>>> file (but, I guess, much later).
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>> >>>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
>> >>>> should
>> >>>>>>> become available as /usr/bin/apache-ignite-sqlline for example,
>> to be
>> >>>> in
>> >>>>>>> $PATH. And figure out how to connect by itself.
>> >>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>> >>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>> >>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
>> fixed
>> >>>> (a
>> >>>>>>> symlink from /usr/share/apache-ignite/config to
>> /etc/apache-ignite,
>> >>>>>>> perhaps, and work dir in /tmp?), and made available as
>> >>>>>>> /usr/bin/apache-ignite-visorcmd.
>> >>>>>>
>> >>>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>> >>>>>> unaware-of-package design and I see the following 3-step way to
>> >> overcome
>> >>>>>> this limitation.
>> >>>>>> At first we hide this executables file in examples (As Iliya
>> >> proposed).
>> >>>>>> Secondly, I can design a patch, which can be applied during package
>> >>>> build,
>> >>>>>> so that those executables work normally form the package
>> installation.
>> >>>>>> And lastly, redesign them to be universal and compatible with all
>> >>>> delivery
>> >>>>>> methods.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> Not everything should go into /usr/share. classpath libs belong to
>> >>>>>>> /usr/lib.
>> >>>>>>
>> >>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>> >>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
>> >>>> later
>> >>>>>> we
>> >>>>>> have to think out of a solution to load libs directly from /usr/lib
>> >>>> (along
>> >>>>>> with configurations / logs / work / etc.)
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> We are building from binary package and not from sources. If it is
>> >>>> used
>> >>>>>> by
>> >>>>>>> internal RPM building this is tolerable, but any distro or
>> repository
>> >>>>>> will
>> >>>>>>> want to build from source.
>> >>>>>>
>> >>>>>> It's not an ordinary task to create "100% honest" package, so I
>> guess
>> >> we
>> >>>>>> definitely won't make to 2.4 release. So I purpose include in the
>> >>>> nearest
>> >>>>>> release our package built from binary archive and do not supply
>> >> sources
>> >>>>>> package (as it is currently done in apache.org/dist for cassandra
>> /
>> >>>> hadoop
>> >>>>>> or alike). And later design package build procedure, which includes
>> >>>> getting
>> >>>>>> sources from sources distribution, or even from Apache Ignite Git
>> >>>>>> repository
>> >>>>>> tag.
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> spec contains version (2.4.0) - will be needed to be changed for
>> >> every
>> >>>>>>> release. I'm not sure if it's possible to extract.
>> >>>>>>
>> >>>>>> Release procedure on TC is already has some task for project
>> version
>> >>>>>> update,
>> >>>>>> so I guess it is not so difficult to update it accordingly (as it
>> is
>> >>>>>> currently done for C++ / .Net internal versions).
>> >>>>>>
>> >>>>>>
>> >>>>>> Ilya Kasnacheev wrote
>> >>>>>>> Package description may be improved. service description lacks
>> '-' in
>> >>>> "In
>> >>>>>>> Memory".
>> >>>>>>
>> >>>>>> Fixed.
>> >>>>>>
>> >>>>>>
>> >>>>>> So the only question to discuss (according Iliya's review) is
>> whether
>> >> to
>> >>>>>> supply package with default-config.xml that has Multicast Discovery
>> >>>> turned
>> >>>>>> on or not?
>> >>>>>>
>> >>>>>> Also I'll appreciate any other comments concerning current package
>> >>>> design.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>> >>>>>>
>> >>>>
>> >>
>> >>
>>
>>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello again!
I have reviewed the modified patch. All the things in my critical list were
fixed. I have just two minor remarks:
- Please move sqlline.sh back from examples to /usr/share/a-i/bin. It works
just as well from there as it was from our distribution. visor doesn't,
hence it stays in examples.
- I'm a bit confused that we no longer auto-start service after package is
installed. You have to do
sudo systemctl start apache-ignite@default-config.xml
manually. I'm used to packages that start the thing they install
immediately. Of course we can just document that clearly on downloads pages
so people won't get lost. What do you all think? Should Ignite auto-start
with default config? Is it possible to do so, but make possible to turn it
off.
So from my side I recommend this pull request for inclusion after 1) is
done and 2) is discussed.
Also, would be nice if you create additional JIRA tickets for issues in my
minor list (ignitesqlline in /usr/bin, ignitevisorcmd in /usr/bin and
working) to be implemented in future releases.
Regards,
--
Ilya Kasnacheev
2017-12-26 15:25 GMT+03:00 Petr Ivanov <mr...@gmail.com>:
> Removed replacement for default-config.xml.
> Also reimplemented service to be able to run multiple instances of Ignite.
>
> See updated PR [1].
>
>
> [1] https://github.com/apache/ignite/pull/3171 <https://github.com/apache/
> ignite/pull/3171>
>
>
>
> > On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org> wrote:
> >
> > PDS and discovery settings are unrelated to the packaging task.
> > Andrey is right, just use existing default config.
> >
> > On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com>
> wrote:
> >
> >> Can we change default configuration file then?
> >> So that both binary and package deliveries contained PDS turned on by
> >> default?
> >>
> >> And what about Multicast Discovery?
> >>
> >>
> >>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
> >> wrote:
> >>>
> >>> I agree with Andrey. The product should be identical regardless of how
> >> you
> >>> download and install it.
> >>>
> >>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org> wrote:
> >>>
> >>>> Hi,
> >>>>
> >>>> I think we should provide the same default configuration for all
> >>>> binary builds. From my point of view RPM/DEB/etc packages should use
> >>>> default configuration file like to standard binary release.
> >>>>
> >>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
> >>>> <il...@gmail.com> wrote:
> >>>>> Hello Igniters!
> >>>>>
> >>>>> What's your take on enabling PDS in 2.4 in default config? This way
> we
> >>>>> could enable it in RPM build with easy heart.
> >>>>>
> >>>>> The rest of your answers seem spot on. I'll happily review your
> amended
> >>>>> patch when it is ready (later this week?)
> >>>>>
> >>>>> --
> >>>>> Ilya Kasnacheev
> >>>>>
> >>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
> >>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
> >>>> alongside
> >>>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
> >>>> usage of
> >>>>>>> tabs is desired in this case.
> >>>>>>
> >>>>>> My fault - got used to editing such files in default vim. Fixed and
> >>>> updated
> >>>>>> vim settings to indent with spaces.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> Maybe it's my CentOS but initially build will fail with the
> following
> >>>>>>> message, which I had to fix in spec
> >>>>>>> + cd 'apache-ignite-*'
> >>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file
> >> or
> >>>>>>> directory
> >>>>>>
> >>>>>> Strange error - shell expansion is not working. Anyway, replaced
> with
> >>>> more
> >>>>>> universal variant.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> We absolutely should not inline configuration files (such as
> >>>>>>> default-config.xml) into build scripts. We should just copy over
> >>>>>>> default-config.xml from config/ to /etc, with dependencies if
> needed.
> >>>>>>
> >>>>>> Extracted corresponding sections into separate files.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> I'm not sure PDS should be ON by default. Better provide it in
> >>>> examples
> >>>>>>> but disable by default.
> >>>>>>
> >>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
> >> default
> >>>> so
> >>>>>> that Ignite users start working with PDS from the very beginning.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> I cannot comment on firewall rules validity, but firewall and
> service
> >>>>>>> configurations should be stored in sources, as files, supplied with
> >>>>>>> release (somewhere in packages/) and installed from there
> >>>>>>
> >>>>>> Unfortunately, I did not find any other way to open ports and
> >> multicast,
> >>>>>> except applying direct iptables rules though firewall-cmd -- so
> there
> >>>> can
> >>>>>> be
> >>>>>> no separate service firewall rules file (as can be found here:
> >>>>>> /usr/lib/firewalld/services). Applied rules from package go straight
> >> to
> >>>>>> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled
> by
> >>>>>> default configuration file and will stick to IP Discovery, I'll try
> to
> >>>>>> revise firewall configuration and create separate service firewall
> >> rules
> >>>>>> file (but, I guess, much later).
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> * sqlline.sh fails to connect initially, needs to be specified
> >>>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
> >>>> should
> >>>>>>> become available as /usr/bin/apache-ignite-sqlline for example, to
> be
> >>>> in
> >>>>>>> $PATH. And figure out how to connect by itself.
> >>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
> >>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
> >>>>>>> /usr/share/apache-ignite/work. We should consider how it can be
> fixed
> >>>> (a
> >>>>>>> symlink from /usr/share/apache-ignite/config to
> /etc/apache-ignite,
> >>>>>>> perhaps, and work dir in /tmp?), and made available as
> >>>>>>> /usr/bin/apache-ignite-visorcmd.
> >>>>>>
> >>>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
> >>>>>> unaware-of-package design and I see the following 3-step way to
> >> overcome
> >>>>>> this limitation.
> >>>>>> At first we hide this executables file in examples (As Iliya
> >> proposed).
> >>>>>> Secondly, I can design a patch, which can be applied during package
> >>>> build,
> >>>>>> so that those executables work normally form the package
> installation.
> >>>>>> And lastly, redesign them to be universal and compatible with all
> >>>> delivery
> >>>>>> methods.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> Not everything should go into /usr/share. classpath libs belong to
> >>>>>>> /usr/lib.
> >>>>>>
> >>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
> >>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
> >>>> later
> >>>>>> we
> >>>>>> have to think out of a solution to load libs directly from /usr/lib
> >>>> (along
> >>>>>> with configurations / logs / work / etc.)
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> We are building from binary package and not from sources. If it is
> >>>> used
> >>>>>> by
> >>>>>>> internal RPM building this is tolerable, but any distro or
> repository
> >>>>>> will
> >>>>>>> want to build from source.
> >>>>>>
> >>>>>> It's not an ordinary task to create "100% honest" package, so I
> guess
> >> we
> >>>>>> definitely won't make to 2.4 release. So I purpose include in the
> >>>> nearest
> >>>>>> release our package built from binary archive and do not supply
> >> sources
> >>>>>> package (as it is currently done in apache.org/dist for cassandra /
> >>>> hadoop
> >>>>>> or alike). And later design package build procedure, which includes
> >>>> getting
> >>>>>> sources from sources distribution, or even from Apache Ignite Git
> >>>>>> repository
> >>>>>> tag.
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> spec contains version (2.4.0) - will be needed to be changed for
> >> every
> >>>>>>> release. I'm not sure if it's possible to extract.
> >>>>>>
> >>>>>> Release procedure on TC is already has some task for project version
> >>>>>> update,
> >>>>>> so I guess it is not so difficult to update it accordingly (as it is
> >>>>>> currently done for C++ / .Net internal versions).
> >>>>>>
> >>>>>>
> >>>>>> Ilya Kasnacheev wrote
> >>>>>>> Package description may be improved. service description lacks '-'
> in
> >>>> "In
> >>>>>>> Memory".
> >>>>>>
> >>>>>> Fixed.
> >>>>>>
> >>>>>>
> >>>>>> So the only question to discuss (according Iliya's review) is
> whether
> >> to
> >>>>>> supply package with default-config.xml that has Multicast Discovery
> >>>> turned
> >>>>>> on or not?
> >>>>>>
> >>>>>> Also I'll appreciate any other comments concerning current package
> >>>> design.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>>>>>
> >>>>
> >>
> >>
>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Removed replacement for default-config.xml.
Also reimplemented service to be able to run multiple instances of Ignite.
See updated PR [1].
[1] https://github.com/apache/ignite/pull/3171 <https://github.com/apache/ignite/pull/3171>
> On 25 Dec 2017, at 18:57, Pavel Tupitsyn <pt...@apache.org> wrote:
>
> PDS and discovery settings are unrelated to the packaging task.
> Andrey is right, just use existing default config.
>
> On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com> wrote:
>
>> Can we change default configuration file then?
>> So that both binary and package deliveries contained PDS turned on by
>> default?
>>
>> And what about Multicast Discovery?
>>
>>
>>> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>>
>>> I agree with Andrey. The product should be identical regardless of how
>> you
>>> download and install it.
>>>
>>> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> I think we should provide the same default configuration for all
>>>> binary builds. From my point of view RPM/DEB/etc packages should use
>>>> default configuration file like to standard binary release.
>>>>
>>>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>>>> <il...@gmail.com> wrote:
>>>>> Hello Igniters!
>>>>>
>>>>> What's your take on enabling PDS in 2.4 in default config? This way we
>>>>> could enable it in RPM build with easy heart.
>>>>>
>>>>> The rest of your answers seem spot on. I'll happily review your amended
>>>>> patch when it is ready (later this week?)
>>>>>
>>>>> --
>>>>> Ilya Kasnacheev
>>>>>
>>>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>>>> alongside
>>>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
>>>> usage of
>>>>>>> tabs is desired in this case.
>>>>>>
>>>>>> My fault - got used to editing such files in default vim. Fixed and
>>>> updated
>>>>>> vim settings to indent with spaces.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> Maybe it's my CentOS but initially build will fail with the following
>>>>>>> message, which I had to fix in spec
>>>>>>> + cd 'apache-ignite-*'
>>>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file
>> or
>>>>>>> directory
>>>>>>
>>>>>> Strange error - shell expansion is not working. Anyway, replaced with
>>>> more
>>>>>> universal variant.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> We absolutely should not inline configuration files (such as
>>>>>>> default-config.xml) into build scripts. We should just copy over
>>>>>>> default-config.xml from config/ to /etc, with dependencies if needed.
>>>>>>
>>>>>> Extracted corresponding sections into separate files.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> I'm not sure PDS should be ON by default. Better provide it in
>>>> examples
>>>>>>> but disable by default.
>>>>>>
>>>>>> AFAIK, PDS is actively pushing forward and has to be enabled by
>> default
>>>> so
>>>>>> that Ignite users start working with PDS from the very beginning.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> I cannot comment on firewall rules validity, but firewall and service
>>>>>>> configurations should be stored in sources, as files, supplied with
>>>>>>> release (somewhere in packages/) and installed from there
>>>>>>
>>>>>> Unfortunately, I did not find any other way to open ports and
>> multicast,
>>>>>> except applying direct iptables rules though firewall-cmd -- so there
>>>> can
>>>>>> be
>>>>>> no separate service firewall rules file (as can be found here:
>>>>>> /usr/lib/firewalld/services). Applied rules from package go straight
>> to
>>>>>> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
>>>>>> default configuration file and will stick to IP Discovery, I'll try to
>>>>>> revise firewall configuration and create separate service firewall
>> rules
>>>>>> file (but, I guess, much later).
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
>>>> should
>>>>>>> become available as /usr/bin/apache-ignite-sqlline for example, to be
>>>> in
>>>>>>> $PATH. And figure out how to connect by itself.
>>>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>>>> /usr/share/apache-ignite/work. We should consider how it can be fixed
>>>> (a
>>>>>>> symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
>>>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>>>
>>>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>>>>>> unaware-of-package design and I see the following 3-step way to
>> overcome
>>>>>> this limitation.
>>>>>> At first we hide this executables file in examples (As Iliya
>> proposed).
>>>>>> Secondly, I can design a patch, which can be applied during package
>>>> build,
>>>>>> so that those executables work normally form the package installation.
>>>>>> And lastly, redesign them to be universal and compatible with all
>>>> delivery
>>>>>> methods.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> Not everything should go into /usr/share. classpath libs belong to
>>>>>>> /usr/lib.
>>>>>>
>>>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
>>>> later
>>>>>> we
>>>>>> have to think out of a solution to load libs directly from /usr/lib
>>>> (along
>>>>>> with configurations / logs / work / etc.)
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> We are building from binary package and not from sources. If it is
>>>> used
>>>>>> by
>>>>>>> internal RPM building this is tolerable, but any distro or repository
>>>>>> will
>>>>>>> want to build from source.
>>>>>>
>>>>>> It's not an ordinary task to create "100% honest" package, so I guess
>> we
>>>>>> definitely won't make to 2.4 release. So I purpose include in the
>>>> nearest
>>>>>> release our package built from binary archive and do not supply
>> sources
>>>>>> package (as it is currently done in apache.org/dist for cassandra /
>>>> hadoop
>>>>>> or alike). And later design package build procedure, which includes
>>>> getting
>>>>>> sources from sources distribution, or even from Apache Ignite Git
>>>>>> repository
>>>>>> tag.
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> spec contains version (2.4.0) - will be needed to be changed for
>> every
>>>>>>> release. I'm not sure if it's possible to extract.
>>>>>>
>>>>>> Release procedure on TC is already has some task for project version
>>>>>> update,
>>>>>> so I guess it is not so difficult to update it accordingly (as it is
>>>>>> currently done for C++ / .Net internal versions).
>>>>>>
>>>>>>
>>>>>> Ilya Kasnacheev wrote
>>>>>>> Package description may be improved. service description lacks '-' in
>>>> "In
>>>>>>> Memory".
>>>>>>
>>>>>> Fixed.
>>>>>>
>>>>>>
>>>>>> So the only question to discuss (according Iliya's review) is whether
>> to
>>>>>> supply package with default-config.xml that has Multicast Discovery
>>>> turned
>>>>>> on or not?
>>>>>>
>>>>>> Also I'll appreciate any other comments concerning current package
>>>> design.
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>>
>>>>
>>
>>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Pavel Tupitsyn <pt...@apache.org>.
PDS and discovery settings are unrelated to the packaging task.
Andrey is right, just use existing default config.
On Mon, Dec 25, 2017 at 6:51 PM, Petr Ivanov <mr...@gmail.com> wrote:
> Can we change default configuration file then?
> So that both binary and package deliveries contained PDS turned on by
> default?
>
> And what about Multicast Discovery?
>
>
> > On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > I agree with Andrey. The product should be identical regardless of how
> you
> > download and install it.
> >
> > On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org> wrote:
> >
> >> Hi,
> >>
> >> I think we should provide the same default configuration for all
> >> binary builds. From my point of view RPM/DEB/etc packages should use
> >> default configuration file like to standard binary release.
> >>
> >> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
> >> <il...@gmail.com> wrote:
> >>> Hello Igniters!
> >>>
> >>> What's your take on enabling PDS in 2.4 in default config? This way we
> >>> could enable it in RPM build with easy heart.
> >>>
> >>> The rest of your answers seem spot on. I'll happily review your amended
> >>> patch when it is ready (later this week?)
> >>>
> >>> --
> >>> Ilya Kasnacheev
> >>>
> >>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
> >>>
> >>>> Ilya Kasnacheev wrote
> >>>>> I have noticed that both spec and DEVNOTES are made to use tabs
> >> alongside
> >>>>> with spaces. I thought we are avoiding tabs? Please confirm that
> >> usage of
> >>>>> tabs is desired in this case.
> >>>>
> >>>> My fault - got used to editing such files in default vim. Fixed and
> >> updated
> >>>> vim settings to indent with spaces.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> Maybe it's my CentOS but initially build will fail with the following
> >>>>> message, which I had to fix in spec
> >>>>> + cd 'apache-ignite-*'
> >>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file
> or
> >>>>> directory
> >>>>
> >>>> Strange error - shell expansion is not working. Anyway, replaced with
> >> more
> >>>> universal variant.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> We absolutely should not inline configuration files (such as
> >>>>> default-config.xml) into build scripts. We should just copy over
> >>>>> default-config.xml from config/ to /etc, with dependencies if needed.
> >>>>
> >>>> Extracted corresponding sections into separate files.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> I'm not sure PDS should be ON by default. Better provide it in
> >> examples
> >>>>> but disable by default.
> >>>>
> >>>> AFAIK, PDS is actively pushing forward and has to be enabled by
> default
> >> so
> >>>> that Ignite users start working with PDS from the very beginning.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> I cannot comment on firewall rules validity, but firewall and service
> >>>>> configurations should be stored in sources, as files, supplied with
> >>>>> release (somewhere in packages/) and installed from there
> >>>>
> >>>> Unfortunately, I did not find any other way to open ports and
> multicast,
> >>>> except applying direct iptables rules though firewall-cmd -- so there
> >> can
> >>>> be
> >>>> no separate service firewall rules file (as can be found here:
> >>>> /usr/lib/firewalld/services). Applied rules from package go straight
> to
> >>>> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
> >>>> default configuration file and will stick to IP Discovery, I'll try to
> >>>> revise firewall configuration and create separate service firewall
> rules
> >>>> file (but, I guess, much later).
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> * sqlline.sh fails to connect initially, needs to be specified
> >>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
> >> should
> >>>>> become available as /usr/bin/apache-ignite-sqlline for example, to be
> >> in
> >>>>> $PATH. And figure out how to connect by itself.
> >>>>> * ignitevisorcmd.sh becomes confused. first it scans
> >>>>> /usr/share/apache-ignite for configs, then it fails to write to
> >>>>> /usr/share/apache-ignite/work. We should consider how it can be fixed
> >> (a
> >>>>> symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
> >>>>> perhaps, and work dir in /tmp?), and made available as
> >>>>> /usr/bin/apache-ignite-visorcmd.
> >>>>
> >>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
> >>>> unaware-of-package design and I see the following 3-step way to
> overcome
> >>>> this limitation.
> >>>> At first we hide this executables file in examples (As Iliya
> proposed).
> >>>> Secondly, I can design a patch, which can be applied during package
> >> build,
> >>>> so that those executables work normally form the package installation.
> >>>> And lastly, redesign them to be universal and compatible with all
> >> delivery
> >>>> methods.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> Not everything should go into /usr/share. classpath libs belong to
> >>>>> /usr/lib.
> >>>>
> >>>> Moved libs to /usr/lib/apache-ignite and mapped to
> >>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
> >> later
> >>>> we
> >>>> have to think out of a solution to load libs directly from /usr/lib
> >> (along
> >>>> with configurations / logs / work / etc.)
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> We are building from binary package and not from sources. If it is
> >> used
> >>>> by
> >>>>> internal RPM building this is tolerable, but any distro or repository
> >>>> will
> >>>>> want to build from source.
> >>>>
> >>>> It's not an ordinary task to create "100% honest" package, so I guess
> we
> >>>> definitely won't make to 2.4 release. So I purpose include in the
> >> nearest
> >>>> release our package built from binary archive and do not supply
> sources
> >>>> package (as it is currently done in apache.org/dist for cassandra /
> >> hadoop
> >>>> or alike). And later design package build procedure, which includes
> >> getting
> >>>> sources from sources distribution, or even from Apache Ignite Git
> >>>> repository
> >>>> tag.
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> spec contains version (2.4.0) - will be needed to be changed for
> every
> >>>>> release. I'm not sure if it's possible to extract.
> >>>>
> >>>> Release procedure on TC is already has some task for project version
> >>>> update,
> >>>> so I guess it is not so difficult to update it accordingly (as it is
> >>>> currently done for C++ / .Net internal versions).
> >>>>
> >>>>
> >>>> Ilya Kasnacheev wrote
> >>>>> Package description may be improved. service description lacks '-' in
> >> "In
> >>>>> Memory".
> >>>>
> >>>> Fixed.
> >>>>
> >>>>
> >>>> So the only question to discuss (according Iliya's review) is whether
> to
> >>>> supply package with default-config.xml that has Multicast Discovery
> >> turned
> >>>> on or not?
> >>>>
> >>>> Also I'll appreciate any other comments concerning current package
> >> design.
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>>>
> >>
>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Petr Ivanov <mr...@gmail.com>.
Can we change default configuration file then?
So that both binary and package deliveries contained PDS turned on by default?
And what about Multicast Discovery?
> On 25 Dec 2017, at 18:41, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> I agree with Andrey. The product should be identical regardless of how you
> download and install it.
>
> On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org> wrote:
>
>> Hi,
>>
>> I think we should provide the same default configuration for all
>> binary builds. From my point of view RPM/DEB/etc packages should use
>> default configuration file like to standard binary release.
>>
>> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
>> <il...@gmail.com> wrote:
>>> Hello Igniters!
>>>
>>> What's your take on enabling PDS in 2.4 in default config? This way we
>>> could enable it in RPM build with easy heart.
>>>
>>> The rest of your answers seem spot on. I'll happily review your amended
>>> patch when it is ready (later this week?)
>>>
>>> --
>>> Ilya Kasnacheev
>>>
>>> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>>>
>>>> Ilya Kasnacheev wrote
>>>>> I have noticed that both spec and DEVNOTES are made to use tabs
>> alongside
>>>>> with spaces. I thought we are avoiding tabs? Please confirm that
>> usage of
>>>>> tabs is desired in this case.
>>>>
>>>> My fault - got used to editing such files in default vim. Fixed and
>> updated
>>>> vim settings to indent with spaces.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> Maybe it's my CentOS but initially build will fail with the following
>>>>> message, which I had to fix in spec
>>>>> + cd 'apache-ignite-*'
>>>>> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file or
>>>>> directory
>>>>
>>>> Strange error - shell expansion is not working. Anyway, replaced with
>> more
>>>> universal variant.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> We absolutely should not inline configuration files (such as
>>>>> default-config.xml) into build scripts. We should just copy over
>>>>> default-config.xml from config/ to /etc, with dependencies if needed.
>>>>
>>>> Extracted corresponding sections into separate files.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> I'm not sure PDS should be ON by default. Better provide it in
>> examples
>>>>> but disable by default.
>>>>
>>>> AFAIK, PDS is actively pushing forward and has to be enabled by default
>> so
>>>> that Ignite users start working with PDS from the very beginning.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> I cannot comment on firewall rules validity, but firewall and service
>>>>> configurations should be stored in sources, as files, supplied with
>>>>> release (somewhere in packages/) and installed from there
>>>>
>>>> Unfortunately, I did not find any other way to open ports and multicast,
>>>> except applying direct iptables rules though firewall-cmd -- so there
>> can
>>>> be
>>>> no separate service firewall rules file (as can be found here:
>>>> /usr/lib/firewalld/services). Applied rules from package go straight to
>>>> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
>>>> default configuration file and will stick to IP Discovery, I'll try to
>>>> revise firewall configuration and create separate service firewall rules
>>>> file (but, I guess, much later).
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> * sqlline.sh fails to connect initially, needs to be specified
>>>>> jdbc:ignite:thin://localhost, then it works. I think that sqlline
>> should
>>>>> become available as /usr/bin/apache-ignite-sqlline for example, to be
>> in
>>>>> $PATH. And figure out how to connect by itself.
>>>>> * ignitevisorcmd.sh becomes confused. first it scans
>>>>> /usr/share/apache-ignite for configs, then it fails to write to
>>>>> /usr/share/apache-ignite/work. We should consider how it can be fixed
>> (a
>>>>> symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
>>>>> perhaps, and work dir in /tmp?), and made available as
>>>>> /usr/bin/apache-ignite-visorcmd.
>>>>
>>>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>>>> unaware-of-package design and I see the following 3-step way to overcome
>>>> this limitation.
>>>> At first we hide this executables file in examples (As Iliya proposed).
>>>> Secondly, I can design a patch, which can be applied during package
>> build,
>>>> so that those executables work normally form the package installation.
>>>> And lastly, redesign them to be universal and compatible with all
>> delivery
>>>> methods.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> Not everything should go into /usr/share. classpath libs belong to
>>>>> /usr/lib.
>>>>
>>>> Moved libs to /usr/lib/apache-ignite and mapped to
>>>> /usr/share/apache-ignite/libs (for backward compatibility). I guess
>> later
>>>> we
>>>> have to think out of a solution to load libs directly from /usr/lib
>> (along
>>>> with configurations / logs / work / etc.)
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> We are building from binary package and not from sources. If it is
>> used
>>>> by
>>>>> internal RPM building this is tolerable, but any distro or repository
>>>> will
>>>>> want to build from source.
>>>>
>>>> It's not an ordinary task to create "100% honest" package, so I guess we
>>>> definitely won't make to 2.4 release. So I purpose include in the
>> nearest
>>>> release our package built from binary archive and do not supply sources
>>>> package (as it is currently done in apache.org/dist for cassandra /
>> hadoop
>>>> or alike). And later design package build procedure, which includes
>> getting
>>>> sources from sources distribution, or even from Apache Ignite Git
>>>> repository
>>>> tag.
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> spec contains version (2.4.0) - will be needed to be changed for every
>>>>> release. I'm not sure if it's possible to extract.
>>>>
>>>> Release procedure on TC is already has some task for project version
>>>> update,
>>>> so I guess it is not so difficult to update it accordingly (as it is
>>>> currently done for C++ / .Net internal versions).
>>>>
>>>>
>>>> Ilya Kasnacheev wrote
>>>>> Package description may be improved. service description lacks '-' in
>> "In
>>>>> Memory".
>>>>
>>>> Fixed.
>>>>
>>>>
>>>> So the only question to discuss (according Iliya's review) is whether to
>>>> supply package with default-config.xml that has Multicast Discovery
>> turned
>>>> on or not?
>>>>
>>>> Also I'll appreciate any other comments concerning current package
>> design.
>>>>
>>>>
>>>>
>>>> --
>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>
>>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Dmitriy Setrakyan <ds...@apache.org>.
I agree with Andrey. The product should be identical regardless of how you
download and install it.
On Mon, Dec 25, 2017 at 7:18 AM, Andrey Gura <ag...@apache.org> wrote:
> Hi,
>
> I think we should provide the same default configuration for all
> binary builds. From my point of view RPM/DEB/etc packages should use
> default configuration file like to standard binary release.
>
> On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
> <il...@gmail.com> wrote:
> > Hello Igniters!
> >
> > What's your take on enabling PDS in 2.4 in default config? This way we
> > could enable it in RPM build with easy heart.
> >
> > The rest of your answers seem spot on. I'll happily review your amended
> > patch when it is ready (later this week?)
> >
> > --
> > Ilya Kasnacheev
> >
> > 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
> >
> >> Ilya Kasnacheev wrote
> >> > I have noticed that both spec and DEVNOTES are made to use tabs
> alongside
> >> > with spaces. I thought we are avoiding tabs? Please confirm that
> usage of
> >> > tabs is desired in this case.
> >>
> >> My fault - got used to editing such files in default vim. Fixed and
> updated
> >> vim settings to indent with spaces.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > Maybe it's my CentOS but initially build will fail with the following
> >> > message, which I had to fix in spec
> >> > + cd 'apache-ignite-*'
> >> > /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file or
> >> > directory
> >>
> >> Strange error - shell expansion is not working. Anyway, replaced with
> more
> >> universal variant.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > We absolutely should not inline configuration files (such as
> >> > default-config.xml) into build scripts. We should just copy over
> >> > default-config.xml from config/ to /etc, with dependencies if needed.
> >>
> >> Extracted corresponding sections into separate files.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > I'm not sure PDS should be ON by default. Better provide it in
> examples
> >> > but disable by default.
> >>
> >> AFAIK, PDS is actively pushing forward and has to be enabled by default
> so
> >> that Ignite users start working with PDS from the very beginning.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > I cannot comment on firewall rules validity, but firewall and service
> >> > configurations should be stored in sources, as files, supplied with
> >> > release (somewhere in packages/) and installed from there
> >>
> >> Unfortunately, I did not find any other way to open ports and multicast,
> >> except applying direct iptables rules though firewall-cmd -- so there
> can
> >> be
> >> no separate service firewall rules file (as can be found here:
> >> /usr/lib/firewalld/services). Applied rules from package go straight to
> >> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
> >> default configuration file and will stick to IP Discovery, I'll try to
> >> revise firewall configuration and create separate service firewall rules
> >> file (but, I guess, much later).
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > * sqlline.sh fails to connect initially, needs to be specified
> >> > jdbc:ignite:thin://localhost, then it works. I think that sqlline
> should
> >> > become available as /usr/bin/apache-ignite-sqlline for example, to be
> in
> >> > $PATH. And figure out how to connect by itself.
> >> > * ignitevisorcmd.sh becomes confused. first it scans
> >> > /usr/share/apache-ignite for configs, then it fails to write to
> >> > /usr/share/apache-ignite/work. We should consider how it can be fixed
> (a
> >> > symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
> >> > perhaps, and work dir in /tmp?), and made available as
> >> > /usr/bin/apache-ignite-visorcmd.
> >>
> >> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
> >> unaware-of-package design and I see the following 3-step way to overcome
> >> this limitation.
> >> At first we hide this executables file in examples (As Iliya proposed).
> >> Secondly, I can design a patch, which can be applied during package
> build,
> >> so that those executables work normally form the package installation.
> >> And lastly, redesign them to be universal and compatible with all
> delivery
> >> methods.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > Not everything should go into /usr/share. classpath libs belong to
> >> > /usr/lib.
> >>
> >> Moved libs to /usr/lib/apache-ignite and mapped to
> >> /usr/share/apache-ignite/libs (for backward compatibility). I guess
> later
> >> we
> >> have to think out of a solution to load libs directly from /usr/lib
> (along
> >> with configurations / logs / work / etc.)
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > We are building from binary package and not from sources. If it is
> used
> >> by
> >> > internal RPM building this is tolerable, but any distro or repository
> >> will
> >> > want to build from source.
> >>
> >> It's not an ordinary task to create "100% honest" package, so I guess we
> >> definitely won't make to 2.4 release. So I purpose include in the
> nearest
> >> release our package built from binary archive and do not supply sources
> >> package (as it is currently done in apache.org/dist for cassandra /
> hadoop
> >> or alike). And later design package build procedure, which includes
> getting
> >> sources from sources distribution, or even from Apache Ignite Git
> >> repository
> >> tag.
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > spec contains version (2.4.0) - will be needed to be changed for every
> >> > release. I'm not sure if it's possible to extract.
> >>
> >> Release procedure on TC is already has some task for project version
> >> update,
> >> so I guess it is not so difficult to update it accordingly (as it is
> >> currently done for C++ / .Net internal versions).
> >>
> >>
> >> Ilya Kasnacheev wrote
> >> > Package description may be improved. service description lacks '-' in
> "In
> >> > Memory".
> >>
> >> Fixed.
> >>
> >>
> >> So the only question to discuss (according Iliya's review) is whether to
> >> supply package with default-config.xml that has Multicast Discovery
> turned
> >> on or not?
> >>
> >> Also I'll appreciate any other comments concerning current package
> design.
> >>
> >>
> >>
> >> --
> >> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Andrey Gura <ag...@apache.org>.
Hi,
I think we should provide the same default configuration for all
binary builds. From my point of view RPM/DEB/etc packages should use
default configuration file like to standard binary release.
On Mon, Dec 25, 2017 at 4:39 PM, Ilya Kasnacheev
<il...@gmail.com> wrote:
> Hello Igniters!
>
> What's your take on enabling PDS in 2.4 in default config? This way we
> could enable it in RPM build with easy heart.
>
> The rest of your answers seem spot on. I'll happily review your amended
> patch when it is ready (later this week?)
>
> --
> Ilya Kasnacheev
>
> 2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
>
>> Ilya Kasnacheev wrote
>> > I have noticed that both spec and DEVNOTES are made to use tabs alongside
>> > with spaces. I thought we are avoiding tabs? Please confirm that usage of
>> > tabs is desired in this case.
>>
>> My fault - got used to editing such files in default vim. Fixed and updated
>> vim settings to indent with spaces.
>>
>>
>> Ilya Kasnacheev wrote
>> > Maybe it's my CentOS but initially build will fail with the following
>> > message, which I had to fix in spec
>> > + cd 'apache-ignite-*'
>> > /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file or
>> > directory
>>
>> Strange error - shell expansion is not working. Anyway, replaced with more
>> universal variant.
>>
>>
>> Ilya Kasnacheev wrote
>> > We absolutely should not inline configuration files (such as
>> > default-config.xml) into build scripts. We should just copy over
>> > default-config.xml from config/ to /etc, with dependencies if needed.
>>
>> Extracted corresponding sections into separate files.
>>
>>
>> Ilya Kasnacheev wrote
>> > I'm not sure PDS should be ON by default. Better provide it in examples
>> > but disable by default.
>>
>> AFAIK, PDS is actively pushing forward and has to be enabled by default so
>> that Ignite users start working with PDS from the very beginning.
>>
>>
>> Ilya Kasnacheev wrote
>> > I cannot comment on firewall rules validity, but firewall and service
>> > configurations should be stored in sources, as files, supplied with
>> > release (somewhere in packages/) and installed from there
>>
>> Unfortunately, I did not find any other way to open ports and multicast,
>> except applying direct iptables rules though firewall-cmd -- so there can
>> be
>> no separate service firewall rules file (as can be found here:
>> /usr/lib/firewalld/services). Applied rules from package go straight to
>> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
>> default configuration file and will stick to IP Discovery, I'll try to
>> revise firewall configuration and create separate service firewall rules
>> file (but, I guess, much later).
>>
>>
>> Ilya Kasnacheev wrote
>> > * sqlline.sh fails to connect initially, needs to be specified
>> > jdbc:ignite:thin://localhost, then it works. I think that sqlline should
>> > become available as /usr/bin/apache-ignite-sqlline for example, to be in
>> > $PATH. And figure out how to connect by itself.
>> > * ignitevisorcmd.sh becomes confused. first it scans
>> > /usr/share/apache-ignite for configs, then it fails to write to
>> > /usr/share/apache-ignite/work. We should consider how it can be fixed (a
>> > symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
>> > perhaps, and work dir in /tmp?), and made available as
>> > /usr/bin/apache-ignite-visorcmd.
>>
>> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
>> unaware-of-package design and I see the following 3-step way to overcome
>> this limitation.
>> At first we hide this executables file in examples (As Iliya proposed).
>> Secondly, I can design a patch, which can be applied during package build,
>> so that those executables work normally form the package installation.
>> And lastly, redesign them to be universal and compatible with all delivery
>> methods.
>>
>>
>> Ilya Kasnacheev wrote
>> > Not everything should go into /usr/share. classpath libs belong to
>> > /usr/lib.
>>
>> Moved libs to /usr/lib/apache-ignite and mapped to
>> /usr/share/apache-ignite/libs (for backward compatibility). I guess later
>> we
>> have to think out of a solution to load libs directly from /usr/lib (along
>> with configurations / logs / work / etc.)
>>
>>
>> Ilya Kasnacheev wrote
>> > We are building from binary package and not from sources. If it is used
>> by
>> > internal RPM building this is tolerable, but any distro or repository
>> will
>> > want to build from source.
>>
>> It's not an ordinary task to create "100% honest" package, so I guess we
>> definitely won't make to 2.4 release. So I purpose include in the nearest
>> release our package built from binary archive and do not supply sources
>> package (as it is currently done in apache.org/dist for cassandra / hadoop
>> or alike). And later design package build procedure, which includes getting
>> sources from sources distribution, or even from Apache Ignite Git
>> repository
>> tag.
>>
>>
>> Ilya Kasnacheev wrote
>> > spec contains version (2.4.0) - will be needed to be changed for every
>> > release. I'm not sure if it's possible to extract.
>>
>> Release procedure on TC is already has some task for project version
>> update,
>> so I guess it is not so difficult to update it accordingly (as it is
>> currently done for C++ / .Net internal versions).
>>
>>
>> Ilya Kasnacheev wrote
>> > Package description may be improved. service description lacks '-' in "In
>> > Memory".
>>
>> Fixed.
>>
>>
>> So the only question to discuss (according Iliya's review) is whether to
>> supply package with default-config.xml that has Multicast Discovery turned
>> on or not?
>>
>> Also I'll appreciate any other comments concerning current package design.
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello Igniters!
What's your take on enabling PDS in 2.4 in default config? This way we
could enable it in RPM build with easy heart.
The rest of your answers seem spot on. I'll happily review your amended
patch when it is ready (later this week?)
--
Ilya Kasnacheev
2017-12-22 18:27 GMT+03:00 vveider <mr...@gmail.com>:
> Ilya Kasnacheev wrote
> > I have noticed that both spec and DEVNOTES are made to use tabs alongside
> > with spaces. I thought we are avoiding tabs? Please confirm that usage of
> > tabs is desired in this case.
>
> My fault - got used to editing such files in default vim. Fixed and updated
> vim settings to indent with spaces.
>
>
> Ilya Kasnacheev wrote
> > Maybe it's my CentOS but initially build will fail with the following
> > message, which I had to fix in spec
> > + cd 'apache-ignite-*'
> > /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file or
> > directory
>
> Strange error - shell expansion is not working. Anyway, replaced with more
> universal variant.
>
>
> Ilya Kasnacheev wrote
> > We absolutely should not inline configuration files (such as
> > default-config.xml) into build scripts. We should just copy over
> > default-config.xml from config/ to /etc, with dependencies if needed.
>
> Extracted corresponding sections into separate files.
>
>
> Ilya Kasnacheev wrote
> > I'm not sure PDS should be ON by default. Better provide it in examples
> > but disable by default.
>
> AFAIK, PDS is actively pushing forward and has to be enabled by default so
> that Ignite users start working with PDS from the very beginning.
>
>
> Ilya Kasnacheev wrote
> > I cannot comment on firewall rules validity, but firewall and service
> > configurations should be stored in sources, as files, supplied with
> > release (somewhere in packages/) and installed from there
>
> Unfortunately, I did not find any other way to open ports and multicast,
> except applying direct iptables rules though firewall-cmd -- so there can
> be
> no separate service firewall rules file (as can be found here:
> /usr/lib/firewalld/services). Applied rules from package go straight to
> /etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
> default configuration file and will stick to IP Discovery, I'll try to
> revise firewall configuration and create separate service firewall rules
> file (but, I guess, much later).
>
>
> Ilya Kasnacheev wrote
> > * sqlline.sh fails to connect initially, needs to be specified
> > jdbc:ignite:thin://localhost, then it works. I think that sqlline should
> > become available as /usr/bin/apache-ignite-sqlline for example, to be in
> > $PATH. And figure out how to connect by itself.
> > * ignitevisorcmd.sh becomes confused. first it scans
> > /usr/share/apache-ignite for configs, then it fails to write to
> > /usr/share/apache-ignite/work. We should consider how it can be fixed (a
> > symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
> > perhaps, and work dir in /tmp?), and made available as
> > /usr/bin/apache-ignite-visorcmd.
>
> The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
> unaware-of-package design and I see the following 3-step way to overcome
> this limitation.
> At first we hide this executables file in examples (As Iliya proposed).
> Secondly, I can design a patch, which can be applied during package build,
> so that those executables work normally form the package installation.
> And lastly, redesign them to be universal and compatible with all delivery
> methods.
>
>
> Ilya Kasnacheev wrote
> > Not everything should go into /usr/share. classpath libs belong to
> > /usr/lib.
>
> Moved libs to /usr/lib/apache-ignite and mapped to
> /usr/share/apache-ignite/libs (for backward compatibility). I guess later
> we
> have to think out of a solution to load libs directly from /usr/lib (along
> with configurations / logs / work / etc.)
>
>
> Ilya Kasnacheev wrote
> > We are building from binary package and not from sources. If it is used
> by
> > internal RPM building this is tolerable, but any distro or repository
> will
> > want to build from source.
>
> It's not an ordinary task to create "100% honest" package, so I guess we
> definitely won't make to 2.4 release. So I purpose include in the nearest
> release our package built from binary archive and do not supply sources
> package (as it is currently done in apache.org/dist for cassandra / hadoop
> or alike). And later design package build procedure, which includes getting
> sources from sources distribution, or even from Apache Ignite Git
> repository
> tag.
>
>
> Ilya Kasnacheev wrote
> > spec contains version (2.4.0) - will be needed to be changed for every
> > release. I'm not sure if it's possible to extract.
>
> Release procedure on TC is already has some task for project version
> update,
> so I guess it is not so difficult to update it accordingly (as it is
> currently done for C++ / .Net internal versions).
>
>
> Ilya Kasnacheev wrote
> > Package description may be improved. service description lacks '-' in "In
> > Memory".
>
> Fixed.
>
>
> So the only question to discuss (according Iliya's review) is whether to
> supply package with default-config.xml that has Multicast Discovery turned
> on or not?
>
> Also I'll appreciate any other comments concerning current package design.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by vveider <mr...@gmail.com>.
Ilya Kasnacheev wrote
> I have noticed that both spec and DEVNOTES are made to use tabs alongside
> with spaces. I thought we are avoiding tabs? Please confirm that usage of
> tabs is desired in this case.
My fault - got used to editing such files in default vim. Fixed and updated
vim settings to indent with spaces.
Ilya Kasnacheev wrote
> Maybe it's my CentOS but initially build will fail with the following
> message, which I had to fix in spec
> + cd 'apache-ignite-*'
> /var/tmp/rpm-tmp.KwOoyl: line 34: cd: apache-ignite-*: No such file or
> directory
Strange error - shell expansion is not working. Anyway, replaced with more
universal variant.
Ilya Kasnacheev wrote
> We absolutely should not inline configuration files (such as
> default-config.xml) into build scripts. We should just copy over
> default-config.xml from config/ to /etc, with dependencies if needed.
Extracted corresponding sections into separate files.
Ilya Kasnacheev wrote
> I'm not sure PDS should be ON by default. Better provide it in examples
> but disable by default.
AFAIK, PDS is actively pushing forward and has to be enabled by default so
that Ignite users start working with PDS from the very beginning.
Ilya Kasnacheev wrote
> I cannot comment on firewall rules validity, but firewall and service
> configurations should be stored in sources, as files, supplied with
> release (somewhere in packages/) and installed from there
Unfortunately, I did not find any other way to open ports and multicast,
except applying direct iptables rules though firewall-cmd -- so there can be
no separate service firewall rules file (as can be found here:
/usr/lib/firewalld/services). Applied rules from package go straight to
/etc/firewalld/direct.xml. If we agree not to put Multicast-enabled by
default configuration file and will stick to IP Discovery, I'll try to
revise firewall configuration and create separate service firewall rules
file (but, I guess, much later).
Ilya Kasnacheev wrote
> * sqlline.sh fails to connect initially, needs to be specified
> jdbc:ignite:thin://localhost, then it works. I think that sqlline should
> become available as /usr/bin/apache-ignite-sqlline for example, to be in
> $PATH. And figure out how to connect by itself.
> * ignitevisorcmd.sh becomes confused. first it scans
> /usr/share/apache-ignite for configs, then it fails to write to
> /usr/share/apache-ignite/work. We should consider how it can be fixed (a
> symlink from /usr/share/apache-ignite/config to /etc/apache-ignite,
> perhaps, and work dir in /tmp?), and made available as
> /usr/bin/apache-ignite-visorcmd.
The problem exists mainly because of sqlline.sh|ignitevisorcmd.sh
unaware-of-package design and I see the following 3-step way to overcome
this limitation.
At first we hide this executables file in examples (As Iliya proposed).
Secondly, I can design a patch, which can be applied during package build,
so that those executables work normally form the package installation.
And lastly, redesign them to be universal and compatible with all delivery
methods.
Ilya Kasnacheev wrote
> Not everything should go into /usr/share. classpath libs belong to
> /usr/lib.
Moved libs to /usr/lib/apache-ignite and mapped to
/usr/share/apache-ignite/libs (for backward compatibility). I guess later we
have to think out of a solution to load libs directly from /usr/lib (along
with configurations / logs / work / etc.)
Ilya Kasnacheev wrote
> We are building from binary package and not from sources. If it is used by
> internal RPM building this is tolerable, but any distro or repository will
> want to build from source.
It's not an ordinary task to create "100% honest" package, so I guess we
definitely won't make to 2.4 release. So I purpose include in the nearest
release our package built from binary archive and do not supply sources
package (as it is currently done in apache.org/dist for cassandra / hadoop
or alike). And later design package build procedure, which includes getting
sources from sources distribution, or even from Apache Ignite Git repository
tag.
Ilya Kasnacheev wrote
> spec contains version (2.4.0) - will be needed to be changed for every
> release. I'm not sure if it's possible to extract.
Release procedure on TC is already has some task for project version update,
so I guess it is not so difficult to update it accordingly (as it is
currently done for C++ / .Net internal versions).
Ilya Kasnacheev wrote
> Package description may be improved. service description lacks '-' in "In
> Memory".
Fixed.
So the only question to discuss (according Iliya's review) is whether to
supply package with default-config.xml that has Multicast Discovery turned
on or not?
Also I'll appreciate any other comments concerning current package design.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I have reviewed your code and commented IGNITE-7107 for a list of potential
problem.
I think everything from the first "blocker" section ought to be fixed
before this patch is merged, while "nice to have" things may be spinned out
as separate tickets. I suggest moving scripts that don't work with
non-writable JAVA_HOME to examples section. This will likely include
benchmarks, which I didn't try (but suggest you to).
Regards,
--
Ilya Kasnacheev
2017-12-19 13:23 GMT+03:00 vveider <mr...@gmail.com>:
> Done. See updated PR#3171 [1]
>
>
> [1] https://github.com/apache/ignite/pull/3171
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by vveider <mr...@gmail.com>.
Done. See updated PR#3171 [1]
[1] https://github.com/apache/ignite/pull/3171
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Denis Magda <dm...@apache.org>.
Hi Petr,
Proposed to rename Ignite binary packages so that there is no “fabric” mentioning:
http://apache-ignite-developers.2346864.n4.nabble.com/Removing-quot-fabric-quot-from-Ignite-binary-package-name-td25419.html <http://apache-ignite-developers.2346864.n4.nabble.com/Removing-quot-fabric-quot-from-Ignite-binary-package-name-td25419.html>
Please consider this for RPM and DEB packages/instructions.
All the docs including DEVNOTES.txt will be updated once the “fabric” name is purged.
—
Denis
> On Dec 17, 2017, at 11:59 PM, vveider <mr...@gmail.com> wrote:
>
> Hi, Denis!
>
> Can you be a little bit more specific about "fabric" mentions? What exactly
> should be renamed?
> Package has final name "apache-ignite-fabric" in order to follow the
> convention of upstream source naming and DEVNOTES.txt instructions just name
> files as they are.
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by vveider <mr...@gmail.com>.
Hi, Denis!
Can you be a little bit more specific about "fabric" mentions? What exactly
should be renamed?
Package has final name "apache-ignite-fabric" in order to follow the
convention of upstream source naming and DEVNOTES.txt instructions just name
files as they are.
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Denis Magda <dm...@apache.org>.
Hi Peter,
Not being an expert in Linux packing system, the overall description looks clear to me.
A minor comment, is that all ‘apache-ignite-fabric’ occurrences have to be renamed to ‘apache-ignite’. Ignite is presently called as an in-memory computing platform (that is also a questionable name) and not a fabric. So for the simplicity it’s just better to use ‘apache-ignite’ for folders and config names.
—
Denis
> On Dec 13, 2017, at 11:33 PM, vveider <mr...@gmail.com> wrote:
>
> I've added IEP document [1] with current packages design for overview.
>
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-11%3A+Introduce+Apache+Ignite+delivery+in+RPM+and+DEB+packages
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by vveider <mr...@gmail.com>.
I've added IEP document [1] with current packages design for overview.
[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-11%3A+Introduce+Apache+Ignite+delivery+in+RPM+and+DEB+packages
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: IGNITE-7107 Apache Ignite RPM packages
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Thanks Petr, this is awesome. To be honest, I can hardly wait until it will
be possible to install Ignite with yum or apt-get.
D.
On Thu, Dec 7, 2017 at 7:02 AM, Petr Ivanov <mr...@gmail.com> wrote:
> HI, all!
>
>
> Within the task IGNITE-7107 I’ve prepare pull request #3171 [1] with:
> - specification to build RPM package out of built binary archive;
> - corresponding instruction in DEVNOTES.txt
>
> Also took responsibility to introduced style correction and unification to
> DEVNOTES.txt
>
>
> [1] - https://github.com/apache/ignite/pull/3171