You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Saikat Maitra <sa...@gmail.com> on 2019/11/27 05:03:23 UTC
[DISCUSS] dependencies and release process for Ignite Extensions
Hello,
I wanted to connect and discuss on the release process for
ignite-extensions. As of today all our integrations since released together
were able to run build based on latest snapshot for example the current
build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
separate project with different release cycle then it make sense to have
dependencies on core modules based on released artifact for example the
dependency for ignite-core would be 2.7.6
Please review and share your thoughts.
PR https://github.com/apache/ignite-extensions/pull/1
Regards
Saikat
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Saikat,
I cannot access a TC project using your link. It says:
"You do not have enough permissions to access project with internal
id: project28"
Could you please give an access or describe shortly how does a build
pipeline look like?
пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
>
> Hello,
>
> I am running into a problem specific to teamcity build for Ignite
> Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am
> getting an error message during build as below
>
> [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext:
> Could not resolve dependencies for project
> org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following
> artifacts could not be resolved:
> org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find artifact
> org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
> https://h2database.com/m2-repo)
>
>
> and if set artifact dependencies for ~Build Apache Ignite~ then I receive
> below error
>
> [ERROR] [ERROR] Could not find the selected project in the reactor:
> :ignite-flink-ext @
>
> Build url
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
>
> Can you please let me know if you faced similar problem with teamcity build?
>
> I can set Ignite Extensions dependencies to released ignite-core artifacts
> version like 2.7.6 and build works fine.
>
> Regards,
> Saikat
>
>
>
>
>
> On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hello Denis,
> >
> > Thank you for your email and sharing your thoughts on the release process.
> > I will update the artifact id and dependencies for ignite-extensions
> > accordingly.
> >
> > I had created Ignite-Extensions project as a root level project and in
> > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
> > whereas I was able to pull dependencies for ignite-core 2.7.6 from maven
> > central. I will look into it further why teamcity build was not able to
> > pull snapshot dependencies.
> >
> > I will also create "ignite-core-2.9+" branch for the upcoming release
> > process.
> >
> > Thank you,
> > Saikat
> >
> >
> >
> >
> > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
> >
> >> Hi Saikat,
> >>
> >> Thanks for driving this activity forward and raising the question. Let me
> >> share my thoughts below and let's see what the broader community thinks.
> >>
> >> Each extension needs to have its own version unrelated to the core and
> >> Maven's groupId parameter for extension artifacts should be
> >> "org.apache.ignite.ext". For instance, the very first release of Flink in
> >> the form of extension should be pulled from Maven this way
> >>
> >> <dependency>
> >> <groupId>org.apache.ignite.ext</groupId>
> >> <artifactId>ignite-flink</artifactId>
> >> <version>1.0.0</version>
> >> </dependency>
> >>
> >> When it comes to the releases, all the extensions need to be verified for
> >> an upcoming release and updated if needed (with the version increase only
> >> for those updated). Thus, looks like the extensions master needs to be
> >> linked to the latest Ignite core snapshot. Whenever the core will be being
> >> prepared and any extensions need to be modified we can take this approach:
> >>
> >> - Create a branch of extensions for the upcoming core release. For
> >> instance, "ignite-core-2.9+" branch. That's just the branch name (and
> >> not
> >> any Maven artifact name) with "+" sign implying that the updated
> >> extensions
> >> will work for Ignite 2.9 and later until we need to update them again
> >> creating a release branch like "ignite-core-2.14+"
> >> - If only a subset of the extensions was updated then we need to
> >> release
> >> those extensions to Maven. The goal is to avoid the practice of
> >> publishing
> >> Flink or any other extension to Maven for every core release if there
> >> are
> >> no changes.
> >> - As for a ZIP archive, we should prepare the archive for a download
> >> with the name like "ignite-core-2.9+"
> >>
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hello,
> >> >
> >> > I wanted to connect and discuss on the release process for
> >> > ignite-extensions. As of today all our integrations since released
> >> together
> >> > were able to run build based on latest snapshot for example the current
> >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
> >> > separate project with different release cycle then it make sense to have
> >> > dependencies on core modules based on released artifact for example the
> >> > dependency for ignite-core would be 2.7.6
> >> >
> >> > Please review and share your thoughts.
> >> >
> >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> >
> >> > Regards
> >> > Saikat
> >> >
> >>
> >
--
Best regards,
Ivan Pavlukhin
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hello,
Please find the PR link below.
https://github.com/apache/ignite-extensions/pull/1/files
Regards,
Saikat
On Thu, Dec 26, 2019 at 1:43 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hello
>
> I have been able to resolve the build issue in teamcity and was able to
> pass the build failures.
>
>
> https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
>
> The issue that was causing failure was that I had put ignite.zip as
> artifact dependency and it was downloading the artifacts including pom file
> inside the build checkout folder of ignite-extensions and as a result the
> pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
> This was resulting in missing flink-ext module error as it was not even
> present in the ignite main pom.xml.
>
> To resolve the issue I just had to download and extract artifact of
> ignite.zip in a separate directory.
>
> If I can get a +1 on this PR then I can go ahead and merge the changes.
>
> Regards,
> Saikat
>
>
>
>
>
>
> On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Ivan,
>>
>> Thank you for sharing your feedback. I will try to install all modules in
>> local agent's repository and run the build.
>>
>> I will share updates.
>>
>> Regards,
>> Saikat
>>
>> On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin <vo...@gmail.com>
>> wrote:
>>
>>> Saikat,
>>>
>>> Let me share my understanding how do we run the majority of our jobs
>>> executing test suites. Hope it will help.
>>> 1. ~Build Apache Ignite~ in a separate run and publish everything as a
>>> single artifact ignite.zip (it contains ignite working directory with
>>> compiled modules, e.g. modules/core/target and others).
>>> 2. Download and extract ignite.zip and run particular test suites
>>> against extracted files.
>>>
>>> In a such setup test suites run the same as you do it on your local
>>> machine and it always works with a single multi-module maven project
>>> (apache-ignite). So, all dependencies are resolved in the working
>>> directory.
>>>
>>> For a separate IgniteExtensions maven project you need to install all
>>> dependencies from downloaded ignite.zip into .m2/repository local to
>>> the agent. I suppose we can do it as follows:
>>> 1. Download ignite.zip and extract it into "ignite" subfolder.
>>> 2. Run a maven command installing all modules into local maven
>>> repository. Something like "mvn install" but here we need to make sure
>>> that nothing is compiled once again.
>>> 3. Build IgniteExtensions.
>>>
>>> There might be a better way to do the same, I just tried to describe a
>>> schema how it can be done.
>>>
>>> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <sa...@gmail.com>:
>>> >
>>> > Hi Ivan, Ilya
>>> >
>>> > Thank you for your reply. I have added you as dev in IgniteExtensions
>>> > project and you should be able to check the build configurations.
>>> >
>>> > Yes, when I set artifact dependencies for ~Build Apache Ignite~
>>> similar to
>>> > other projects then I receive below error
>>> >
>>> > [ERROR] [ERROR] Could not find the selected project in the reactor:
>>> > :ignite-flink-ext @
>>> >
>>> > I do not see the same error when I am running the build in local and
>>> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my
>>> > local .m2 repository but it is failing in teamcity.
>>> >
>>> > Regards,
>>> > Saikat
>>> >
>>> >
>>> >
>>> >
>>> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev <
>>> ilya.kasnacheev@gmail.com>
>>> > wrote:
>>> >
>>> > > Hello!
>>> > >
>>> > > I think that your build should depend on an Apache Ignite build(s)
>>> or just
>>> > > use an already released version.
>>> > >
>>> > > In the same fashion as all our teamcity depend on "Build Apache
>>> Ignite" and
>>> > > use its artifacts.
>>> > >
>>> > > Here you want Ignite snapshot but Teamcity does not know where to
>>> take it
>>> > > from.
>>> > >
>>> > > Regards,
>>> > > --
>>> > > Ilya Kasnacheev
>>> > >
>>> > >
>>> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
>>> > >
>>> > > > Hello,
>>> > > >
>>> > > > I am running into a problem specific to teamcity build for Ignite
>>> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT
>>> I am
>>> > > > getting an error message during build as below
>>> > > >
>>> > > > [06:24:12][Step 4/5] Failed to execute goal on project
>>> ignite-flink-ext:
>>> > > > Could not resolve dependencies for project
>>> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The
>>> following
>>> > > > artifacts could not be resolved:
>>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
>>> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
>>> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
>>> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find
>>> > > artifact
>>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com
>>> (
>>> > > > https://h2database.com/m2-repo)
>>> > > >
>>> > > >
>>> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I
>>> receive
>>> > > > below error
>>> > > >
>>> > > > [ERROR] [ERROR] Could not find the selected project in the reactor:
>>> > > > :ignite-flink-ext @
>>> > > >
>>> > > > Build url
>>> > > >
>>> > > >
>>> > >
>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
>>> > > >
>>> > > > Can you please let me know if you faced similar problem with
>>> teamcity
>>> > > > build?
>>> > > >
>>> > > > I can set Ignite Extensions dependencies to released ignite-core
>>> > > artifacts
>>> > > > version like 2.7.6 and build works fine.
>>> > > >
>>> > > > Regards,
>>> > > > Saikat
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <
>>> saikat.maitra@gmail.com>
>>> > > > wrote:
>>> > > >
>>> > > > > Hello Denis,
>>> > > > >
>>> > > > > Thank you for your email and sharing your thoughts on the release
>>> > > > process.
>>> > > > > I will update the artifact id and dependencies for
>>> ignite-extensions
>>> > > > > accordingly.
>>> > > > >
>>> > > > > I had created Ignite-Extensions project as a root level project
>>> and in
>>> > > > > teamcity I was facing issues pulling dependencies for
>>> 2.8.0-SNAPSHOT
>>> > > > > whereas I was able to pull dependencies for ignite-core 2.7.6
>>> from
>>> > > maven
>>> > > > > central. I will look into it further why teamcity build was not
>>> able to
>>> > > > > pull snapshot dependencies.
>>> > > > >
>>> > > > > I will also create "ignite-core-2.9+" branch for the upcoming
>>> release
>>> > > > > process.
>>> > > > >
>>> > > > > Thank you,
>>> > > > > Saikat
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > >
>>> > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org>
>>> wrote:
>>> > > > >
>>> > > > >> Hi Saikat,
>>> > > > >>
>>> > > > >> Thanks for driving this activity forward and raising the
>>> question. Let
>>> > > > me
>>> > > > >> share my thoughts below and let's see what the broader community
>>> > > thinks.
>>> > > > >>
>>> > > > >> Each extension needs to have its own version unrelated to the
>>> core and
>>> > > > >> Maven's groupId parameter for extension artifacts should be
>>> > > > >> "org.apache.ignite.ext". For instance, the very first release
>>> of Flink
>>> > > > in
>>> > > > >> the form of extension should be pulled from Maven this way
>>> > > > >>
>>> > > > >> <dependency>
>>> > > > >> <groupId>org.apache.ignite.ext</groupId>
>>> > > > >> <artifactId>ignite-flink</artifactId>
>>> > > > >> <version>1.0.0</version>
>>> > > > >> </dependency>
>>> > > > >>
>>> > > > >> When it comes to the releases, all the extensions need to be
>>> verified
>>> > > > for
>>> > > > >> an upcoming release and updated if needed (with the version
>>> increase
>>> > > > only
>>> > > > >> for those updated). Thus, looks like the extensions master
>>> needs to be
>>> > > > >> linked to the latest Ignite core snapshot. Whenever the core
>>> will be
>>> > > > being
>>> > > > >> prepared and any extensions need to be modified we can take this
>>> > > > approach:
>>> > > > >>
>>> > > > >> - Create a branch of extensions for the upcoming core
>>> release. For
>>> > > > >> instance, "ignite-core-2.9+" branch. That's just the branch
>>> name
>>> > > (and
>>> > > > >> not
>>> > > > >> any Maven artifact name) with "+" sign implying that the
>>> updated
>>> > > > >> extensions
>>> > > > >> will work for Ignite 2.9 and later until we need to update
>>> them
>>> > > again
>>> > > > >> creating a release branch like "ignite-core-2.14+"
>>> > > > >> - If only a subset of the extensions was updated then we
>>> need to
>>> > > > >> release
>>> > > > >> those extensions to Maven. The goal is to avoid the practice
>>> of
>>> > > > >> publishing
>>> > > > >> Flink or any other extension to Maven for every core release
>>> if
>>> > > there
>>> > > > >> are
>>> > > > >> no changes.
>>> > > > >> - As for a ZIP archive, we should prepare the archive for a
>>> > > download
>>> > > > >> with the name like "ignite-core-2.9+"
>>> > > > >>
>>> > > > >>
>>> > > > >> -
>>> > > > >> Denis
>>> > > > >>
>>> > > > >>
>>> > > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <
>>> > > saikat.maitra@gmail.com>
>>> > > > >> wrote:
>>> > > > >>
>>> > > > >> > Hello,
>>> > > > >> >
>>> > > > >> > I wanted to connect and discuss on the release process for
>>> > > > >> > ignite-extensions. As of today all our integrations since
>>> released
>>> > > > >> together
>>> > > > >> > were able to run build based on latest snapshot for example
>>> the
>>> > > > current
>>> > > > >> > build depends on 2.8.0-SNAPSHOT. If we are making
>>> ignite-extensions
>>> > > as
>>> > > > >> > separate project with different release cycle then it make
>>> sense to
>>> > > > have
>>> > > > >> > dependencies on core modules based on released artifact for
>>> example
>>> > > > the
>>> > > > >> > dependency for ignite-core would be 2.7.6
>>> > > > >> >
>>> > > > >> > Please review and share your thoughts.
>>> > > > >> >
>>> > > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>>> > > > >> >
>>> > > > >> > Regards
>>> > > > >> > Saikat
>>> > > > >> >
>>> > > > >>
>>> > > > >
>>> > > >
>>> > >
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Ivan Pavlukhin
>>>
>>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hello
I have been able to resolve the build issue in teamcity and was able to
pass the build failures.
https://ci.ignite.apache.org/viewLog.html?buildId=4883394&buildTypeId=IgniteExtensions_Build
The issue that was causing failure was that I had put ignite.zip as
artifact dependency and it was downloading the artifacts including pom file
inside the build checkout folder of ignite-extensions and as a result the
pom.xml of ignite-extensions was getting overwritten by ignite's pom.xml.
This was resulting in missing flink-ext module error as it was not even
present in the ignite main pom.xml.
To resolve the issue I just had to download and extract artifact of
ignite.zip in a separate directory.
If I can get a +1 on this PR then I can go ahead and merge the changes.
Regards,
Saikat
On Tue, Dec 10, 2019 at 8:26 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hi Ivan,
>
> Thank you for sharing your feedback. I will try to install all modules in
> local agent's repository and run the build.
>
> I will share updates.
>
> Regards,
> Saikat
>
> On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin <vo...@gmail.com>
> wrote:
>
>> Saikat,
>>
>> Let me share my understanding how do we run the majority of our jobs
>> executing test suites. Hope it will help.
>> 1. ~Build Apache Ignite~ in a separate run and publish everything as a
>> single artifact ignite.zip (it contains ignite working directory with
>> compiled modules, e.g. modules/core/target and others).
>> 2. Download and extract ignite.zip and run particular test suites
>> against extracted files.
>>
>> In a such setup test suites run the same as you do it on your local
>> machine and it always works with a single multi-module maven project
>> (apache-ignite). So, all dependencies are resolved in the working
>> directory.
>>
>> For a separate IgniteExtensions maven project you need to install all
>> dependencies from downloaded ignite.zip into .m2/repository local to
>> the agent. I suppose we can do it as follows:
>> 1. Download ignite.zip and extract it into "ignite" subfolder.
>> 2. Run a maven command installing all modules into local maven
>> repository. Something like "mvn install" but here we need to make sure
>> that nothing is compiled once again.
>> 3. Build IgniteExtensions.
>>
>> There might be a better way to do the same, I just tried to describe a
>> schema how it can be done.
>>
>> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <sa...@gmail.com>:
>> >
>> > Hi Ivan, Ilya
>> >
>> > Thank you for your reply. I have added you as dev in IgniteExtensions
>> > project and you should be able to check the build configurations.
>> >
>> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar
>> to
>> > other projects then I receive below error
>> >
>> > [ERROR] [ERROR] Could not find the selected project in the reactor:
>> > :ignite-flink-ext @
>> >
>> > I do not see the same error when I am running the build in local and
>> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my
>> > local .m2 repository but it is failing in teamcity.
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> >
>> >
>> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev <
>> ilya.kasnacheev@gmail.com>
>> > wrote:
>> >
>> > > Hello!
>> > >
>> > > I think that your build should depend on an Apache Ignite build(s) or
>> just
>> > > use an already released version.
>> > >
>> > > In the same fashion as all our teamcity depend on "Build Apache
>> Ignite" and
>> > > use its artifacts.
>> > >
>> > > Here you want Ignite snapshot but Teamcity does not know where to
>> take it
>> > > from.
>> > >
>> > > Regards,
>> > > --
>> > > Ilya Kasnacheev
>> > >
>> > >
>> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
>> > >
>> > > > Hello,
>> > > >
>> > > > I am running into a problem specific to teamcity build for Ignite
>> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I
>> am
>> > > > getting an error message during build as below
>> > > >
>> > > > [06:24:12][Step 4/5] Failed to execute goal on project
>> ignite-flink-ext:
>> > > > Could not resolve dependencies for project
>> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The
>> following
>> > > > artifacts could not be resolved:
>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
>> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
>> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
>> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find
>> > > artifact
>> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com
>> (
>> > > > https://h2database.com/m2-repo)
>> > > >
>> > > >
>> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I
>> receive
>> > > > below error
>> > > >
>> > > > [ERROR] [ERROR] Could not find the selected project in the reactor:
>> > > > :ignite-flink-ext @
>> > > >
>> > > > Build url
>> > > >
>> > > >
>> > >
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
>> > > >
>> > > > Can you please let me know if you faced similar problem with
>> teamcity
>> > > > build?
>> > > >
>> > > > I can set Ignite Extensions dependencies to released ignite-core
>> > > artifacts
>> > > > version like 2.7.6 and build works fine.
>> > > >
>> > > > Regards,
>> > > > Saikat
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <
>> saikat.maitra@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Hello Denis,
>> > > > >
>> > > > > Thank you for your email and sharing your thoughts on the release
>> > > > process.
>> > > > > I will update the artifact id and dependencies for
>> ignite-extensions
>> > > > > accordingly.
>> > > > >
>> > > > > I had created Ignite-Extensions project as a root level project
>> and in
>> > > > > teamcity I was facing issues pulling dependencies for
>> 2.8.0-SNAPSHOT
>> > > > > whereas I was able to pull dependencies for ignite-core 2.7.6 from
>> > > maven
>> > > > > central. I will look into it further why teamcity build was not
>> able to
>> > > > > pull snapshot dependencies.
>> > > > >
>> > > > > I will also create "ignite-core-2.9+" branch for the upcoming
>> release
>> > > > > process.
>> > > > >
>> > > > > Thank you,
>> > > > > Saikat
>> > > > >
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org>
>> wrote:
>> > > > >
>> > > > >> Hi Saikat,
>> > > > >>
>> > > > >> Thanks for driving this activity forward and raising the
>> question. Let
>> > > > me
>> > > > >> share my thoughts below and let's see what the broader community
>> > > thinks.
>> > > > >>
>> > > > >> Each extension needs to have its own version unrelated to the
>> core and
>> > > > >> Maven's groupId parameter for extension artifacts should be
>> > > > >> "org.apache.ignite.ext". For instance, the very first release of
>> Flink
>> > > > in
>> > > > >> the form of extension should be pulled from Maven this way
>> > > > >>
>> > > > >> <dependency>
>> > > > >> <groupId>org.apache.ignite.ext</groupId>
>> > > > >> <artifactId>ignite-flink</artifactId>
>> > > > >> <version>1.0.0</version>
>> > > > >> </dependency>
>> > > > >>
>> > > > >> When it comes to the releases, all the extensions need to be
>> verified
>> > > > for
>> > > > >> an upcoming release and updated if needed (with the version
>> increase
>> > > > only
>> > > > >> for those updated). Thus, looks like the extensions master needs
>> to be
>> > > > >> linked to the latest Ignite core snapshot. Whenever the core
>> will be
>> > > > being
>> > > > >> prepared and any extensions need to be modified we can take this
>> > > > approach:
>> > > > >>
>> > > > >> - Create a branch of extensions for the upcoming core
>> release. For
>> > > > >> instance, "ignite-core-2.9+" branch. That's just the branch
>> name
>> > > (and
>> > > > >> not
>> > > > >> any Maven artifact name) with "+" sign implying that the
>> updated
>> > > > >> extensions
>> > > > >> will work for Ignite 2.9 and later until we need to update
>> them
>> > > again
>> > > > >> creating a release branch like "ignite-core-2.14+"
>> > > > >> - If only a subset of the extensions was updated then we need
>> to
>> > > > >> release
>> > > > >> those extensions to Maven. The goal is to avoid the practice
>> of
>> > > > >> publishing
>> > > > >> Flink or any other extension to Maven for every core release
>> if
>> > > there
>> > > > >> are
>> > > > >> no changes.
>> > > > >> - As for a ZIP archive, we should prepare the archive for a
>> > > download
>> > > > >> with the name like "ignite-core-2.9+"
>> > > > >>
>> > > > >>
>> > > > >> -
>> > > > >> Denis
>> > > > >>
>> > > > >>
>> > > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <
>> > > saikat.maitra@gmail.com>
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hello,
>> > > > >> >
>> > > > >> > I wanted to connect and discuss on the release process for
>> > > > >> > ignite-extensions. As of today all our integrations since
>> released
>> > > > >> together
>> > > > >> > were able to run build based on latest snapshot for example the
>> > > > current
>> > > > >> > build depends on 2.8.0-SNAPSHOT. If we are making
>> ignite-extensions
>> > > as
>> > > > >> > separate project with different release cycle then it make
>> sense to
>> > > > have
>> > > > >> > dependencies on core modules based on released artifact for
>> example
>> > > > the
>> > > > >> > dependency for ignite-core would be 2.7.6
>> > > > >> >
>> > > > >> > Please review and share your thoughts.
>> > > > >> >
>> > > > >> > PR https://github.com/apache/ignite-extensions/pull/1
>> > > > >> >
>> > > > >> > Regards
>> > > > >> > Saikat
>> > > > >> >
>> > > > >>
>> > > > >
>> > > >
>> > >
>>
>>
>>
>> --
>> Best regards,
>> Ivan Pavlukhin
>>
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Ivan,
Thank you for sharing your feedback. I will try to install all modules in
local agent's repository and run the build.
I will share updates.
Regards,
Saikat
On Tue, Dec 10, 2019 at 9:56 AM Ivan Pavlukhin <vo...@gmail.com> wrote:
> Saikat,
>
> Let me share my understanding how do we run the majority of our jobs
> executing test suites. Hope it will help.
> 1. ~Build Apache Ignite~ in a separate run and publish everything as a
> single artifact ignite.zip (it contains ignite working directory with
> compiled modules, e.g. modules/core/target and others).
> 2. Download and extract ignite.zip and run particular test suites
> against extracted files.
>
> In a such setup test suites run the same as you do it on your local
> machine and it always works with a single multi-module maven project
> (apache-ignite). So, all dependencies are resolved in the working
> directory.
>
> For a separate IgniteExtensions maven project you need to install all
> dependencies from downloaded ignite.zip into .m2/repository local to
> the agent. I suppose we can do it as follows:
> 1. Download ignite.zip and extract it into "ignite" subfolder.
> 2. Run a maven command installing all modules into local maven
> repository. Something like "mvn install" but here we need to make sure
> that nothing is compiled once again.
> 3. Build IgniteExtensions.
>
> There might be a better way to do the same, I just tried to describe a
> schema how it can be done.
>
> вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <sa...@gmail.com>:
> >
> > Hi Ivan, Ilya
> >
> > Thank you for your reply. I have added you as dev in IgniteExtensions
> > project and you should be able to check the build configurations.
> >
> > Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar
> to
> > other projects then I receive below error
> >
> > [ERROR] [ERROR] Could not find the selected project in the reactor:
> > :ignite-flink-ext @
> >
> > I do not see the same error when I am running the build in local and
> > maven package command can still get 2.9.0-SNAPSHOT dependencies from my
> > local .m2 repository but it is failing in teamcity.
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> > On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com>
> > wrote:
> >
> > > Hello!
> > >
> > > I think that your build should depend on an Apache Ignite build(s) or
> just
> > > use an already released version.
> > >
> > > In the same fashion as all our teamcity depend on "Build Apache
> Ignite" and
> > > use its artifacts.
> > >
> > > Here you want Ignite snapshot but Teamcity does not know where to take
> it
> > > from.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
> > >
> > > > Hello,
> > > >
> > > > I am running into a problem specific to teamcity build for Ignite
> > > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I
> am
> > > > getting an error message during build as below
> > > >
> > > > [06:24:12][Step 4/5] Failed to execute goal on project
> ignite-flink-ext:
> > > > Could not resolve dependencies for project
> > > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The
> following
> > > > artifacts could not be resolved:
> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
> > > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
> > > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
> > > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find
> > > artifact
> > > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
> > > > https://h2database.com/m2-repo)
> > > >
> > > >
> > > > and if set artifact dependencies for ~Build Apache Ignite~ then I
> receive
> > > > below error
> > > >
> > > > [ERROR] [ERROR] Could not find the selected project in the reactor:
> > > > :ignite-flink-ext @
> > > >
> > > > Build url
> > > >
> > > >
> > >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
> > > >
> > > > Can you please let me know if you faced similar problem with teamcity
> > > > build?
> > > >
> > > > I can set Ignite Extensions dependencies to released ignite-core
> > > artifacts
> > > > version like 2.7.6 and build works fine.
> > > >
> > > > Regards,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <
> saikat.maitra@gmail.com>
> > > > wrote:
> > > >
> > > > > Hello Denis,
> > > > >
> > > > > Thank you for your email and sharing your thoughts on the release
> > > > process.
> > > > > I will update the artifact id and dependencies for
> ignite-extensions
> > > > > accordingly.
> > > > >
> > > > > I had created Ignite-Extensions project as a root level project
> and in
> > > > > teamcity I was facing issues pulling dependencies for
> 2.8.0-SNAPSHOT
> > > > > whereas I was able to pull dependencies for ignite-core 2.7.6 from
> > > maven
> > > > > central. I will look into it further why teamcity build was not
> able to
> > > > > pull snapshot dependencies.
> > > > >
> > > > > I will also create "ignite-core-2.9+" branch for the upcoming
> release
> > > > > process.
> > > > >
> > > > > Thank you,
> > > > > Saikat
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org>
> wrote:
> > > > >
> > > > >> Hi Saikat,
> > > > >>
> > > > >> Thanks for driving this activity forward and raising the
> question. Let
> > > > me
> > > > >> share my thoughts below and let's see what the broader community
> > > thinks.
> > > > >>
> > > > >> Each extension needs to have its own version unrelated to the
> core and
> > > > >> Maven's groupId parameter for extension artifacts should be
> > > > >> "org.apache.ignite.ext". For instance, the very first release of
> Flink
> > > > in
> > > > >> the form of extension should be pulled from Maven this way
> > > > >>
> > > > >> <dependency>
> > > > >> <groupId>org.apache.ignite.ext</groupId>
> > > > >> <artifactId>ignite-flink</artifactId>
> > > > >> <version>1.0.0</version>
> > > > >> </dependency>
> > > > >>
> > > > >> When it comes to the releases, all the extensions need to be
> verified
> > > > for
> > > > >> an upcoming release and updated if needed (with the version
> increase
> > > > only
> > > > >> for those updated). Thus, looks like the extensions master needs
> to be
> > > > >> linked to the latest Ignite core snapshot. Whenever the core will
> be
> > > > being
> > > > >> prepared and any extensions need to be modified we can take this
> > > > approach:
> > > > >>
> > > > >> - Create a branch of extensions for the upcoming core release.
> For
> > > > >> instance, "ignite-core-2.9+" branch. That's just the branch
> name
> > > (and
> > > > >> not
> > > > >> any Maven artifact name) with "+" sign implying that the
> updated
> > > > >> extensions
> > > > >> will work for Ignite 2.9 and later until we need to update them
> > > again
> > > > >> creating a release branch like "ignite-core-2.14+"
> > > > >> - If only a subset of the extensions was updated then we need
> to
> > > > >> release
> > > > >> those extensions to Maven. The goal is to avoid the practice of
> > > > >> publishing
> > > > >> Flink or any other extension to Maven for every core release if
> > > there
> > > > >> are
> > > > >> no changes.
> > > > >> - As for a ZIP archive, we should prepare the archive for a
> > > download
> > > > >> with the name like "ignite-core-2.9+"
> > > > >>
> > > > >>
> > > > >> -
> > > > >> Denis
> > > > >>
> > > > >>
> > > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <
> > > saikat.maitra@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >> > Hello,
> > > > >> >
> > > > >> > I wanted to connect and discuss on the release process for
> > > > >> > ignite-extensions. As of today all our integrations since
> released
> > > > >> together
> > > > >> > were able to run build based on latest snapshot for example the
> > > > current
> > > > >> > build depends on 2.8.0-SNAPSHOT. If we are making
> ignite-extensions
> > > as
> > > > >> > separate project with different release cycle then it make
> sense to
> > > > have
> > > > >> > dependencies on core modules based on released artifact for
> example
> > > > the
> > > > >> > dependency for ignite-core would be 2.7.6
> > > > >> >
> > > > >> > Please review and share your thoughts.
> > > > >> >
> > > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > > >> >
> > > > >> > Regards
> > > > >> > Saikat
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
>
>
>
> --
> Best regards,
> Ivan Pavlukhin
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Ivan Pavlukhin <vo...@gmail.com>.
Saikat,
Let me share my understanding how do we run the majority of our jobs
executing test suites. Hope it will help.
1. ~Build Apache Ignite~ in a separate run and publish everything as a
single artifact ignite.zip (it contains ignite working directory with
compiled modules, e.g. modules/core/target and others).
2. Download and extract ignite.zip and run particular test suites
against extracted files.
In a such setup test suites run the same as you do it on your local
machine and it always works with a single multi-module maven project
(apache-ignite). So, all dependencies are resolved in the working
directory.
For a separate IgniteExtensions maven project you need to install all
dependencies from downloaded ignite.zip into .m2/repository local to
the agent. I suppose we can do it as follows:
1. Download ignite.zip and extract it into "ignite" subfolder.
2. Run a maven command installing all modules into local maven
repository. Something like "mvn install" but here we need to make sure
that nothing is compiled once again.
3. Build IgniteExtensions.
There might be a better way to do the same, I just tried to describe a
schema how it can be done.
вт, 10 дек. 2019 г. в 06:30, Saikat Maitra <sa...@gmail.com>:
>
> Hi Ivan, Ilya
>
> Thank you for your reply. I have added you as dev in IgniteExtensions
> project and you should be able to check the build configurations.
>
> Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar to
> other projects then I receive below error
>
> [ERROR] [ERROR] Could not find the selected project in the reactor:
> :ignite-flink-ext @
>
> I do not see the same error when I am running the build in local and
> maven package command can still get 2.9.0-SNAPSHOT dependencies from my
> local .m2 repository but it is failing in teamcity.
>
> Regards,
> Saikat
>
>
>
>
> On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
> > Hello!
> >
> > I think that your build should depend on an Apache Ignite build(s) or just
> > use an already released version.
> >
> > In the same fashion as all our teamcity depend on "Build Apache Ignite" and
> > use its artifacts.
> >
> > Here you want Ignite snapshot but Teamcity does not know where to take it
> > from.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
> >
> > > Hello,
> > >
> > > I am running into a problem specific to teamcity build for Ignite
> > > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am
> > > getting an error message during build as below
> > >
> > > [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext:
> > > Could not resolve dependencies for project
> > > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following
> > > artifacts could not be resolved:
> > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
> > > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
> > > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
> > > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find
> > artifact
> > > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
> > > https://h2database.com/m2-repo)
> > >
> > >
> > > and if set artifact dependencies for ~Build Apache Ignite~ then I receive
> > > below error
> > >
> > > [ERROR] [ERROR] Could not find the selected project in the reactor:
> > > :ignite-flink-ext @
> > >
> > > Build url
> > >
> > >
> > https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
> > >
> > > Can you please let me know if you faced similar problem with teamcity
> > > build?
> > >
> > > I can set Ignite Extensions dependencies to released ignite-core
> > artifacts
> > > version like 2.7.6 and build works fine.
> > >
> > > Regards,
> > > Saikat
> > >
> > >
> > >
> > >
> > >
> > > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <sa...@gmail.com>
> > > wrote:
> > >
> > > > Hello Denis,
> > > >
> > > > Thank you for your email and sharing your thoughts on the release
> > > process.
> > > > I will update the artifact id and dependencies for ignite-extensions
> > > > accordingly.
> > > >
> > > > I had created Ignite-Extensions project as a root level project and in
> > > > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
> > > > whereas I was able to pull dependencies for ignite-core 2.7.6 from
> > maven
> > > > central. I will look into it further why teamcity build was not able to
> > > > pull snapshot dependencies.
> > > >
> > > > I will also create "ignite-core-2.9+" branch for the upcoming release
> > > > process.
> > > >
> > > > Thank you,
> > > > Saikat
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
> > > >
> > > >> Hi Saikat,
> > > >>
> > > >> Thanks for driving this activity forward and raising the question. Let
> > > me
> > > >> share my thoughts below and let's see what the broader community
> > thinks.
> > > >>
> > > >> Each extension needs to have its own version unrelated to the core and
> > > >> Maven's groupId parameter for extension artifacts should be
> > > >> "org.apache.ignite.ext". For instance, the very first release of Flink
> > > in
> > > >> the form of extension should be pulled from Maven this way
> > > >>
> > > >> <dependency>
> > > >> <groupId>org.apache.ignite.ext</groupId>
> > > >> <artifactId>ignite-flink</artifactId>
> > > >> <version>1.0.0</version>
> > > >> </dependency>
> > > >>
> > > >> When it comes to the releases, all the extensions need to be verified
> > > for
> > > >> an upcoming release and updated if needed (with the version increase
> > > only
> > > >> for those updated). Thus, looks like the extensions master needs to be
> > > >> linked to the latest Ignite core snapshot. Whenever the core will be
> > > being
> > > >> prepared and any extensions need to be modified we can take this
> > > approach:
> > > >>
> > > >> - Create a branch of extensions for the upcoming core release. For
> > > >> instance, "ignite-core-2.9+" branch. That's just the branch name
> > (and
> > > >> not
> > > >> any Maven artifact name) with "+" sign implying that the updated
> > > >> extensions
> > > >> will work for Ignite 2.9 and later until we need to update them
> > again
> > > >> creating a release branch like "ignite-core-2.14+"
> > > >> - If only a subset of the extensions was updated then we need to
> > > >> release
> > > >> those extensions to Maven. The goal is to avoid the practice of
> > > >> publishing
> > > >> Flink or any other extension to Maven for every core release if
> > there
> > > >> are
> > > >> no changes.
> > > >> - As for a ZIP archive, we should prepare the archive for a
> > download
> > > >> with the name like "ignite-core-2.9+"
> > > >>
> > > >>
> > > >> -
> > > >> Denis
> > > >>
> > > >>
> > > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <
> > saikat.maitra@gmail.com>
> > > >> wrote:
> > > >>
> > > >> > Hello,
> > > >> >
> > > >> > I wanted to connect and discuss on the release process for
> > > >> > ignite-extensions. As of today all our integrations since released
> > > >> together
> > > >> > were able to run build based on latest snapshot for example the
> > > current
> > > >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions
> > as
> > > >> > separate project with different release cycle then it make sense to
> > > have
> > > >> > dependencies on core modules based on released artifact for example
> > > the
> > > >> > dependency for ignite-core would be 2.7.6
> > > >> >
> > > >> > Please review and share your thoughts.
> > > >> >
> > > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > > >> >
> > > >> > Regards
> > > >> > Saikat
> > > >> >
> > > >>
> > > >
> > >
> >
--
Best regards,
Ivan Pavlukhin
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Ivan, Ilya
Thank you for your reply. I have added you as dev in IgniteExtensions
project and you should be able to check the build configurations.
Yes, when I set artifact dependencies for ~Build Apache Ignite~ similar to
other projects then I receive below error
[ERROR] [ERROR] Could not find the selected project in the reactor:
:ignite-flink-ext @
I do not see the same error when I am running the build in local and
maven package command can still get 2.9.0-SNAPSHOT dependencies from my
local .m2 repository but it is failing in teamcity.
Regards,
Saikat
On Mon, Dec 9, 2019 at 5:18 AM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I think that your build should depend on an Apache Ignite build(s) or just
> use an already released version.
>
> In the same fashion as all our teamcity depend on "Build Apache Ignite" and
> use its artifacts.
>
> Here you want Ignite snapshot but Teamcity does not know where to take it
> from.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
>
> > Hello,
> >
> > I am running into a problem specific to teamcity build for Ignite
> > Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am
> > getting an error message during build as below
> >
> > [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext:
> > Could not resolve dependencies for project
> > org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following
> > artifacts could not be resolved:
> > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
> > org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
> > org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
> > org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find
> artifact
> > org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
> > https://h2database.com/m2-repo)
> >
> >
> > and if set artifact dependencies for ~Build Apache Ignite~ then I receive
> > below error
> >
> > [ERROR] [ERROR] Could not find the selected project in the reactor:
> > :ignite-flink-ext @
> >
> > Build url
> >
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
> >
> > Can you please let me know if you faced similar problem with teamcity
> > build?
> >
> > I can set Ignite Extensions dependencies to released ignite-core
> artifacts
> > version like 2.7.6 and build works fine.
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> > On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <sa...@gmail.com>
> > wrote:
> >
> > > Hello Denis,
> > >
> > > Thank you for your email and sharing your thoughts on the release
> > process.
> > > I will update the artifact id and dependencies for ignite-extensions
> > > accordingly.
> > >
> > > I had created Ignite-Extensions project as a root level project and in
> > > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
> > > whereas I was able to pull dependencies for ignite-core 2.7.6 from
> maven
> > > central. I will look into it further why teamcity build was not able to
> > > pull snapshot dependencies.
> > >
> > > I will also create "ignite-core-2.9+" branch for the upcoming release
> > > process.
> > >
> > > Thank you,
> > > Saikat
> > >
> > >
> > >
> > >
> > > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
> > >
> > >> Hi Saikat,
> > >>
> > >> Thanks for driving this activity forward and raising the question. Let
> > me
> > >> share my thoughts below and let's see what the broader community
> thinks.
> > >>
> > >> Each extension needs to have its own version unrelated to the core and
> > >> Maven's groupId parameter for extension artifacts should be
> > >> "org.apache.ignite.ext". For instance, the very first release of Flink
> > in
> > >> the form of extension should be pulled from Maven this way
> > >>
> > >> <dependency>
> > >> <groupId>org.apache.ignite.ext</groupId>
> > >> <artifactId>ignite-flink</artifactId>
> > >> <version>1.0.0</version>
> > >> </dependency>
> > >>
> > >> When it comes to the releases, all the extensions need to be verified
> > for
> > >> an upcoming release and updated if needed (with the version increase
> > only
> > >> for those updated). Thus, looks like the extensions master needs to be
> > >> linked to the latest Ignite core snapshot. Whenever the core will be
> > being
> > >> prepared and any extensions need to be modified we can take this
> > approach:
> > >>
> > >> - Create a branch of extensions for the upcoming core release. For
> > >> instance, "ignite-core-2.9+" branch. That's just the branch name
> (and
> > >> not
> > >> any Maven artifact name) with "+" sign implying that the updated
> > >> extensions
> > >> will work for Ignite 2.9 and later until we need to update them
> again
> > >> creating a release branch like "ignite-core-2.14+"
> > >> - If only a subset of the extensions was updated then we need to
> > >> release
> > >> those extensions to Maven. The goal is to avoid the practice of
> > >> publishing
> > >> Flink or any other extension to Maven for every core release if
> there
> > >> are
> > >> no changes.
> > >> - As for a ZIP archive, we should prepare the archive for a
> download
> > >> with the name like "ignite-core-2.9+"
> > >>
> > >>
> > >> -
> > >> Denis
> > >>
> > >>
> > >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <
> saikat.maitra@gmail.com>
> > >> wrote:
> > >>
> > >> > Hello,
> > >> >
> > >> > I wanted to connect and discuss on the release process for
> > >> > ignite-extensions. As of today all our integrations since released
> > >> together
> > >> > were able to run build based on latest snapshot for example the
> > current
> > >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions
> as
> > >> > separate project with different release cycle then it make sense to
> > have
> > >> > dependencies on core modules based on released artifact for example
> > the
> > >> > dependency for ignite-core would be 2.7.6
> > >> >
> > >> > Please review and share your thoughts.
> > >> >
> > >> > PR https://github.com/apache/ignite-extensions/pull/1
> > >> >
> > >> > Regards
> > >> > Saikat
> > >> >
> > >>
> > >
> >
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think that your build should depend on an Apache Ignite build(s) or just
use an already released version.
In the same fashion as all our teamcity depend on "Build Apache Ignite" and
use its artifacts.
Here you want Ignite snapshot but Teamcity does not know where to take it
from.
Regards,
--
Ilya Kasnacheev
пн, 9 дек. 2019 г. в 06:40, Saikat Maitra <sa...@gmail.com>:
> Hello,
>
> I am running into a problem specific to teamcity build for Ignite
> Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am
> getting an error message during build as below
>
> [06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext:
> Could not resolve dependencies for project
> org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following
> artifacts could not be resolved:
> org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
> org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find artifact
> org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
> https://h2database.com/m2-repo)
>
>
> and if set artifact dependencies for ~Build Apache Ignite~ then I receive
> below error
>
> [ERROR] [ERROR] Could not find the selected project in the reactor:
> :ignite-flink-ext @
>
> Build url
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
>
> Can you please let me know if you faced similar problem with teamcity
> build?
>
> I can set Ignite Extensions dependencies to released ignite-core artifacts
> version like 2.7.6 and build works fine.
>
> Regards,
> Saikat
>
>
>
>
>
> On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hello Denis,
> >
> > Thank you for your email and sharing your thoughts on the release
> process.
> > I will update the artifact id and dependencies for ignite-extensions
> > accordingly.
> >
> > I had created Ignite-Extensions project as a root level project and in
> > teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
> > whereas I was able to pull dependencies for ignite-core 2.7.6 from maven
> > central. I will look into it further why teamcity build was not able to
> > pull snapshot dependencies.
> >
> > I will also create "ignite-core-2.9+" branch for the upcoming release
> > process.
> >
> > Thank you,
> > Saikat
> >
> >
> >
> >
> > On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
> >
> >> Hi Saikat,
> >>
> >> Thanks for driving this activity forward and raising the question. Let
> me
> >> share my thoughts below and let's see what the broader community thinks.
> >>
> >> Each extension needs to have its own version unrelated to the core and
> >> Maven's groupId parameter for extension artifacts should be
> >> "org.apache.ignite.ext". For instance, the very first release of Flink
> in
> >> the form of extension should be pulled from Maven this way
> >>
> >> <dependency>
> >> <groupId>org.apache.ignite.ext</groupId>
> >> <artifactId>ignite-flink</artifactId>
> >> <version>1.0.0</version>
> >> </dependency>
> >>
> >> When it comes to the releases, all the extensions need to be verified
> for
> >> an upcoming release and updated if needed (with the version increase
> only
> >> for those updated). Thus, looks like the extensions master needs to be
> >> linked to the latest Ignite core snapshot. Whenever the core will be
> being
> >> prepared and any extensions need to be modified we can take this
> approach:
> >>
> >> - Create a branch of extensions for the upcoming core release. For
> >> instance, "ignite-core-2.9+" branch. That's just the branch name (and
> >> not
> >> any Maven artifact name) with "+" sign implying that the updated
> >> extensions
> >> will work for Ignite 2.9 and later until we need to update them again
> >> creating a release branch like "ignite-core-2.14+"
> >> - If only a subset of the extensions was updated then we need to
> >> release
> >> those extensions to Maven. The goal is to avoid the practice of
> >> publishing
> >> Flink or any other extension to Maven for every core release if there
> >> are
> >> no changes.
> >> - As for a ZIP archive, we should prepare the archive for a download
> >> with the name like "ignite-core-2.9+"
> >>
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <sa...@gmail.com>
> >> wrote:
> >>
> >> > Hello,
> >> >
> >> > I wanted to connect and discuss on the release process for
> >> > ignite-extensions. As of today all our integrations since released
> >> together
> >> > were able to run build based on latest snapshot for example the
> current
> >> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
> >> > separate project with different release cycle then it make sense to
> have
> >> > dependencies on core modules based on released artifact for example
> the
> >> > dependency for ignite-core would be 2.7.6
> >> >
> >> > Please review and share your thoughts.
> >> >
> >> > PR https://github.com/apache/ignite-extensions/pull/1
> >> >
> >> > Regards
> >> > Saikat
> >> >
> >>
> >
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hello,
I am running into a problem specific to teamcity build for Ignite
Extensions project. When I set the dependencies to 2.9.0-SNAPSHOT I am
getting an error message during build as below
[06:24:12][Step 4/5] Failed to execute goal on project ignite-flink-ext:
Could not resolve dependencies for project
org.apache.ignite.ext:ignite-flink-ext:jar:1.0.0-SNAPSHOT: The following
artifacts could not be resolved:
org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT,
org.apache.ignite:ignite-core:jar:tests:2.9.0-SNAPSHOT,
org.apache.ignite:ignite-log4j:jar:2.9.0-SNAPSHOT,
org.apache.ignite:ignite-spring:jar:2.9.0-SNAPSHOT: Could not find artifact
org.apache.ignite:ignite-core:jar:2.9.0-SNAPSHOT in h2database.com (
https://h2database.com/m2-repo)
and if set artifact dependencies for ~Build Apache Ignite~ then I receive
below error
[ERROR] [ERROR] Could not find the selected project in the reactor:
:ignite-flink-ext @
Build url
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Build&branch_IgniteExtensions=pull%2F1%2Fhead&tab=buildTypeStatusDiv
Can you please let me know if you faced similar problem with teamcity build?
I can set Ignite Extensions dependencies to released ignite-core artifacts
version like 2.7.6 and build works fine.
Regards,
Saikat
On Sat, Nov 30, 2019 at 1:50 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hello Denis,
>
> Thank you for your email and sharing your thoughts on the release process.
> I will update the artifact id and dependencies for ignite-extensions
> accordingly.
>
> I had created Ignite-Extensions project as a root level project and in
> teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
> whereas I was able to pull dependencies for ignite-core 2.7.6 from maven
> central. I will look into it further why teamcity build was not able to
> pull snapshot dependencies.
>
> I will also create "ignite-core-2.9+" branch for the upcoming release
> process.
>
> Thank you,
> Saikat
>
>
>
>
> On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
>
>> Hi Saikat,
>>
>> Thanks for driving this activity forward and raising the question. Let me
>> share my thoughts below and let's see what the broader community thinks.
>>
>> Each extension needs to have its own version unrelated to the core and
>> Maven's groupId parameter for extension artifacts should be
>> "org.apache.ignite.ext". For instance, the very first release of Flink in
>> the form of extension should be pulled from Maven this way
>>
>> <dependency>
>> <groupId>org.apache.ignite.ext</groupId>
>> <artifactId>ignite-flink</artifactId>
>> <version>1.0.0</version>
>> </dependency>
>>
>> When it comes to the releases, all the extensions need to be verified for
>> an upcoming release and updated if needed (with the version increase only
>> for those updated). Thus, looks like the extensions master needs to be
>> linked to the latest Ignite core snapshot. Whenever the core will be being
>> prepared and any extensions need to be modified we can take this approach:
>>
>> - Create a branch of extensions for the upcoming core release. For
>> instance, "ignite-core-2.9+" branch. That's just the branch name (and
>> not
>> any Maven artifact name) with "+" sign implying that the updated
>> extensions
>> will work for Ignite 2.9 and later until we need to update them again
>> creating a release branch like "ignite-core-2.14+"
>> - If only a subset of the extensions was updated then we need to
>> release
>> those extensions to Maven. The goal is to avoid the practice of
>> publishing
>> Flink or any other extension to Maven for every core release if there
>> are
>> no changes.
>> - As for a ZIP archive, we should prepare the archive for a download
>> with the name like "ignite-core-2.9+"
>>
>>
>> -
>> Denis
>>
>>
>> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <sa...@gmail.com>
>> wrote:
>>
>> > Hello,
>> >
>> > I wanted to connect and discuss on the release process for
>> > ignite-extensions. As of today all our integrations since released
>> together
>> > were able to run build based on latest snapshot for example the current
>> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
>> > separate project with different release cycle then it make sense to have
>> > dependencies on core modules based on released artifact for example the
>> > dependency for ignite-core would be 2.7.6
>> >
>> > Please review and share your thoughts.
>> >
>> > PR https://github.com/apache/ignite-extensions/pull/1
>> >
>> > Regards
>> > Saikat
>> >
>>
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Saikat Maitra <sa...@gmail.com>.
Hello Denis,
Thank you for your email and sharing your thoughts on the release process.
I will update the artifact id and dependencies for ignite-extensions
accordingly.
I had created Ignite-Extensions project as a root level project and in
teamcity I was facing issues pulling dependencies for 2.8.0-SNAPSHOT
whereas I was able to pull dependencies for ignite-core 2.7.6 from maven
central. I will look into it further why teamcity build was not able to
pull snapshot dependencies.
I will also create "ignite-core-2.9+" branch for the upcoming release
process.
Thank you,
Saikat
On Wed, Nov 27, 2019 at 1:05 PM Denis Magda <dm...@apache.org> wrote:
> Hi Saikat,
>
> Thanks for driving this activity forward and raising the question. Let me
> share my thoughts below and let's see what the broader community thinks.
>
> Each extension needs to have its own version unrelated to the core and
> Maven's groupId parameter for extension artifacts should be
> "org.apache.ignite.ext". For instance, the very first release of Flink in
> the form of extension should be pulled from Maven this way
>
> <dependency>
> <groupId>org.apache.ignite.ext</groupId>
> <artifactId>ignite-flink</artifactId>
> <version>1.0.0</version>
> </dependency>
>
> When it comes to the releases, all the extensions need to be verified for
> an upcoming release and updated if needed (with the version increase only
> for those updated). Thus, looks like the extensions master needs to be
> linked to the latest Ignite core snapshot. Whenever the core will be being
> prepared and any extensions need to be modified we can take this approach:
>
> - Create a branch of extensions for the upcoming core release. For
> instance, "ignite-core-2.9+" branch. That's just the branch name (and
> not
> any Maven artifact name) with "+" sign implying that the updated
> extensions
> will work for Ignite 2.9 and later until we need to update them again
> creating a release branch like "ignite-core-2.14+"
> - If only a subset of the extensions was updated then we need to release
> those extensions to Maven. The goal is to avoid the practice of
> publishing
> Flink or any other extension to Maven for every core release if there
> are
> no changes.
> - As for a ZIP archive, we should prepare the archive for a download
> with the name like "ignite-core-2.9+"
>
>
> -
> Denis
>
>
> On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
> > Hello,
> >
> > I wanted to connect and discuss on the release process for
> > ignite-extensions. As of today all our integrations since released
> together
> > were able to run build based on latest snapshot for example the current
> > build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
> > separate project with different release cycle then it make sense to have
> > dependencies on core modules based on released artifact for example the
> > dependency for ignite-core would be 2.7.6
> >
> > Please review and share your thoughts.
> >
> > PR https://github.com/apache/ignite-extensions/pull/1
> >
> > Regards
> > Saikat
> >
>
Re: [DISCUSS] dependencies and release process for Ignite Extensions
Posted by Denis Magda <dm...@apache.org>.
Hi Saikat,
Thanks for driving this activity forward and raising the question. Let me
share my thoughts below and let's see what the broader community thinks.
Each extension needs to have its own version unrelated to the core and
Maven's groupId parameter for extension artifacts should be
"org.apache.ignite.ext". For instance, the very first release of Flink in
the form of extension should be pulled from Maven this way
<dependency>
<groupId>org.apache.ignite.ext</groupId>
<artifactId>ignite-flink</artifactId>
<version>1.0.0</version>
</dependency>
When it comes to the releases, all the extensions need to be verified for
an upcoming release and updated if needed (with the version increase only
for those updated). Thus, looks like the extensions master needs to be
linked to the latest Ignite core snapshot. Whenever the core will be being
prepared and any extensions need to be modified we can take this approach:
- Create a branch of extensions for the upcoming core release. For
instance, "ignite-core-2.9+" branch. That's just the branch name (and not
any Maven artifact name) with "+" sign implying that the updated extensions
will work for Ignite 2.9 and later until we need to update them again
creating a release branch like "ignite-core-2.14+"
- If only a subset of the extensions was updated then we need to release
those extensions to Maven. The goal is to avoid the practice of publishing
Flink or any other extension to Maven for every core release if there are
no changes.
- As for a ZIP archive, we should prepare the archive for a download
with the name like "ignite-core-2.9+"
-
Denis
On Tue, Nov 26, 2019 at 9:03 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hello,
>
> I wanted to connect and discuss on the release process for
> ignite-extensions. As of today all our integrations since released together
> were able to run build based on latest snapshot for example the current
> build depends on 2.8.0-SNAPSHOT. If we are making ignite-extensions as
> separate project with different release cycle then it make sense to have
> dependencies on core modules based on released artifact for example the
> dependency for ignite-core would be 2.7.6
>
> Please review and share your thoughts.
>
> PR https://github.com/apache/ignite-extensions/pull/1
>
> Regards
> Saikat
>