You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Stephen Darlington <st...@gridgain.com> on 2022/01/26 18:07:32 UTC
Re: Move Azure, AWS, GCE to the ignite-extensions
Do we know what happened with this thread? I just saw a question in the user list asking about where the ignite-aws-ext module is now that Ignite 2.12 is out.
Regards,
Stephen
> On 24 Nov 2021, at 11:11, Maxim Muzafarov <mm...@apache.org> wrote:
>
> Petr,
>
> There is only the GCE suite left to be configured. I've created an
> issue [1] to do this. Please, take a look.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-15981
>
> On Wed, 24 Nov 2021 at 12:00, Petr Ivanov <mr...@gmail.com> wrote:
>>
>> Hi, Maksim.
>>
>>
>> Can you file a ticket about recreating test suites for extensions, please?
>> I will attend to it in nearest time.
>>
>>
>>> On 23 Nov 2021, at 17:14, Maxim Muzafarov <mm...@apache.org> wrote:
>>>
>>> Hello Petr,
>>>
>>> Can you assist me with configuring the GCE [1] suite on the TC
>>> Extensions project? Currently, I have an issue with moving environment
>>> variables from the old GCE suite [2] to the new one.
>>>
>>> I need to create the following envs:
>>> - env.test.gce.account.id
>>> - env.test.gce.p12.path
>>> - env.test.gce.project.name
>>>
>>> However the `id` seems to be a password, so it's hidden on the admin
>>> panel. Can you please help me with this?
>>>
>>> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Tests_Gce&branch_IgniteExtensions_Tests=%3Cdefault%3E&tab=buildTypeStatusDiv
>>> [2] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_GceOld&branch_IgniteTests24Java8=ignite-2.12&tab=buildTypeStatusDiv
>>>
>>> On Mon, 25 Oct 2021 at 14:22, Maxim Muzafarov <mm...@apache.org> wrote:
>>>>
>>>> Folks,
>>>>
>>>> I've moved the azure, gce, aws modules to the ignite-extensions project.
>>>> https://issues.apache.org/jira/browse/IGNITE-15541
>>>>
>>>> Building the modules in the ignite-extension project will prepare an
>>>> appropriate release zip file containing all the necessary
>>>> dependencies:
>>>> - ignite-aws-ext.zip
>>>> - ignite-gce-ext.zip
>>>> - ignite-auzre-ext.zip
>>>>
>>>>
>>>> On Wed, 13 Oct 2021 at 17:09, Stephen Darlington
>>>> <st...@gridgain.com> wrote:
>>>>>
>>>>> Okay, I phrased that badly. I mean an extra platform-specific ZIP file that I used to augment the generic Ignite ZIP file.
>>>>>
>>>>> So, to run on Azure I’d download ignite.zip + azure.zip.
>>>>>
>>>>> Extending ignite.sh would also be great, kind of like what’s happening with Ignite 3 as far as I can tell.
>>>>>
>>>>> What I’m advocating is not needing to use Maven just to run Ignite on Azure, AWS, etc.
>>>>>
>>>>>> On 13 Oct 2021, at 14:35, Petr Ivanov <mr...@gmail.com> wrote:
>>>>>>
>>>>>> Our self-contained zip file currently is over 400Mb and continues to grow.
>>>>>> Even considering that internet speeds has grown too, it is nonsense to force user to download such an archive where 90% are useless for most cases.
>>>>>>
>>>>>> Also we can:
>>>>>> — pack all extensions in single binary with latests releases (and update after each extension release) or even one by one
>>>>>> — extend ignite.sh to download remote libs when extension is activated via command line
>>>>>>
>>>>>>
>>>>>> Antoine de Saint-Exupéry once said that 'perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away'.
>>>>>> We are not obliged to make Apache Ignite ideal, but we certainly can move that way — I am sure the result will exceed expectations.
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 13 Oct 2021, at 16:02, Stephen Darlington <st...@gridgain.com> wrote:
>>>>>>>
>>>>>>> Having extensions in Maven Central makes perfect sense for tools that need to be built and integrated with other code, Spring integrations for example.
>>>>>>>
>>>>>>> That’s not the case for extensions that are required just to run Ignite. A self-contained zip file for each platform would work.
>>>>>>>
>>>>>>>> On 13 Oct 2021, at 13:41, Maxim Muzafarov <mm...@apache.org> wrote:
>>>>>>>>
>>>>>>>> Nikolay,
>>>>>>>>
>>>>>>>> All extensions will be available at the maven central for download.
>>>>>>>>
>>>>>>>> Previously extensions have a dependent version on the ignite core, so
>>>>>>>> each time the Ignite was released it made sense to include all the
>>>>>>>> extensions into the uber-zip file. Each extension has its own release
>>>>>>>> version now, so an extension can be upgraded and used independently,
>>>>>>>> what is the reason include it in the single uber-zip file? Probably it
>>>>>>>> would be better to provide a self-contained zip file for each cloud
>>>>>>>> platform.
>>>>>>>>
>>>>>>>> If I've missed your issue, so can you clarify the problem in more detail?
>>>>>>>>
>>>>>>>> On Wed, 13 Oct 2021 at 14:37, Nikolay Izhikov <ni...@apache.org> wrote:
>>>>>>>>>
>>>>>>>>> Maxim.
>>>>>>>>>
>>>>>>>>>> Currently, they are copied from the optional
>>>>>>>>>> directory of the ignite binary package but would be copied from an
>>>>>>>>>> appropriate ignite extension binary package.
>>>>>>>>>
>>>>>>>>> But how, the user will download this binary package?
>>>>>>>>> Right now, all the user need is Ignite distributive.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 13 окт. 2021 г., в 14:32, Maxim Muzafarov <mm...@apache.org> написал(а):
>>>>>>>>>>
>>>>>>>>>> Stephen,
>>>>>>>>>>
>>>>>>>>>> I guess the required classes of IP-finders should be in the classpath
>>>>>>>>>> (libs directory). Currently, they are copied from the optional
>>>>>>>>>> directory of the ignite binary package but would be copied from an
>>>>>>>>>> appropriate ignite extension binary package. Probably I'm missing
>>>>>>>>>> something but almost nothing changes in that process from my point of
>>>>>>>>>> view. The documentation pages will be updated prior to the release.
>>>>>>>>>>
>>>>>>>>>> On Wed, 13 Oct 2021 at 13:44, Stephen Darlington
>>>>>>>>>> <st...@gridgain.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> I understand the motivation from a development point of view, but how will this work for end users? Currently, the documentation talks about extensions only in terms of importing maven dependencies (download.cgi <https://ignite.apache.org/download.cgi#extensions>). If I’m trying to start a cluster on Azure, how does that work? Do I need to build my own server?
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>> Stephen
>>>>>>>>>>>
>>>>>>>>>>>> On 13 Oct 2021, at 11:35, Nikita Amelchev <na...@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> +1 to migrate and include to the Ignite 2.12 scope
>>>>>>>>>>>>
>>>>>>>>>>>> пн, 20 сент. 2021 г. в 17:09, Denis Magda <dm...@apache.org>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Perfect, thanks, Maxim!
>>>>>>>>>>>>>
>>>>>>>>>>>>> -
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Mon, Sep 20, 2021 at 8:29 AM Maxim Muzafarov <mm...@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Folks,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've created an issue [1] to move all cloud-based IP-finders to the
>>>>>>>>>>>>>> ignite-extensions. The motivation is the same as with migration of
>>>>>>>>>>>>>> Spring Data integration - to remove integration dependency of the
>>>>>>>>>>>>>> release cycle on Ignite releases.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-15541
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Best wishes,
>>>>>>>>>>>> Amelchev Nikita
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>
Re: Move Azure, AWS, GCE to the ignite-extensions
Posted by Maxim Muzafarov <mm...@apache.org>.
Hello Stephen,
I'm planning to release the extensions soon. Tests are OK, however,
some TC Suites still need to be configured.
On Wed, 26 Jan 2022 at 21:07, Stephen Darlington
<st...@gridgain.com> wrote:
>
> Do we know what happened with this thread? I just saw a question in the user list asking about where the ignite-aws-ext module is now that Ignite 2.12 is out.
>
> Regards,
> Stephen
>
> > On 24 Nov 2021, at 11:11, Maxim Muzafarov <mm...@apache.org> wrote:
> >
> > Petr,
> >
> > There is only the GCE suite left to be configured. I've created an
> > issue [1] to do this. Please, take a look.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-15981
> >
> > On Wed, 24 Nov 2021 at 12:00, Petr Ivanov <mr...@gmail.com> wrote:
> >>
> >> Hi, Maksim.
> >>
> >>
> >> Can you file a ticket about recreating test suites for extensions, please?
> >> I will attend to it in nearest time.
> >>
> >>
> >>> On 23 Nov 2021, at 17:14, Maxim Muzafarov <mm...@apache.org> wrote:
> >>>
> >>> Hello Petr,
> >>>
> >>> Can you assist me with configuring the GCE [1] suite on the TC
> >>> Extensions project? Currently, I have an issue with moving environment
> >>> variables from the old GCE suite [2] to the new one.
> >>>
> >>> I need to create the following envs:
> >>> - env.test.gce.account.id
> >>> - env.test.gce.p12.path
> >>> - env.test.gce.project.name
> >>>
> >>> However the `id` seems to be a password, so it's hidden on the admin
> >>> panel. Can you please help me with this?
> >>>
> >>> [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteExtensions_Tests_Gce&branch_IgniteExtensions_Tests=%3Cdefault%3E&tab=buildTypeStatusDiv
> >>> [2] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_GceOld&branch_IgniteTests24Java8=ignite-2.12&tab=buildTypeStatusDiv
> >>>
> >>> On Mon, 25 Oct 2021 at 14:22, Maxim Muzafarov <mm...@apache.org> wrote:
> >>>>
> >>>> Folks,
> >>>>
> >>>> I've moved the azure, gce, aws modules to the ignite-extensions project.
> >>>> https://issues.apache.org/jira/browse/IGNITE-15541
> >>>>
> >>>> Building the modules in the ignite-extension project will prepare an
> >>>> appropriate release zip file containing all the necessary
> >>>> dependencies:
> >>>> - ignite-aws-ext.zip
> >>>> - ignite-gce-ext.zip
> >>>> - ignite-auzre-ext.zip
> >>>>
> >>>>
> >>>> On Wed, 13 Oct 2021 at 17:09, Stephen Darlington
> >>>> <st...@gridgain.com> wrote:
> >>>>>
> >>>>> Okay, I phrased that badly. I mean an extra platform-specific ZIP file that I used to augment the generic Ignite ZIP file.
> >>>>>
> >>>>> So, to run on Azure I’d download ignite.zip + azure.zip.
> >>>>>
> >>>>> Extending ignite.sh would also be great, kind of like what’s happening with Ignite 3 as far as I can tell.
> >>>>>
> >>>>> What I’m advocating is not needing to use Maven just to run Ignite on Azure, AWS, etc.
> >>>>>
> >>>>>> On 13 Oct 2021, at 14:35, Petr Ivanov <mr...@gmail.com> wrote:
> >>>>>>
> >>>>>> Our self-contained zip file currently is over 400Mb and continues to grow.
> >>>>>> Even considering that internet speeds has grown too, it is nonsense to force user to download such an archive where 90% are useless for most cases.
> >>>>>>
> >>>>>> Also we can:
> >>>>>> — pack all extensions in single binary with latests releases (and update after each extension release) or even one by one
> >>>>>> — extend ignite.sh to download remote libs when extension is activated via command line
> >>>>>>
> >>>>>>
> >>>>>> Antoine de Saint-Exupéry once said that 'perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away'.
> >>>>>> We are not obliged to make Apache Ignite ideal, but we certainly can move that way — I am sure the result will exceed expectations.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On 13 Oct 2021, at 16:02, Stephen Darlington <st...@gridgain.com> wrote:
> >>>>>>>
> >>>>>>> Having extensions in Maven Central makes perfect sense for tools that need to be built and integrated with other code, Spring integrations for example.
> >>>>>>>
> >>>>>>> That’s not the case for extensions that are required just to run Ignite. A self-contained zip file for each platform would work.
> >>>>>>>
> >>>>>>>> On 13 Oct 2021, at 13:41, Maxim Muzafarov <mm...@apache.org> wrote:
> >>>>>>>>
> >>>>>>>> Nikolay,
> >>>>>>>>
> >>>>>>>> All extensions will be available at the maven central for download.
> >>>>>>>>
> >>>>>>>> Previously extensions have a dependent version on the ignite core, so
> >>>>>>>> each time the Ignite was released it made sense to include all the
> >>>>>>>> extensions into the uber-zip file. Each extension has its own release
> >>>>>>>> version now, so an extension can be upgraded and used independently,
> >>>>>>>> what is the reason include it in the single uber-zip file? Probably it
> >>>>>>>> would be better to provide a self-contained zip file for each cloud
> >>>>>>>> platform.
> >>>>>>>>
> >>>>>>>> If I've missed your issue, so can you clarify the problem in more detail?
> >>>>>>>>
> >>>>>>>> On Wed, 13 Oct 2021 at 14:37, Nikolay Izhikov <ni...@apache.org> wrote:
> >>>>>>>>>
> >>>>>>>>> Maxim.
> >>>>>>>>>
> >>>>>>>>>> Currently, they are copied from the optional
> >>>>>>>>>> directory of the ignite binary package but would be copied from an
> >>>>>>>>>> appropriate ignite extension binary package.
> >>>>>>>>>
> >>>>>>>>> But how, the user will download this binary package?
> >>>>>>>>> Right now, all the user need is Ignite distributive.
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>> 13 окт. 2021 г., в 14:32, Maxim Muzafarov <mm...@apache.org> написал(а):
> >>>>>>>>>>
> >>>>>>>>>> Stephen,
> >>>>>>>>>>
> >>>>>>>>>> I guess the required classes of IP-finders should be in the classpath
> >>>>>>>>>> (libs directory). Currently, they are copied from the optional
> >>>>>>>>>> directory of the ignite binary package but would be copied from an
> >>>>>>>>>> appropriate ignite extension binary package. Probably I'm missing
> >>>>>>>>>> something but almost nothing changes in that process from my point of
> >>>>>>>>>> view. The documentation pages will be updated prior to the release.
> >>>>>>>>>>
> >>>>>>>>>> On Wed, 13 Oct 2021 at 13:44, Stephen Darlington
> >>>>>>>>>> <st...@gridgain.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> I understand the motivation from a development point of view, but how will this work for end users? Currently, the documentation talks about extensions only in terms of importing maven dependencies (download.cgi <https://ignite.apache.org/download.cgi#extensions>). If I’m trying to start a cluster on Azure, how does that work? Do I need to build my own server?
> >>>>>>>>>>>
> >>>>>>>>>>> Regards,
> >>>>>>>>>>> Stephen
> >>>>>>>>>>>
> >>>>>>>>>>>> On 13 Oct 2021, at 11:35, Nikita Amelchev <na...@apache.org> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>> +1 to migrate and include to the Ignite 2.12 scope
> >>>>>>>>>>>>
> >>>>>>>>>>>> пн, 20 сент. 2021 г. в 17:09, Denis Magda <dm...@apache.org>:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Perfect, thanks, Maxim!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Mon, Sep 20, 2021 at 8:29 AM Maxim Muzafarov <mm...@apache.org> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Folks,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I've created an issue [1] to move all cloud-based IP-finders to the
> >>>>>>>>>>>>>> ignite-extensions. The motivation is the same as with migration of
> >>>>>>>>>>>>>> Spring Data integration - to remove integration dependency of the
> >>>>>>>>>>>>>> release cycle on Ignite releases.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-15541
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Best wishes,
> >>>>>>>>>>>> Amelchev Nikita
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>
>