Re: mesos IgniteProvider improvement

[Roman Shtykh <rs...@yahoo.com.INVALID>]

Ilya,
Sorry for getting back to you so late.
No idea, why it was necessary to make them configurable during the build process.I would go with this solution, since the whole process depends on the the links and will not likely be changed in near future.- Update the links in the file once and remove them from build process.
What do you think?
P.S. I moved the topic away from Ignite 2.7 release.

--
Roman 

    On Friday, September 14, 2018, 7:22:27 p.m. GMT+9, Ilya Kasnacheev <il...@gmail.com> wrote:  
 
 Hello!
So now there's an issue that this script makes source change after every build, show up in git status.
What we could do to it:- Commit the changes after the build, once. In hopes that it won't change very often. With benefit that we could do that right now, before the code freeze.
- Move these values to a properties file from both pom.xml and IgniteProvider.java. Any problems with this approach? We'll just read them from classpath properties file.
- Update the links in the file once and remove them from build process. Why were they added to build process in the first place - to make them configurable during build?

Regards,
-- 
Ilya Kasnacheev


вт, 11 сент. 2018 г. в 5:53, Roman Shtykh <rs...@yahoo.com>:

Ilya,
The "latest" version is the default, and resolved by https://ignite.apache.org/latest which is used by our web site when a user download the latest Ignite version. And I think this is the authority to judge of the latest official release (pom.xml you suggest can have SNAPSHOTs etc.).Also, as I explained during our review sessions, ignite-mesos-2.6.0 is a driver and doesn't mean you need to have Ignite 2.6.0. User can run any version of Ignite he/she specifies. By default, it's "latest" but a user can specify any version needed, even from a non-archive URL.
In short, what we have now1. mesos driver (ignite-mesos-x.x.x) will use "latest" version by default -> it will try to resolve the latest officially releases version of Apache Ignite, find the closest mirror and download Ignite in a minute. If the version resolution fails, we fall back to the slow apache archive (as you suggest; in my opinion we better fail-fast instead of waiting for hours to download, so the user can choose another download option (3))2. If the user specifies the version explicitly, it goes to the slow apache archive.
3. The user can put ignite zip file on his/her http server and provide the URL as a parameter to the driver, if options 1 and 2 don't work.
As you see, there are 3 options. And I just fix the 1st one with https://issues.apache.org/jira/browse/IGNITE-9388 and don't change the original logic (which I find reasonable) documented on our site -- I don't see how it blocks anything.

Roman Shtykh 

    On Monday, September 10, 2018, 6:16:15 p.m. GMT+9, Ilya Kasnacheev <il...@gmail.com> wrote:  
 
 Hello!
There's still two issues with the submission.
The first one is that we're downloading "latest" version from preferred mirror but a specified version, such as "2.6", we're also going to download from "slow" archive.apache.org/dist.That's a great limitation for this change, since most real deployments of Apache Ignite will have their Ignite version pegged to a specific release. But in this case there's no win in download speed.In my opinion it is a blocker.

The second one is that we can't download anything when we failed to resolve "latest". My idea is that we should try and download last known version in this case, which can be pushed to source from pom.xml, as we already do with URLs. So if you could not resolve "latest" you will download 2.7.0.
Buuut, maybe it's not necessary, maybe we should just discourage "latest", which is in my opinion almost always a bad idea.
WDYT?

Regards,
-- 
Ilya Kasnacheev


вс, 9 сент. 2018 г. в 5:47, Roman Shtykh <rs...@yahoo.com>:

Hi Ilya,
Sorry, missed that.
Added now.

--
Roman Shtykh 

    On Thursday, September 6, 2018, 6:16:58 p.m. GMT+9, Ilya Kasnacheev <il...@gmail.com> wrote:  
 
 Hello!
The last of my requests still standing is that we should fall-back to single URL download in case of error with 'latest' version. Everything else looks good to me.
Can we do that? I'm really worried that Apache API will go sour.

Regards,
-- 
Ilya Kasnacheev


чт, 6 сент. 2018 г. в 8:56, Roman Shtykh <rs...@yahoo.com>:

Hi Ilya,
Thanks again.
1) Done.2) Used catch() for latest version.

Please see my comments on github.
--
Roman Shtykh 

    On Wednesday, September 5, 2018, 11:30:10 p.m. GMT+9, Ilya Kasnacheev <il...@gmail.com> wrote:  
 
 Hello!
I've left a new wave of replies.
Basically, 1) let's keep DOWNLOAD_URL_PATTERN string value inlined so that it will work even if build process is broken (would be useful for e.g. developing out of IDE)
And also I urge you to catch() around new fragile Apache JSON API resolving, and download the 'current' version instead, as defined by ignite-mesos version.
This is because this module is not under continuouos scrutiny so extra care should be applied.-- 
Ilya Kasnacheev


вт, 4 сент. 2018 г. в 13:42, Roman Shtykh <rs...@yahoo.com>:

Thanks, Ilya!I will check your comments, and discuss it at JIRA.

--
Roman Shtykh 

    On Tuesday, September 4, 2018, 7:17:53 p.m. GMT+9, Ilya Kasnacheev <il...@gmail.com> wrote:  
 
 Hello!
IGNITE-9408 looks good to me and may be merged right away.
IGNITE-9388 needs more work in my opinion, I have commented the PR. I also advice having test for this functionality.

Regards,
-- 
Ilya Kasnacheev


вт, 4 сент. 2018 г. в 6:52, Roman Shtykh <rs...@yahoo.com.invalid>:

Igniters,
I would like Mesos integration update be included in the upcoming release.Can anyone review prs for the following issues?
IGNITE-9388: mesos IgniteProvider tries to access obsolete ignite.run or download from slow archiveIGNITE-9408: Update mesos version

Roman Shtykh 

    On Thursday, August 30, 2018, 9:25:43 p.m. GMT+9, Vyacheslav Daradur <da...@gmail.com> wrote:  

 Hi Igniters!

I'm working on the following Service Grid tasks:
- IGNITE-8361 Use discovery messages for service deployment
- IGNITE-8362 Collect service deployment results asynchronously on coordinator
- IGNITE-8363 Handle topology changes during service deployment
- IGNITE-8364 Propagate deployed services to joining nodes
- IGNITE-8365 Introduce service failure events
- IGNITE-3392 Propagate service deployment results from assigned nodes
to initiator

Let's call them *phase 1* because the should be implemented together
(atomically).

I do my best to finish phase 1 for including to 2.7 release.

But I'm not sure that the solution will be fully completed till the
beginning of October.

On Wed, Aug 29, 2018 at 7:18 PM Nikolay Izhikov <ni...@apache.org> wrote:
>
> Hell, Yakov
>
> I'm ok with your proposal.
>
>        * Scope freeze - September 17 - We should have a full list of tickets for 2.7 here.
>        * Code freeze - October 01 - We should merge all 2.7 tickets to master here.
>        * Vote on RC1 - October 11.
>        * Vote on release - October 15.
>
> В Ср, 29/08/2018 в 12:39 +0300, Yakov Zhdanov пишет:
> > Nikolay,
> >
> > I think we should have 2 weeks after code freeze which by the way may
> > include RC1 voting stage. This way I would like us to agree that release
> > candidate should be sent to vote on Oct, 11th and we can release on Oct,
> > 15th.
> >
> > What do you think?
> >
> > --Yakov



-- 
Best Regards, Vyacheslav D.