You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Denis Magda <dm...@gridgain.com> on 2017/10/02 05:35:46 UTC
Re: [DISCUSS] Ignite Update Checker
There is an obstacle. There is no way to execute the script using PHP or
similar sever side language and trigger GA as discussed earlier:
https://issues.apache.org/jira/browse/INFRA-15182
How else can we tackle this?
Denis
On Thursday, September 7, 2017, Dmitriy Setrakyan <ds...@apache.org>
wrote:
> I think it is safe to assume at this point that everyone is in general
> agreement, since there are no active objections.
>
> I have filed a ticket for the 2.3 release. Let's try to make it happen:
> https://issues.apache.org/jira/browse/IGNITE-6305
>
> D.
>
> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <dsetrakyan@apache.org
> <javascript:;>>
> wrote:
>
> >
> >
> > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com
> <javascript:;>>
> > wrote:
> >
> >> Yeah, I guess that's doable as well and requires less management effort
> >> than my suggestion. We could use events [1] to store payload data (e.g.
> >> IP,
> >> version, etc.)
> >
> >
> > Yes, we could use events or some other similar API provided by GA.
> >
> >
> >> What the download page CGI developed in? PHP?
> >>
> >
> > To be honest, no clue. I guess someone in the community can figure it
> out:
> > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >
> >
> >> However, I'm not sure whether storing this data in a 3rd party (Google)
> is
> >> compliant with the ASF policy. I guess it's no biggie, but if there's
> >> doubt
> >> in the PMC, it's better to ask legal@.
> >
> >
> > I am not sure there is anything to ask about. The whole Ignite website is
> > GA enabled, and all we are doing is accessing a standard web page from
> the
> > Ignite web site. The information gathered from GA is available only to
> the
> > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > question.
> >
> > Even ASF website itself uses GA: https://www.apache.org/
> > foundation/policies/privacy.html
> >
> >
> >> I think Cos said it's OK; maybe Roman can pitch in.
> >>
> >
> > Sure, would be nice to hear from Roman as well.
> >
> >
> >> Cheers.
> >>
> >> [1]
> >> https://developers.google.com/analytics/devguides/collection
> >> /analyticsjs/events
> >>
> >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org <javascript:;>>
> >> wrote:
> >>
> >> > Raul,
> >> >
> >> > Could point about Javascript, it will not work because it executes in
> >> the
> >> > browser. This means we need a server-side script, like CGI we are
> using
> >> on
> >> > our download page.
> >> >
> >> > How about this approach. We create something like ignite-version.cgi
> >> script
> >> > which will invoke a call to GA and then return the latest version.
> >> >
> >> > This should work, right?
> >> >
> >> > D.
> >> >
> >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <raul.asf@evosent.com
> <javascript:;>>
> >> > wrote:
> >> >
> >> > > Hey Dmitriy and all
> >> > >
> >> > > Also, since we have GA enabled for the website, we can track how
> many
> >> > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > >
> >> > >
> >> > > Unfortunately this won't work because GA is loaded via Javascript on
> >> the
> >> > > browser, so Google will never receive the page hit.
> >> > >
> >> > > Given the constraints, the most viable solution is an HTTPS endpoint
> >> > > running on ASF infra that Ignite invokes via a GET or POST request.
> >> The
> >> > > simplest thing is to write each request in a log file, along with
> the
> >> > > timestamp, the version reported by the client, maybe the IP (not
> sure
> >> > about
> >> > > the ASF rules about this concerning privacy, I guess it's OK if you
> >> make
> >> > it
> >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> creates
> >> on
> >> > > first startup or something.
> >> > >
> >> > > This endpoint would need some basic DDoS protection and blacklisting
> >> to
> >> > > prevent data spoofing.
> >> > >
> >> > > If we'll be implementing this endpoint anyway, then there's no point
> >> > > placing another file on Git or elsewhere for reporting the latest
> >> > versions:
> >> > > the endpoint itself can return them.
> >> > >
> >> > > WDYT?
> >> > >
> >> > > Cheers.
> >> > >
> >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >> > dsetrakyan@apache.org <javascript:;>>
> >> > > wrote:
> >> > >
> >> > > > Cos, Raul,
> >> > > >
> >> > > > Thanks for the feedback. I completely agree about Maven Central
> >> being a
> >> > > 3rd
> >> > > > party repo (did not think about that initially). All your
> >> suggestions
> >> > > make
> >> > > > sense, but I would like to keep it as simple as possible, and so
> far
> >> > > > everything suggested required GIT dependencies and extra work.
> >> > > >
> >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> >> > website
> >> > > > which will have only the latest version. Every time a node starts,
> >> it
> >> > > > receives the latest version from the page and suggests that users
> >> > upgrade
> >> > > > if needed.
> >> > > >
> >> > > > Also, since we have GA enabled for the website, we can track how
> >> many
> >> > > times
> >> > > > this page was accessed, which will be equal to the number of
> starts.
> >> > This
> >> > > > way, the counter information is tracked and monitored by the
> Ignite
> >> > PMC.
> >> > > >
> >> > > > This approach looks pretty innocent to me and everything is kept
> and
> >> > > > managed within Apache.
> >> > > >
> >> > > > Thoughts?
> >> > > >
> >> > > > D.
> >> > > >
> >> > > >
> >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >> cos@apache.org <javascript:;>>
> >> > > > wrote:
> >> > > >
> >> > > > > I agree with Raul.
> >> > > > > - providing a ping-back address to a 3rd party might be frown
> >> upon by
> >> > > > some.
> >> > > > > And might have a consequences like stats collection about
> users'
> >> > > > > infrastructure.
> >> > > > > - checking an ASF git-repo is easy and won't download any binary
> >> > data:
> >> > > > > everything is clear text and could be easily monitored by any
> of
> >> > the
> >> > > > > network
> >> > > > > diagnostic tools, shall it be required. But it involves a bit
> of
> >> > the
> >> > > > > release
> >> > > > > discipline.
> >> > > > > - the binary data download in the runtime is my main concern.
> >> This is
> >> > > the
> >> > > > > vector of MMA. What if someone gains the control over the
> >> > repository
> >> > > > and
> >> > > > > replaces the file with some malicious content.
> >> > > > >
> >> > > > > As for the particular mechanism: IIRC Ignite used to make a call
> >> to
> >> > an
> >> > > > > external script to check upon the atest software version
> available
> >> > for
> >> > > > > download. In the past, the endpoint was running on a 3rd party
> >> > server,
> >> > > I
> >> > > > > believe the best way would be to put this script on ASF infra
> and
> >> > have
> >> > > > the
> >> > > > > "update checker" running in a completely controlled environment.
> >> > > > Actually,
> >> > > > > I
> >> > > > > recall we had this very discussion during the Incubation; I can
> >> > > probably
> >> > > > > dig
> >> > > > > out the corresponding thread.
> >> > > > >
> >> > > > > Thoughts?
> >> > > > > Cok
> >> > > > >
> >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >> > > > > > Hey guys
> >> > > > > >
> >> > > > > > In my opinion, maven.org is still owned by a third party
> >> > (Sonatype).
> >> > > > We
> >> > > > > > don't know what kind of data analysis or intelligence
> extraction
> >> > they
> >> > > > > run.
> >> > > > > >
> >> > > > > > If Ignite servers all over the world were hitting maven.org
> >> > > > periodically
> >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a clear
> >> > > > indication
> >> > > > > > of who is running an Ignite server.
> >> > > > > >
> >> > > > > > They could reverse lookup the IP address and find out what
> >> > > corporation
> >> > > > it
> >> > > > > > is.
> >> > > > > >
> >> > > > > > How about having Ignite check the ASF Git directly?
> >> > > > > >
> >> > > > > > We could use the Git ssh API, but that would require a new
> >> > > dependency,
> >> > > > > > which I advise against.
> >> > > > > >
> >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> >> tags:
> >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >> > > > > >
> >> > > > > > Another option is to place a txt file in the root of the
> master
> >> > > branch
> >> > > > > (e.g
> >> > > > > > LATEST), containing a list of the latest GA versions for each
> >> major
> >> > > > > version
> >> > > > > > line (1.x, 2.x).
> >> > > > > >
> >> > > > > > I would advocate this last option, but it requires somebody
> >> > > remembering
> >> > > > > to
> >> > > > > > update the file with every release, unless we automate it
> with a
> >> > > Maven
> >> > > > > > plugin.
> >> > > > > >
> >> > > > > > Hope that helps!
> >> > > > > >
> >> > > > > >
> >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> <javascript:;>> wrote:
> >> > > > > >
> >> > > > > > I see nothing wrong with this approach.
> >> > > > > >
> >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is it
> >> good
> >> > > to
> >> > > > > go?
> >> > > > > >
> >> > > > > > —
> >> > > > > > Denis
> >> > > > > >
> >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >> > > > dsetrakyan@apache.org <javascript:;>
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > >
> >> > > > > > > Is everyone OK with this approach? Should I file a ticket on
> >> it?
> >> > > > > > >
> >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >> > > > > dsetrakyan@apache.org <javascript:;>>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > >> Igniters,
> >> > > > > > >>
> >> > > > > > >> There has been lots of talk of proposals about various
> usage
> >> > > metrics
> >> > > > > for
> >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> the
> >> > topic
> >> > > > and
> >> > > > > > >> propose something very simple and non-intrusive.
> >> > > > > > >>
> >> > > > > > >> 1. Update Checker
> >> > > > > > >> The main purpose of the update checker is not to collect
> >> > metrics,
> >> > > > but
> >> > > > > to
> >> > > > > > >> notify users about a new version of Ignite by accessing
> >> > maven.org
> >> > > > and
> >> > > > > > >> getting the version out of the metadata file:
> >> > > > > > >> http://repo2.maven.org/maven2/
> org/apache/ignite/ignite-core/
> >> > > > > > >> maven-metadata.xml
> >> > > > > > >>
> >> > > > > > >> This way we do not send any information anywhere and, at
> the
> >> > same
> >> > > > > time,
> >> > > > > > >> urge our users to download and start using the latest
> >> version of
> >> > > > > Ignite.
> >> > > > > > >>
> >> > > > > > >> 2. Startup Counter
> >> > > > > > >> This piece is optional, but we can also get an insight in
> how
> >> > many
> >> > > > > times
> >> > > > > > a
> >> > > > > > >> certain Ignite release gets started. This is just a cool
> >> metric
> >> > > for
> >> > > > > the
> >> > > > > > >> community to gauge the project popularity. You can think of
> >> it
> >> > as
> >> > > > of a
> >> > > > > > page
> >> > > > > > >> visit counter shown on many websites. We can even decide to
> >> > > display
> >> > > > > this
> >> > > > > > >> counter on the Ignite website as well.
> >> > > > > > >>
> >> > > > > > >> To do this, we can simply add a JAR in maven for every
> >> release,
> >> > > e.g.
> >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> >> Every
> >> > > time
> >> > > > > an
> >> > > > > > >> Ignite node starts, it will download this JAR in the
> >> background.
> >> > > > Then
> >> > > > > we
> >> > > > > > >> will be able to view the number of the total downloads for
> >> this
> >> > > JAR
> >> > > > in
> >> > > > > > >> Maven Central, which is essentially the number of starts of
> >> > Ignite
> >> > > > > nodes.
> >> > > > > > >>
> >> > > > > > >> *Note that neither of the above suggestions require Ignite
> to
> >> > send
> >> > > > or
> >> > > > > > >> track any user information whatsoever.*
> >> > > > > > >>
> >> > > > > > >> Please reply suggesting weather you are OK with this
> >> approach.
> >> > > > > > >>
> >> > > > > > >> D.
> >> > > > > > >>
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>
Re: [DISCUSS] Ignite Update Checker
Posted by Denis Magda <dm...@apache.org>.
Cos,
I’m not sure the redirect will suit us. Remember the update notifier should do 2 things:
1. get the most recent version of Ignite stored in the file on the site [1].
2. Trigger GA.
The redirect does 2. but skips 1. Any technical tricks we can apply here?
[1] https://ignite.apache.org/latest
—
Denis
> On Oct 31, 2017, at 1:23 PM, Konstantin Boudnik <co...@boudnik.org> wrote:
>
> Yes, that what I meant. It might not work per website (depends on how
> the server is configured), but server-wide redirects could be
> arranged.
> Also, there's a new discussion on the topic going on legal-discuss@
> right now [1]
>
>
> [1] https://is.gd/RPaWQt
> --
> With regards,
> Konstantin (Cos) Boudnik
>
>
> On Tue, Oct 31, 2017 at 1:21 PM, Dmitriy Setrakyan
> <ds...@apache.org> wrote:
>> Cos, thanks for helping out! By redirect, do you mean changing the
>> .htaccess file to point a certain URL containing Ignite version to a GA
>> URL? In that case, do you know if it will be possible to still send the
>> latest Ignite version back as a response?
>>
>> D.
>>
>> On Tue, Oct 31, 2017 at 11:30 AM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>>> Looks like we are getting somewhere with this. Please check [1] for
>>> the latest comment from the Infra team. I guess we can start with
>>> asking for a server-side redirect to GA as the immediate step. And
>>> explore it further possibilities as per Greg's recommendations.
>>>
>>>
>>> [1] https://is.gd/CL88O6
>>> --
>>> With regards,
>>> Konstantin (Cos) Boudnik
>>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>>>
>>> Disclaimer: Opinions expressed in this email are those of the author,
>>> and do not necessarily represent the views of any company the author
>>> might be affiliated with at the moment of writing.
>>>
>>>
>>> On Wed, Oct 25, 2017 at 8:21 AM, Denis Magda <dm...@apache.org> wrote:
>>>> Hi Cos,
>>>>
>>>> Sure, will see you around there.
>>>>
>>>> Anyway, a short summary is the following.
>>>>
>>>> Starting Ignite 2.3 the nodes will be connecting to a *static* file [1]
>>> deployed on Ignite site to obtain the most recent version. If the file has
>>> a later version than the nodes will print out a message encouraging a user
>>> to do an upgrade.
>>>>
>>>> On top of that, some community members proposed to trigger GA once the
>>> file [1] is loaded. But to achieve this the file has to become dynamic
>>> being able to execute simple CGI scripts. The INFRA turned this request
>>> down.
>>>>
>>>> [1] https://ignite.apache.org/latest
>>>> [2] https://issues.apache.org/jira/browse/INFRA-15182
>>>>
>>>> —
>>>> Denis
>>>>
>>>>> On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <co...@apache.org>
>>> wrote:
>>>>>
>>>>> Guys,
>>>>>
>>>>> I had a quick chat with Nikita earlier today and it seems that we have
>>> came across of a blocker of some kind. With my member's hat on I would love
>>> to help to find a resolution that let us (and potentially other
>>> communities) to move forward.
>>>>>
>>>>> Let's craft the message and circulate it with Infra and Greg. Denis,
>>> could you find me tomorrow at the IMCS and give me an insight into the
>>> technical side of it? I feel a bit at loss after going through this
>>> thread...
>>>>> --
>>>>> Regards,
>>>>> Cos
>>>>>
>>>>> On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org>
>>> wrote:
>>>>>> Dmitriy,
>>>>>>
>>>>>> That’s the rule. See replies in the ticket [1]
>>>>>>
>>>>>> *Background: the TLP server is already pretty darned busy just serving
>>>>>> *static* sites. Dynamic operation for near-200 PMCs would bury the
>>>>>> machine. Our policy of "static-only websites" has been in place since
>>>>>> the Foundation started*
>>>>>>
>>>>>> Download scripts seem to be the only exception and we as PMC don’t even
>>>>>> have access to them.
>>>>>>
>>>>>> If you want to keep pushing this direction let’s craft a message to
>>>>>> Greg and Daniel directly. I don’t know what else to ask for here rather
>>>>>> than a virtual machine that’s conceivably to much for a single script
>>>>>> like that.
>>>>>>
>>>>>> [1] https://issues.apache.org/jira/browse/INFRA-15182
>>>>>> <https://issues.apache.org/jira/browse/INFRA-15182>
>>>>>>
>>>>>> —
>>>>>> Denis
>>>>>>
>>>>>>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> wrote:
>>>>>>>
>>>>>>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
>>>>>> <vo...@gridgain.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I am not sure it is good idea to send requests to 3rd-party
>>>>>> addresses from
>>>>>>>> Ignite node. Let's do not make the same mistakes again.
>>>>>>>>
>>>>>>>
>>>>>>> Agree with Vladimir.
>>>>>>>
>>>>>>> We obviously have CGI support on the website. Can someone explain why
>>>>>> CGI
>>>>>>> is not possible to use?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
>>>>>> <an...@gridgain.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> We may directly send request to GA from Ignite node:
>>>>>>>>> https://developers.google.com/analytics/devguides/
>>>>>>>> collection/protocol/v1/
>>>>>>>>> <https://developers.google.com/analytics/devguides/
>>>>>>>> collection/protocol/v1/
>>>>>>>>>>
>>>>>>>>> Latest version can be received from maven central:
>>>>>>>>> https://repo1.maven.org/maven2/org/apache/ignite/
>>>>>>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>>>>>>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>>>>>>>>> написал(а):
>>>>>>>>>>
>>>>>>>>>> Denis,
>>>>>>>>>>
>>>>>>>>>> I am not sure I understand. We already do have CGI enabled for
>>>>>>>>>> download.cgi. Is there something else we need?
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> There is an obstacle. There is no way to execute the script using
>>>>>> PHP
>>>>>>>> or
>>>>>>>>>>> similar sever side language and trigger GA as discussed earlier:
>>>>>>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>>>>>>>>>>>
>>>>>>>>>>> How else can we tackle this?
>>>>>>>>>>>
>>>>>>>>>>> Denis
>>>>>>>>>>>
>>>>>>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I think it is safe to assume at this point that everyone is in
>>>>>>>> general
>>>>>>>>>>>> agreement, since there are no active objections.
>>>>>>>>>>>>
>>>>>>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>>>>>>>> happen:
>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>>>>>>>>>>
>>>>>>>>>>>> D.
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>>>>>>>>>>> dsetrakyan@apache.org
>>>>>>>>>>>> <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>>>>>>>> raul.asf@evosent.com
>>>>>>>>>>>> <javascript:;>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Yeah, I guess that's doable as well and requires less
>>>>>> management
>>>>>>>>>>> effort
>>>>>>>>>>>>>> than my suggestion. We could use events [1] to store payload
>>>>>> data
>>>>>>>>>>> (e.g.
>>>>>>>>>>>>>> IP,
>>>>>>>>>>>>>> version, etc.)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yes, we could use events or some other similar API provided by
>>>>>> GA.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> What the download page CGI developed in? PHP?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> To be honest, no clue. I guess someone in the community can
>>>>>> figure
>>>>>>>> it
>>>>>>>>>>>> out:
>>>>>>>>>>>>>
>>>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>>>>>>>>>>> (Google)
>>>>>>>>>>>> is
>>>>>>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>>>>>>>> there's
>>>>>>>>>>>>>> doubt
>>>>>>>>>>>>>> in the PMC, it's better to ask legal@.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>>>>>>>> website
>>>>>>>>>>> is
>>>>>>>>>>>>> GA enabled, and all we are doing is accessing a standard web
>>>>>> page
>>>>>>>> from
>>>>>>>>>>>> the
>>>>>>>>>>>>> Ignite web site. The information gathered from GA is available
>>>>>> only
>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
>>>>>> this
>>>>>>>>>>>>> question.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>>>>>>>>>>> foundation/policies/privacy.html
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Sure, would be nice to hear from Roman as well.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>> https://developers.google.com/analytics/devguides/collection
>>>>>>>>>>>>>> /analyticsjs/events
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Raul,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Could point about Javascript, it will not work because it
>>>>>> executes
>>>>>>>>>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> browser. This means we need a server-side script, like CGI we
>>>>>> are
>>>>>>>>>>>> using
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>> our download page.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How about this approach. We create something like
>>>>>>>> ignite-version.cgi
>>>>>>>>>>>>>> script
>>>>>>>>>>>>>>> which will invoke a call to GA and then return the latest
>>>>>> version.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This should work, right?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>>>>>>>>>>> raul.asf@evosent.com
>>>>>>>>>>>> <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hey Dmitriy and all
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
>>>>>> how
>>>>>>>>>>>> many
>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>>>>>> of
>>>>>>>>>>>> starts.
>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>>>>>> the
>>>>>>>>>>>> Ignite
>>>>>>>>>>>>>>> PMC.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
>>>>>> Javascript
>>>>>>>>>>> on
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>>>>>>>>>>> endpoint
>>>>>>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>>>>>>>>>>> request.
>>>>>>>>>>>>>> The
>>>>>>>>>>>>>>>> simplest thing is to write each request in a log file, along
>>>>>> with
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
>>>>>> (not
>>>>>>>>>>>> sure
>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
>>>>>> if
>>>>>>>>>>> you
>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
>>>>>> node
>>>>>>>>>>>> creates
>>>>>>>>>>>>>> on
>>>>>>>>>>>>>>>> first startup or something.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This endpoint would need some basic DDoS protection and
>>>>>>>>>>> blacklisting
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> prevent data spoofing.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
>>>>>> no
>>>>>>>>>>> point
>>>>>>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
>>>>>> latest
>>>>>>>>>>>>>>> versions:
>>>>>>>>>>>>>>>> the endpoint itself can return them.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cheers.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Cos, Raul,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
>>>>>> Central
>>>>>>>>>>>>>> being a
>>>>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>>>>> party repo (did not think about that initially). All your
>>>>>>>>>>>>>> suggestions
>>>>>>>>>>>>>>>> make
>>>>>>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
>>>>>> and so
>>>>>>>>>>>> far
>>>>>>>>>>>>>>>>> everything suggested required GIT dependencies and extra
>>>>>> work.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
>>>>>> Ignite
>>>>>>>>>>>>>>> website
>>>>>>>>>>>>>>>>> which will have only the latest version. Every time a node
>>>>>>>>>>> starts,
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>> receives the latest version from the page and suggests that
>>>>>>>>>>> users
>>>>>>>>>>>>>>> upgrade
>>>>>>>>>>>>>>>>> if needed.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
>>>>>> track how
>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>>>>>> of
>>>>>>>>>>>> starts.
>>>>>>>>>>>>>>> This
>>>>>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>>>>>> the
>>>>>>>>>>>> Ignite
>>>>>>>>>>>>>>> PMC.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
>>>>>> kept
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> managed within Apache.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>>>>>>>>>>> cos@apache.org <javascript:;>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I agree with Raul.
>>>>>>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
>>>>>> frown
>>>>>>>>>>>>>> upon by
>>>>>>>>>>>>>>>>> some.
>>>>>>>>>>>>>>>>>> And might have a consequences like stats collection about
>>>>>>>>>>>> users'
>>>>>>>>>>>>>>>>>> infrastructure.
>>>>>>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>>>>>>>>>>> binary
>>>>>>>>>>>>>>> data:
>>>>>>>>>>>>>>>>>> everything is clear text and could be easily monitored by
>>>>>>>>>>> any
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>>>>>>>>>>> bit
>>>>>>>>>>>> of
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>> discipline.
>>>>>>>>>>>>>>>>>> - the binary data download in the runtime is my main
>>>>>> concern.
>>>>>>>>>>>>>> This is
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>>>>>>>>>>> repository
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
>>>>>> a
>>>>>>>>>>> call
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> external script to check upon the atest software version
>>>>>>>>>>>> available
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
>>>>>> party
>>>>>>>>>>>>>>> server,
>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> believe the best way would be to put this script on ASF
>>>>>> infra
>>>>>>>>>>>> and
>>>>>>>>>>>>>>> have
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> "update checker" running in a completely controlled
>>>>>>>>>>> environment.
>>>>>>>>>>>>>>>>> Actually,
>>>>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
>>>>>> I
>>>>>>>>>>> can
>>>>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>>>>> dig
>>>>>>>>>>>>>>>>>> out the corresponding thread.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>>>>> Cok
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>>>>>>>>>>> Hey guys
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>>>>>>>>>>> (Sonatype).
>>>>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>>>>>>>>>>>> extraction
>>>>>>>>>>>>>>> they
>>>>>>>>>>>>>>>>>> run.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
>>>>>> maven.org
>>>>>>>>>>>>>>>>> periodically
>>>>>>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>>>>>>>>>>> clear
>>>>>>>>>>>>>>>>> indication
>>>>>>>>>>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
>>>>>> what
>>>>>>>>>>>>>>>> corporation
>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>> is.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
>>>>>> new
>>>>>>>>>>>>>>>> dependency,
>>>>>>>>>>>>>>>>>>> which I advise against.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
>>>>>> Git
>>>>>>>>>>>>>> tags:
>>>>>>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>>>>>>>>>>>> master
>>>>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>>>>> (e.g
>>>>>>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>>>>>>>>>>> each
>>>>>>>>>>>>>> major
>>>>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I would advocate this last option, but it requires
>>>>>> somebody
>>>>>>>>>>>>>>>> remembering
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>> update the file with every release, unless we automate it
>>>>>>>>>>>> with a
>>>>>>>>>>>>>>>> Maven
>>>>>>>>>>>>>>>>>>> plugin.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>>>>>>>>>>> <javascript:;>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
>>>>>> Is
>>>>>>>>>>> it
>>>>>>>>>>>>>> good
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> go?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
>>>>>> ticket
>>>>>>>>>>> on
>>>>>>>>>>>>>> it?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>>>>>>>>>>>> usage
>>>>>>>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
>>>>>> resurrect
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> topic
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
>>>>>> collect
>>>>>>>>>>>>>>> metrics,
>>>>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>>>>>>>>>>> maven.org
>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>>>>>>>>>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
>>>>>> at
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> same
>>>>>>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>>>>>> urge our users to download and start using the latest
>>>>>>>>>>>>>> version of
>>>>>>>>>>>>>>>>>> Ignite.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
>>>>>> in
>>>>>>>>>>>> how
>>>>>>>>>>>>>>> many
>>>>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
>>>>>> cool
>>>>>>>>>>>>>> metric
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
>>>>>> think
>>>>>>>>>>> of
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>>>>> page
>>>>>>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
>>>>>> decide
>>>>>>>>>>> to
>>>>>>>>>>>>>>>> display
>>>>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>>>>>>>>>>> release,
>>>>>>>>>>>>>>>> e.g.
>>>>>>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
>>>>>> byte.
>>>>>>>>>>>>>> Every
>>>>>>>>>>>>>>>> time
>>>>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>>>>>>>>>>> background.
>>>>>>>>>>>>>>>>> Then
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>>>>>>>>>>> for
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> JAR
>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
>>>>>> starts
>>>>>>>>>>> of
>>>>>>>>>>>>>>> Ignite
>>>>>>>>>>>>>>>>>> nodes.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>>>>>>>>>>> Ignite
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> send
>>>>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>
>>>
Re: [DISCUSS] Ignite Update Checker
Posted by Konstantin Boudnik <co...@boudnik.org>.
Yes, that what I meant. It might not work per website (depends on how
the server is configured), but server-wide redirects could be
arranged.
Also, there's a new discussion on the topic going on legal-discuss@
right now [1]
[1] https://is.gd/RPaWQt
--
With regards,
Konstantin (Cos) Boudnik
On Tue, Oct 31, 2017 at 1:21 PM, Dmitriy Setrakyan
<ds...@apache.org> wrote:
> Cos, thanks for helping out! By redirect, do you mean changing the
> .htaccess file to point a certain URL containing Ignite version to a GA
> URL? In that case, do you know if it will be possible to still send the
> latest Ignite version back as a response?
>
> D.
>
> On Tue, Oct 31, 2017 at 11:30 AM, Konstantin Boudnik <co...@apache.org> wrote:
>
>> Looks like we are getting somewhere with this. Please check [1] for
>> the latest comment from the Infra team. I guess we can start with
>> asking for a server-side redirect to GA as the immediate step. And
>> explore it further possibilities as per Greg's recommendations.
>>
>>
>> [1] https://is.gd/CL88O6
>> --
>> With regards,
>> Konstantin (Cos) Boudnik
>> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>>
>> Disclaimer: Opinions expressed in this email are those of the author,
>> and do not necessarily represent the views of any company the author
>> might be affiliated with at the moment of writing.
>>
>>
>> On Wed, Oct 25, 2017 at 8:21 AM, Denis Magda <dm...@apache.org> wrote:
>> > Hi Cos,
>> >
>> > Sure, will see you around there.
>> >
>> > Anyway, a short summary is the following.
>> >
>> > Starting Ignite 2.3 the nodes will be connecting to a *static* file [1]
>> deployed on Ignite site to obtain the most recent version. If the file has
>> a later version than the nodes will print out a message encouraging a user
>> to do an upgrade.
>> >
>> > On top of that, some community members proposed to trigger GA once the
>> file [1] is loaded. But to achieve this the file has to become dynamic
>> being able to execute simple CGI scripts. The INFRA turned this request
>> down.
>> >
>> > [1] https://ignite.apache.org/latest
>> > [2] https://issues.apache.org/jira/browse/INFRA-15182
>> >
>> > —
>> > Denis
>> >
>> >> On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <co...@apache.org>
>> wrote:
>> >>
>> >> Guys,
>> >>
>> >> I had a quick chat with Nikita earlier today and it seems that we have
>> came across of a blocker of some kind. With my member's hat on I would love
>> to help to find a resolution that let us (and potentially other
>> communities) to move forward.
>> >>
>> >> Let's craft the message and circulate it with Infra and Greg. Denis,
>> could you find me tomorrow at the IMCS and give me an insight into the
>> technical side of it? I feel a bit at loss after going through this
>> thread...
>> >> --
>> >> Regards,
>> >> Cos
>> >>
>> >> On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org>
>> wrote:
>> >>> Dmitriy,
>> >>>
>> >>> That’s the rule. See replies in the ticket [1]
>> >>>
>> >>> *Background: the TLP server is already pretty darned busy just serving
>> >>> *static* sites. Dynamic operation for near-200 PMCs would bury the
>> >>> machine. Our policy of "static-only websites" has been in place since
>> >>> the Foundation started*
>> >>>
>> >>> Download scripts seem to be the only exception and we as PMC don’t even
>> >>> have access to them.
>> >>>
>> >>> If you want to keep pushing this direction let’s craft a message to
>> >>> Greg and Daniel directly. I don’t know what else to ask for here rather
>> >>> than a virtual machine that’s conceivably to much for a single script
>> >>> like that.
>> >>>
>> >>> [1] https://issues.apache.org/jira/browse/INFRA-15182
>> >>> <https://issues.apache.org/jira/browse/INFRA-15182>
>> >>>
>> >>> —
>> >>> Denis
>> >>>
>> >>>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
>> >>> wrote:
>> >>>>
>> >>>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
>> >>> <vo...@gridgain.com>
>> >>>> wrote:
>> >>>>
>> >>>>> I am not sure it is good idea to send requests to 3rd-party
>> >>> addresses from
>> >>>>> Ignite node. Let's do not make the same mistakes again.
>> >>>>>
>> >>>>
>> >>>> Agree with Vladimir.
>> >>>>
>> >>>> We obviously have CGI support on the website. Can someone explain why
>> >>> CGI
>> >>>> is not possible to use?
>> >>>>
>> >>>>
>> >>>>>
>> >>>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
>> >>> <an...@gridgain.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> We may directly send request to GA from Ignite node:
>> >>>>>> https://developers.google.com/analytics/devguides/
>> >>>>> collection/protocol/v1/
>> >>>>>> <https://developers.google.com/analytics/devguides/
>> >>>>> collection/protocol/v1/
>> >>>>>>>
>> >>>>>> Latest version can be received from maven central:
>> >>>>>> https://repo1.maven.org/maven2/org/apache/ignite/
>> >>>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>> >>>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>> >>>>>>
>> >>>>>>
>> >>>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>> >>>>>> написал(а):
>> >>>>>>>
>> >>>>>>> Denis,
>> >>>>>>>
>> >>>>>>> I am not sure I understand. We already do have CGI enabled for
>> >>>>>>> download.cgi. Is there something else we need?
>> >>>>>>>
>> >>>>>>> D.
>> >>>>>>>
>> >>>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>> >>>>> wrote:
>> >>>>>>>
>> >>>>>>>> There is an obstacle. There is no way to execute the script using
>> >>> PHP
>> >>>>> or
>> >>>>>>>> similar sever side language and trigger GA as discussed earlier:
>> >>>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>> >>>>>>>>
>> >>>>>>>> How else can we tackle this?
>> >>>>>>>>
>> >>>>>>>> Denis
>> >>>>>>>>
>> >>>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>> >>>>>> dsetrakyan@apache.org>
>> >>>>>>>> wrote:
>> >>>>>>>>
>> >>>>>>>>> I think it is safe to assume at this point that everyone is in
>> >>>>> general
>> >>>>>>>>> agreement, since there are no active objections.
>> >>>>>>>>>
>> >>>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>> >>>>> happen:
>> >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>> >>>>>>>>>
>> >>>>>>>>> D.
>> >>>>>>>>>
>> >>>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>> >>>>>>>> dsetrakyan@apache.org
>> >>>>>>>>> <javascript:;>>
>> >>>>>>>>> wrote:
>> >>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>> >>>>> raul.asf@evosent.com
>> >>>>>>>>> <javascript:;>>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>>> Yeah, I guess that's doable as well and requires less
>> >>> management
>> >>>>>>>> effort
>> >>>>>>>>>>> than my suggestion. We could use events [1] to store payload
>> >>> data
>> >>>>>>>> (e.g.
>> >>>>>>>>>>> IP,
>> >>>>>>>>>>> version, etc.)
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Yes, we could use events or some other similar API provided by
>> >>> GA.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> What the download page CGI developed in? PHP?
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> To be honest, no clue. I guess someone in the community can
>> >>> figure
>> >>>>> it
>> >>>>>>>>> out:
>> >>>>>>>>>>
>> >>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>> >>>>>>>> (Google)
>> >>>>>>>>> is
>> >>>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>> >>>>> there's
>> >>>>>>>>>>> doubt
>> >>>>>>>>>>> in the PMC, it's better to ask legal@.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>> >>>>> website
>> >>>>>>>> is
>> >>>>>>>>>> GA enabled, and all we are doing is accessing a standard web
>> >>> page
>> >>>>> from
>> >>>>>>>>> the
>> >>>>>>>>>> Ignite web site. The information gathered from GA is available
>> >>> only
>> >>>>> to
>> >>>>>>>>> the
>> >>>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
>> >>> this
>> >>>>>>>>>> question.
>> >>>>>>>>>>
>> >>>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>> >>>>>>>>>> foundation/policies/privacy.html
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> Sure, would be nice to hear from Roman as well.
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>> Cheers.
>> >>>>>>>>>>>
>> >>>>>>>>>>> [1]
>> >>>>>>>>>>> https://developers.google.com/analytics/devguides/collection
>> >>>>>>>>>>> /analyticsjs/events
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>> >>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Raul,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Could point about Javascript, it will not work because it
>> >>> executes
>> >>>>>>>> in
>> >>>>>>>>>>> the
>> >>>>>>>>>>>> browser. This means we need a server-side script, like CGI we
>> >>> are
>> >>>>>>>>> using
>> >>>>>>>>>>> on
>> >>>>>>>>>>>> our download page.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> How about this approach. We create something like
>> >>>>> ignite-version.cgi
>> >>>>>>>>>>> script
>> >>>>>>>>>>>> which will invoke a call to GA and then return the latest
>> >>> version.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> This should work, right?
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> D.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>> >>>>>>>> raul.asf@evosent.com
>> >>>>>>>>> <javascript:;>>
>> >>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>> Hey Dmitriy and all
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
>> >>> how
>> >>>>>>>>> many
>> >>>>>>>>>>>> times
>> >>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>> >>> of
>> >>>>>>>>> starts.
>> >>>>>>>>>>>> This
>> >>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>> >>> the
>> >>>>>>>>> Ignite
>> >>>>>>>>>>>> PMC.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
>> >>> Javascript
>> >>>>>>>> on
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>> browser, so Google will never receive the page hit.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>> >>>>>>>> endpoint
>> >>>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>> >>>>>>>> request.
>> >>>>>>>>>>> The
>> >>>>>>>>>>>>> simplest thing is to write each request in a log file, along
>> >>> with
>> >>>>>>>>> the
>> >>>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
>> >>> (not
>> >>>>>>>>> sure
>> >>>>>>>>>>>> about
>> >>>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
>> >>> if
>> >>>>>>>> you
>> >>>>>>>>>>> make
>> >>>>>>>>>>>> it
>> >>>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
>> >>> node
>> >>>>>>>>> creates
>> >>>>>>>>>>> on
>> >>>>>>>>>>>>> first startup or something.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> This endpoint would need some basic DDoS protection and
>> >>>>>>>> blacklisting
>> >>>>>>>>>>> to
>> >>>>>>>>>>>>> prevent data spoofing.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
>> >>> no
>> >>>>>>>> point
>> >>>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
>> >>> latest
>> >>>>>>>>>>>> versions:
>> >>>>>>>>>>>>> the endpoint itself can return them.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> WDYT?
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Cheers.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>> >>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>> >>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Cos, Raul,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
>> >>> Central
>> >>>>>>>>>>> being a
>> >>>>>>>>>>>>> 3rd
>> >>>>>>>>>>>>>> party repo (did not think about that initially). All your
>> >>>>>>>>>>> suggestions
>> >>>>>>>>>>>>> make
>> >>>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
>> >>> and so
>> >>>>>>>>> far
>> >>>>>>>>>>>>>> everything suggested required GIT dependencies and extra
>> >>> work.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
>> >>> Ignite
>> >>>>>>>>>>>> website
>> >>>>>>>>>>>>>> which will have only the latest version. Every time a node
>> >>>>>>>> starts,
>> >>>>>>>>>>> it
>> >>>>>>>>>>>>>> receives the latest version from the page and suggests that
>> >>>>>>>> users
>> >>>>>>>>>>>> upgrade
>> >>>>>>>>>>>>>> if needed.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
>> >>> track how
>> >>>>>>>>>>> many
>> >>>>>>>>>>>>> times
>> >>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>> >>> of
>> >>>>>>>>> starts.
>> >>>>>>>>>>>> This
>> >>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>> >>> the
>> >>>>>>>>> Ignite
>> >>>>>>>>>>>> PMC.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
>> >>> kept
>> >>>>>>>>> and
>> >>>>>>>>>>>>>> managed within Apache.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Thoughts?
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> D.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>> >>>>>>>>>>> cos@apache.org <javascript:;>>
>> >>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I agree with Raul.
>> >>>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
>> >>> frown
>> >>>>>>>>>>> upon by
>> >>>>>>>>>>>>>> some.
>> >>>>>>>>>>>>>>> And might have a consequences like stats collection about
>> >>>>>>>>> users'
>> >>>>>>>>>>>>>>> infrastructure.
>> >>>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>> >>>>>>>> binary
>> >>>>>>>>>>>> data:
>> >>>>>>>>>>>>>>> everything is clear text and could be easily monitored by
>> >>>>>>>> any
>> >>>>>>>>> of
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> network
>> >>>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>> >>>>>>>> bit
>> >>>>>>>>> of
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>> discipline.
>> >>>>>>>>>>>>>>> - the binary data download in the runtime is my main
>> >>> concern.
>> >>>>>>>>>>> This is
>> >>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>> >>>>>>>>>>>> repository
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>> replaces the file with some malicious content.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
>> >>> a
>> >>>>>>>> call
>> >>>>>>>>>>> to
>> >>>>>>>>>>>> an
>> >>>>>>>>>>>>>>> external script to check upon the atest software version
>> >>>>>>>>> available
>> >>>>>>>>>>>> for
>> >>>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
>> >>> party
>> >>>>>>>>>>>> server,
>> >>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>> believe the best way would be to put this script on ASF
>> >>> infra
>> >>>>>>>>> and
>> >>>>>>>>>>>> have
>> >>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>> "update checker" running in a completely controlled
>> >>>>>>>> environment.
>> >>>>>>>>>>>>>> Actually,
>> >>>>>>>>>>>>>>> I
>> >>>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
>> >>> I
>> >>>>>>>> can
>> >>>>>>>>>>>>> probably
>> >>>>>>>>>>>>>>> dig
>> >>>>>>>>>>>>>>> out the corresponding thread.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Thoughts?
>> >>>>>>>>>>>>>>> Cok
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>> >>>>>>>>>>>>>>>> Hey guys
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>> >>>>>>>>>>>> (Sonatype).
>> >>>>>>>>>>>>>> We
>> >>>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>> >>>>>>>>> extraction
>> >>>>>>>>>>>> they
>> >>>>>>>>>>>>>>> run.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
>> >>> maven.org
>> >>>>>>>>>>>>>> periodically
>> >>>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>> >>>>>>>> clear
>> >>>>>>>>>>>>>> indication
>> >>>>>>>>>>>>>>>> of who is running an Ignite server.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
>> >>> what
>> >>>>>>>>>>>>> corporation
>> >>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>> is.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
>> >>> new
>> >>>>>>>>>>>>> dependency,
>> >>>>>>>>>>>>>>>> which I advise against.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
>> >>> Git
>> >>>>>>>>>>> tags:
>> >>>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>> >>>>>>>>> master
>> >>>>>>>>>>>>> branch
>> >>>>>>>>>>>>>>> (e.g
>> >>>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>> >>>>>>>> each
>> >>>>>>>>>>> major
>> >>>>>>>>>>>>>>> version
>> >>>>>>>>>>>>>>>> line (1.x, 2.x).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I would advocate this last option, but it requires
>> >>> somebody
>> >>>>>>>>>>>>> remembering
>> >>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>> update the file with every release, unless we automate it
>> >>>>>>>>> with a
>> >>>>>>>>>>>>> Maven
>> >>>>>>>>>>>>>>>> plugin.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Hope that helps!
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>> >>>>>>>>> <javascript:;>> wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I see nothing wrong with this approach.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
>> >>> Is
>> >>>>>>>> it
>> >>>>>>>>>>> good
>> >>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>> go?
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> —
>> >>>>>>>>>>>>>>>> Denis
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>> >>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
>> >>> ticket
>> >>>>>>>> on
>> >>>>>>>>>>> it?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>> >>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Igniters,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>> >>>>>>>>> usage
>> >>>>>>>>>>>>> metrics
>> >>>>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
>> >>> resurrect
>> >>>>>>>>> the
>> >>>>>>>>>>>> topic
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> 1. Update Checker
>> >>>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
>> >>> collect
>> >>>>>>>>>>>> metrics,
>> >>>>>>>>>>>>>> but
>> >>>>>>>>>>>>>>> to
>> >>>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>> >>>>>>>>>>>> maven.org
>> >>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> getting the version out of the metadata file:
>> >>>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>> >>>>>>>>> org/apache/ignite/ignite-core/
>> >>>>>>>>>>>>>>>>>> maven-metadata.xml
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
>> >>> at
>> >>>>>>>>> the
>> >>>>>>>>>>>> same
>> >>>>>>>>>>>>>>> time,
>> >>>>>>>>>>>>>>>>>> urge our users to download and start using the latest
>> >>>>>>>>>>> version of
>> >>>>>>>>>>>>>>> Ignite.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> 2. Startup Counter
>> >>>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
>> >>> in
>> >>>>>>>>> how
>> >>>>>>>>>>>> many
>> >>>>>>>>>>>>>>> times
>> >>>>>>>>>>>>>>>> a
>> >>>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
>> >>> cool
>> >>>>>>>>>>> metric
>> >>>>>>>>>>>>> for
>> >>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
>> >>> think
>> >>>>>>>> of
>> >>>>>>>>>>> it
>> >>>>>>>>>>>> as
>> >>>>>>>>>>>>>> of a
>> >>>>>>>>>>>>>>>> page
>> >>>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
>> >>> decide
>> >>>>>>>> to
>> >>>>>>>>>>>>> display
>> >>>>>>>>>>>>>>> this
>> >>>>>>>>>>>>>>>>>> counter on the Ignite website as well.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>> >>>>>>>>>>> release,
>> >>>>>>>>>>>>> e.g.
>> >>>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
>> >>> byte.
>> >>>>>>>>>>> Every
>> >>>>>>>>>>>>> time
>> >>>>>>>>>>>>>>> an
>> >>>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>> >>>>>>>>>>> background.
>> >>>>>>>>>>>>>> Then
>> >>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>> >>>>>>>> for
>> >>>>>>>>>>> this
>> >>>>>>>>>>>>> JAR
>> >>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
>> >>> starts
>> >>>>>>>> of
>> >>>>>>>>>>>> Ignite
>> >>>>>>>>>>>>>>> nodes.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>> >>>>>>>> Ignite
>> >>>>>>>>> to
>> >>>>>>>>>>>> send
>> >>>>>>>>>>>>>> or
>> >>>>>>>>>>>>>>>>>> track any user information whatsoever.*
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>> >>>>>>>>>>> approach.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> D.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>
>> >
>>
Re: [DISCUSS] Ignite Update Checker
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Cos, thanks for helping out! By redirect, do you mean changing the
.htaccess file to point a certain URL containing Ignite version to a GA
URL? In that case, do you know if it will be possible to still send the
latest Ignite version back as a response?
D.
On Tue, Oct 31, 2017 at 11:30 AM, Konstantin Boudnik <co...@apache.org> wrote:
> Looks like we are getting somewhere with this. Please check [1] for
> the latest comment from the Infra team. I guess we can start with
> asking for a server-side redirect to GA as the immediate step. And
> explore it further possibilities as per Greg's recommendations.
>
>
> [1] https://is.gd/CL88O6
> --
> With regards,
> Konstantin (Cos) Boudnik
> 2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
>
> Disclaimer: Opinions expressed in this email are those of the author,
> and do not necessarily represent the views of any company the author
> might be affiliated with at the moment of writing.
>
>
> On Wed, Oct 25, 2017 at 8:21 AM, Denis Magda <dm...@apache.org> wrote:
> > Hi Cos,
> >
> > Sure, will see you around there.
> >
> > Anyway, a short summary is the following.
> >
> > Starting Ignite 2.3 the nodes will be connecting to a *static* file [1]
> deployed on Ignite site to obtain the most recent version. If the file has
> a later version than the nodes will print out a message encouraging a user
> to do an upgrade.
> >
> > On top of that, some community members proposed to trigger GA once the
> file [1] is loaded. But to achieve this the file has to become dynamic
> being able to execute simple CGI scripts. The INFRA turned this request
> down.
> >
> > [1] https://ignite.apache.org/latest
> > [2] https://issues.apache.org/jira/browse/INFRA-15182
> >
> > —
> > Denis
> >
> >> On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <co...@apache.org>
> wrote:
> >>
> >> Guys,
> >>
> >> I had a quick chat with Nikita earlier today and it seems that we have
> came across of a blocker of some kind. With my member's hat on I would love
> to help to find a resolution that let us (and potentially other
> communities) to move forward.
> >>
> >> Let's craft the message and circulate it with Infra and Greg. Denis,
> could you find me tomorrow at the IMCS and give me an insight into the
> technical side of it? I feel a bit at loss after going through this
> thread...
> >> --
> >> Regards,
> >> Cos
> >>
> >> On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org>
> wrote:
> >>> Dmitriy,
> >>>
> >>> That’s the rule. See replies in the ticket [1]
> >>>
> >>> *Background: the TLP server is already pretty darned busy just serving
> >>> *static* sites. Dynamic operation for near-200 PMCs would bury the
> >>> machine. Our policy of "static-only websites" has been in place since
> >>> the Foundation started*
> >>>
> >>> Download scripts seem to be the only exception and we as PMC don’t even
> >>> have access to them.
> >>>
> >>> If you want to keep pushing this direction let’s craft a message to
> >>> Greg and Daniel directly. I don’t know what else to ask for here rather
> >>> than a virtual machine that’s conceivably to much for a single script
> >>> like that.
> >>>
> >>> [1] https://issues.apache.org/jira/browse/INFRA-15182
> >>> <https://issues.apache.org/jira/browse/INFRA-15182>
> >>>
> >>> —
> >>> Denis
> >>>
> >>>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
> >>> wrote:
> >>>>
> >>>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
> >>> <vo...@gridgain.com>
> >>>> wrote:
> >>>>
> >>>>> I am not sure it is good idea to send requests to 3rd-party
> >>> addresses from
> >>>>> Ignite node. Let's do not make the same mistakes again.
> >>>>>
> >>>>
> >>>> Agree with Vladimir.
> >>>>
> >>>> We obviously have CGI support on the website. Can someone explain why
> >>> CGI
> >>>> is not possible to use?
> >>>>
> >>>>
> >>>>>
> >>>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
> >>> <an...@gridgain.com>
> >>>>> wrote:
> >>>>>
> >>>>>> We may directly send request to GA from Ignite node:
> >>>>>> https://developers.google.com/analytics/devguides/
> >>>>> collection/protocol/v1/
> >>>>>> <https://developers.google.com/analytics/devguides/
> >>>>> collection/protocol/v1/
> >>>>>>>
> >>>>>> Latest version can be received from maven central:
> >>>>>> https://repo1.maven.org/maven2/org/apache/ignite/
> >>>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
> >>>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> >>>>>>
> >>>>>>
> >>>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
> >>>>>> написал(а):
> >>>>>>>
> >>>>>>> Denis,
> >>>>>>>
> >>>>>>> I am not sure I understand. We already do have CGI enabled for
> >>>>>>> download.cgi. Is there something else we need?
> >>>>>>>
> >>>>>>> D.
> >>>>>>>
> >>>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
> >>>>> wrote:
> >>>>>>>
> >>>>>>>> There is an obstacle. There is no way to execute the script using
> >>> PHP
> >>>>> or
> >>>>>>>> similar sever side language and trigger GA as discussed earlier:
> >>>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
> >>>>>>>>
> >>>>>>>> How else can we tackle this?
> >>>>>>>>
> >>>>>>>> Denis
> >>>>>>>>
> >>>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
> >>>>>> dsetrakyan@apache.org>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I think it is safe to assume at this point that everyone is in
> >>>>> general
> >>>>>>>>> agreement, since there are no active objections.
> >>>>>>>>>
> >>>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
> >>>>> happen:
> >>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> >>>>>>>> dsetrakyan@apache.org
> >>>>>>>>> <javascript:;>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> >>>>> raul.asf@evosent.com
> >>>>>>>>> <javascript:;>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Yeah, I guess that's doable as well and requires less
> >>> management
> >>>>>>>> effort
> >>>>>>>>>>> than my suggestion. We could use events [1] to store payload
> >>> data
> >>>>>>>> (e.g.
> >>>>>>>>>>> IP,
> >>>>>>>>>>> version, etc.)
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Yes, we could use events or some other similar API provided by
> >>> GA.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> What the download page CGI developed in? PHP?
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> To be honest, no clue. I guess someone in the community can
> >>> figure
> >>>>> it
> >>>>>>>>> out:
> >>>>>>>>>>
> >>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
> >>>>>>>> (Google)
> >>>>>>>>> is
> >>>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
> >>>>> there's
> >>>>>>>>>>> doubt
> >>>>>>>>>>> in the PMC, it's better to ask legal@.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
> >>>>> website
> >>>>>>>> is
> >>>>>>>>>> GA enabled, and all we are doing is accessing a standard web
> >>> page
> >>>>> from
> >>>>>>>>> the
> >>>>>>>>>> Ignite web site. The information gathered from GA is available
> >>> only
> >>>>> to
> >>>>>>>>> the
> >>>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
> >>> this
> >>>>>>>>>> question.
> >>>>>>>>>>
> >>>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
> >>>>>>>>>> foundation/policies/privacy.html
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Sure, would be nice to hear from Roman as well.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>> Cheers.
> >>>>>>>>>>>
> >>>>>>>>>>> [1]
> >>>>>>>>>>> https://developers.google.com/analytics/devguides/collection
> >>>>>>>>>>> /analyticsjs/events
> >>>>>>>>>>>
> >>>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Raul,
> >>>>>>>>>>>>
> >>>>>>>>>>>> Could point about Javascript, it will not work because it
> >>> executes
> >>>>>>>> in
> >>>>>>>>>>> the
> >>>>>>>>>>>> browser. This means we need a server-side script, like CGI we
> >>> are
> >>>>>>>>> using
> >>>>>>>>>>> on
> >>>>>>>>>>>> our download page.
> >>>>>>>>>>>>
> >>>>>>>>>>>> How about this approach. We create something like
> >>>>> ignite-version.cgi
> >>>>>>>>>>> script
> >>>>>>>>>>>> which will invoke a call to GA and then return the latest
> >>> version.
> >>>>>>>>>>>>
> >>>>>>>>>>>> This should work, right?
> >>>>>>>>>>>>
> >>>>>>>>>>>> D.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> >>>>>>>> raul.asf@evosent.com
> >>>>>>>>> <javascript:;>>
> >>>>>>>>>>>> wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>> Hey Dmitriy and all
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
> >>> how
> >>>>>>>>> many
> >>>>>>>>>>>> times
> >>>>>>>>>>>>>> this page was accessed, which will be equal to the number
> >>> of
> >>>>>>>>> starts.
> >>>>>>>>>>>> This
> >>>>>>>>>>>>>> way, the counter information is tracked and monitored by
> >>> the
> >>>>>>>>> Ignite
> >>>>>>>>>>>> PMC.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
> >>> Javascript
> >>>>>>>> on
> >>>>>>>>>>> the
> >>>>>>>>>>>>> browser, so Google will never receive the page hit.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
> >>>>>>>> endpoint
> >>>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
> >>>>>>>> request.
> >>>>>>>>>>> The
> >>>>>>>>>>>>> simplest thing is to write each request in a log file, along
> >>> with
> >>>>>>>>> the
> >>>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
> >>> (not
> >>>>>>>>> sure
> >>>>>>>>>>>> about
> >>>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
> >>> if
> >>>>>>>> you
> >>>>>>>>>>> make
> >>>>>>>>>>>> it
> >>>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
> >>> node
> >>>>>>>>> creates
> >>>>>>>>>>> on
> >>>>>>>>>>>>> first startup or something.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> This endpoint would need some basic DDoS protection and
> >>>>>>>> blacklisting
> >>>>>>>>>>> to
> >>>>>>>>>>>>> prevent data spoofing.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
> >>> no
> >>>>>>>> point
> >>>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
> >>> latest
> >>>>>>>>>>>> versions:
> >>>>>>>>>>>>> the endpoint itself can return them.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cheers.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> Cos, Raul,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
> >>> Central
> >>>>>>>>>>> being a
> >>>>>>>>>>>>> 3rd
> >>>>>>>>>>>>>> party repo (did not think about that initially). All your
> >>>>>>>>>>> suggestions
> >>>>>>>>>>>>> make
> >>>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
> >>> and so
> >>>>>>>>> far
> >>>>>>>>>>>>>> everything suggested required GIT dependencies and extra
> >>> work.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
> >>> Ignite
> >>>>>>>>>>>> website
> >>>>>>>>>>>>>> which will have only the latest version. Every time a node
> >>>>>>>> starts,
> >>>>>>>>>>> it
> >>>>>>>>>>>>>> receives the latest version from the page and suggests that
> >>>>>>>> users
> >>>>>>>>>>>> upgrade
> >>>>>>>>>>>>>> if needed.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
> >>> track how
> >>>>>>>>>>> many
> >>>>>>>>>>>>> times
> >>>>>>>>>>>>>> this page was accessed, which will be equal to the number
> >>> of
> >>>>>>>>> starts.
> >>>>>>>>>>>> This
> >>>>>>>>>>>>>> way, the counter information is tracked and monitored by
> >>> the
> >>>>>>>>> Ignite
> >>>>>>>>>>>> PMC.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
> >>> kept
> >>>>>>>>> and
> >>>>>>>>>>>>>> managed within Apache.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >>>>>>>>>>> cos@apache.org <javascript:;>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I agree with Raul.
> >>>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
> >>> frown
> >>>>>>>>>>> upon by
> >>>>>>>>>>>>>> some.
> >>>>>>>>>>>>>>> And might have a consequences like stats collection about
> >>>>>>>>> users'
> >>>>>>>>>>>>>>> infrastructure.
> >>>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
> >>>>>>>> binary
> >>>>>>>>>>>> data:
> >>>>>>>>>>>>>>> everything is clear text and could be easily monitored by
> >>>>>>>> any
> >>>>>>>>> of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> network
> >>>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
> >>>>>>>> bit
> >>>>>>>>> of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>> discipline.
> >>>>>>>>>>>>>>> - the binary data download in the runtime is my main
> >>> concern.
> >>>>>>>>>>> This is
> >>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
> >>>>>>>>>>>> repository
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> replaces the file with some malicious content.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
> >>> a
> >>>>>>>> call
> >>>>>>>>>>> to
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>>> external script to check upon the atest software version
> >>>>>>>>> available
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
> >>> party
> >>>>>>>>>>>> server,
> >>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> believe the best way would be to put this script on ASF
> >>> infra
> >>>>>>>>> and
> >>>>>>>>>>>> have
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> "update checker" running in a completely controlled
> >>>>>>>> environment.
> >>>>>>>>>>>>>> Actually,
> >>>>>>>>>>>>>>> I
> >>>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
> >>> I
> >>>>>>>> can
> >>>>>>>>>>>>> probably
> >>>>>>>>>>>>>>> dig
> >>>>>>>>>>>>>>> out the corresponding thread.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>>>>> Cok
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >>>>>>>>>>>>>>>> Hey guys
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
> >>>>>>>>>>>> (Sonatype).
> >>>>>>>>>>>>>> We
> >>>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
> >>>>>>>>> extraction
> >>>>>>>>>>>> they
> >>>>>>>>>>>>>>> run.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
> >>> maven.org
> >>>>>>>>>>>>>> periodically
> >>>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
> >>>>>>>> clear
> >>>>>>>>>>>>>> indication
> >>>>>>>>>>>>>>>> of who is running an Ignite server.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
> >>> what
> >>>>>>>>>>>>> corporation
> >>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>> is.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
> >>> new
> >>>>>>>>>>>>> dependency,
> >>>>>>>>>>>>>>>> which I advise against.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
> >>> Git
> >>>>>>>>>>> tags:
> >>>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
> >>>>>>>>> master
> >>>>>>>>>>>>> branch
> >>>>>>>>>>>>>>> (e.g
> >>>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
> >>>>>>>> each
> >>>>>>>>>>> major
> >>>>>>>>>>>>>>> version
> >>>>>>>>>>>>>>>> line (1.x, 2.x).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I would advocate this last option, but it requires
> >>> somebody
> >>>>>>>>>>>>> remembering
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>> update the file with every release, unless we automate it
> >>>>>>>>> with a
> >>>>>>>>>>>>> Maven
> >>>>>>>>>>>>>>>> plugin.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Hope that helps!
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> >>>>>>>>> <javascript:;>> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I see nothing wrong with this approach.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
> >>> Is
> >>>>>>>> it
> >>>>>>>>>>> good
> >>>>>>>>>>>>> to
> >>>>>>>>>>>>>>> go?
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> —
> >>>>>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
> >>> ticket
> >>>>>>>> on
> >>>>>>>>>>> it?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Igniters,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
> >>>>>>>>> usage
> >>>>>>>>>>>>> metrics
> >>>>>>>>>>>>>>> for
> >>>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
> >>> resurrect
> >>>>>>>>> the
> >>>>>>>>>>>> topic
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 1. Update Checker
> >>>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
> >>> collect
> >>>>>>>>>>>> metrics,
> >>>>>>>>>>>>>> but
> >>>>>>>>>>>>>>> to
> >>>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
> >>>>>>>>>>>> maven.org
> >>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> getting the version out of the metadata file:
> >>>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
> >>>>>>>>> org/apache/ignite/ignite-core/
> >>>>>>>>>>>>>>>>>> maven-metadata.xml
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
> >>> at
> >>>>>>>>> the
> >>>>>>>>>>>> same
> >>>>>>>>>>>>>>> time,
> >>>>>>>>>>>>>>>>>> urge our users to download and start using the latest
> >>>>>>>>>>> version of
> >>>>>>>>>>>>>>> Ignite.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 2. Startup Counter
> >>>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
> >>> in
> >>>>>>>>> how
> >>>>>>>>>>>> many
> >>>>>>>>>>>>>>> times
> >>>>>>>>>>>>>>>> a
> >>>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
> >>> cool
> >>>>>>>>>>> metric
> >>>>>>>>>>>>> for
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
> >>> think
> >>>>>>>> of
> >>>>>>>>>>> it
> >>>>>>>>>>>> as
> >>>>>>>>>>>>>> of a
> >>>>>>>>>>>>>>>> page
> >>>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
> >>> decide
> >>>>>>>> to
> >>>>>>>>>>>>> display
> >>>>>>>>>>>>>>> this
> >>>>>>>>>>>>>>>>>> counter on the Ignite website as well.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
> >>>>>>>>>>> release,
> >>>>>>>>>>>>> e.g.
> >>>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
> >>> byte.
> >>>>>>>>>>> Every
> >>>>>>>>>>>>> time
> >>>>>>>>>>>>>>> an
> >>>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
> >>>>>>>>>>> background.
> >>>>>>>>>>>>>> Then
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
> >>>>>>>> for
> >>>>>>>>>>> this
> >>>>>>>>>>>>> JAR
> >>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
> >>> starts
> >>>>>>>> of
> >>>>>>>>>>>> Ignite
> >>>>>>>>>>>>>>> nodes.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
> >>>>>>>> Ignite
> >>>>>>>>> to
> >>>>>>>>>>>> send
> >>>>>>>>>>>>>> or
> >>>>>>>>>>>>>>>>>> track any user information whatsoever.*
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
> >>>>>>>>>>> approach.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >
>
Re: [DISCUSS] Ignite Update Checker
Posted by Konstantin Boudnik <co...@apache.org>.
Looks like we are getting somewhere with this. Please check [1] for
the latest comment from the Infra team. I guess we can start with
asking for a server-side redirect to GA as the immediate step. And
explore it further possibilities as per Greg's recommendations.
[1] https://is.gd/CL88O6
--
With regards,
Konstantin (Cos) Boudnik
2CAC 8312 4870 D885 8616 6115 220F 6980 1F27 E622
Disclaimer: Opinions expressed in this email are those of the author,
and do not necessarily represent the views of any company the author
might be affiliated with at the moment of writing.
On Wed, Oct 25, 2017 at 8:21 AM, Denis Magda <dm...@apache.org> wrote:
> Hi Cos,
>
> Sure, will see you around there.
>
> Anyway, a short summary is the following.
>
> Starting Ignite 2.3 the nodes will be connecting to a *static* file [1] deployed on Ignite site to obtain the most recent version. If the file has a later version than the nodes will print out a message encouraging a user to do an upgrade.
>
> On top of that, some community members proposed to trigger GA once the file [1] is loaded. But to achieve this the file has to become dynamic being able to execute simple CGI scripts. The INFRA turned this request down.
>
> [1] https://ignite.apache.org/latest
> [2] https://issues.apache.org/jira/browse/INFRA-15182
>
> —
> Denis
>
>> On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <co...@apache.org> wrote:
>>
>> Guys,
>>
>> I had a quick chat with Nikita earlier today and it seems that we have came across of a blocker of some kind. With my member's hat on I would love to help to find a resolution that let us (and potentially other communities) to move forward.
>>
>> Let's craft the message and circulate it with Infra and Greg. Denis, could you find me tomorrow at the IMCS and give me an insight into the technical side of it? I feel a bit at loss after going through this thread...
>> --
>> Regards,
>> Cos
>>
>> On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org> wrote:
>>> Dmitriy,
>>>
>>> That’s the rule. See replies in the ticket [1]
>>>
>>> *Background: the TLP server is already pretty darned busy just serving
>>> *static* sites. Dynamic operation for near-200 PMCs would bury the
>>> machine. Our policy of "static-only websites" has been in place since
>>> the Foundation started*
>>>
>>> Download scripts seem to be the only exception and we as PMC don’t even
>>> have access to them.
>>>
>>> If you want to keep pushing this direction let’s craft a message to
>>> Greg and Daniel directly. I don’t know what else to ask for here rather
>>> than a virtual machine that’s conceivably to much for a single script
>>> like that.
>>>
>>> [1] https://issues.apache.org/jira/browse/INFRA-15182
>>> <https://issues.apache.org/jira/browse/INFRA-15182>
>>>
>>> —
>>> Denis
>>>
>>>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
>>> wrote:
>>>>
>>>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
>>> <vo...@gridgain.com>
>>>> wrote:
>>>>
>>>>> I am not sure it is good idea to send requests to 3rd-party
>>> addresses from
>>>>> Ignite node. Let's do not make the same mistakes again.
>>>>>
>>>>
>>>> Agree with Vladimir.
>>>>
>>>> We obviously have CGI support on the website. Can someone explain why
>>> CGI
>>>> is not possible to use?
>>>>
>>>>
>>>>>
>>>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
>>> <an...@gridgain.com>
>>>>> wrote:
>>>>>
>>>>>> We may directly send request to GA from Ignite node:
>>>>>> https://developers.google.com/analytics/devguides/
>>>>> collection/protocol/v1/
>>>>>> <https://developers.google.com/analytics/devguides/
>>>>> collection/protocol/v1/
>>>>>>>
>>>>>> Latest version can be received from maven central:
>>>>>> https://repo1.maven.org/maven2/org/apache/ignite/
>>>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>>>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>>>>>>
>>>>>>
>>>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>>>>>> написал(а):
>>>>>>>
>>>>>>> Denis,
>>>>>>>
>>>>>>> I am not sure I understand. We already do have CGI enabled for
>>>>>>> download.cgi. Is there something else we need?
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>>>>> wrote:
>>>>>>>
>>>>>>>> There is an obstacle. There is no way to execute the script using
>>> PHP
>>>>> or
>>>>>>>> similar sever side language and trigger GA as discussed earlier:
>>>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>>>>>>>>
>>>>>>>> How else can we tackle this?
>>>>>>>>
>>>>>>>> Denis
>>>>>>>>
>>>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>>>>>> dsetrakyan@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I think it is safe to assume at this point that everyone is in
>>>>> general
>>>>>>>>> agreement, since there are no active objections.
>>>>>>>>>
>>>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>>>>> happen:
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>>>>>>>> dsetrakyan@apache.org
>>>>>>>>> <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>>>>> raul.asf@evosent.com
>>>>>>>>> <javascript:;>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Yeah, I guess that's doable as well and requires less
>>> management
>>>>>>>> effort
>>>>>>>>>>> than my suggestion. We could use events [1] to store payload
>>> data
>>>>>>>> (e.g.
>>>>>>>>>>> IP,
>>>>>>>>>>> version, etc.)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Yes, we could use events or some other similar API provided by
>>> GA.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> What the download page CGI developed in? PHP?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> To be honest, no clue. I guess someone in the community can
>>> figure
>>>>> it
>>>>>>>>> out:
>>>>>>>>>>
>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>>>>>>>> (Google)
>>>>>>>>> is
>>>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>>>>> there's
>>>>>>>>>>> doubt
>>>>>>>>>>> in the PMC, it's better to ask legal@.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>>>>> website
>>>>>>>> is
>>>>>>>>>> GA enabled, and all we are doing is accessing a standard web
>>> page
>>>>> from
>>>>>>>>> the
>>>>>>>>>> Ignite web site. The information gathered from GA is available
>>> only
>>>>> to
>>>>>>>>> the
>>>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
>>> this
>>>>>>>>>> question.
>>>>>>>>>>
>>>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>>>>>>>> foundation/policies/privacy.html
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Sure, would be nice to hear from Roman as well.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Cheers.
>>>>>>>>>>>
>>>>>>>>>>> [1]
>>>>>>>>>>> https://developers.google.com/analytics/devguides/collection
>>>>>>>>>>> /analyticsjs/events
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Raul,
>>>>>>>>>>>>
>>>>>>>>>>>> Could point about Javascript, it will not work because it
>>> executes
>>>>>>>> in
>>>>>>>>>>> the
>>>>>>>>>>>> browser. This means we need a server-side script, like CGI we
>>> are
>>>>>>>>> using
>>>>>>>>>>> on
>>>>>>>>>>>> our download page.
>>>>>>>>>>>>
>>>>>>>>>>>> How about this approach. We create something like
>>>>> ignite-version.cgi
>>>>>>>>>>> script
>>>>>>>>>>>> which will invoke a call to GA and then return the latest
>>> version.
>>>>>>>>>>>>
>>>>>>>>>>>> This should work, right?
>>>>>>>>>>>>
>>>>>>>>>>>> D.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>>>>>>>> raul.asf@evosent.com
>>>>>>>>> <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Dmitriy and all
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
>>> how
>>>>>>>>> many
>>>>>>>>>>>> times
>>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>>> of
>>>>>>>>> starts.
>>>>>>>>>>>> This
>>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>>> the
>>>>>>>>> Ignite
>>>>>>>>>>>> PMC.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
>>> Javascript
>>>>>>>> on
>>>>>>>>>>> the
>>>>>>>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>>>>>>>> endpoint
>>>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>>>>>>>> request.
>>>>>>>>>>> The
>>>>>>>>>>>>> simplest thing is to write each request in a log file, along
>>> with
>>>>>>>>> the
>>>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
>>> (not
>>>>>>>>> sure
>>>>>>>>>>>> about
>>>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
>>> if
>>>>>>>> you
>>>>>>>>>>> make
>>>>>>>>>>>> it
>>>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
>>> node
>>>>>>>>> creates
>>>>>>>>>>> on
>>>>>>>>>>>>> first startup or something.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This endpoint would need some basic DDoS protection and
>>>>>>>> blacklisting
>>>>>>>>>>> to
>>>>>>>>>>>>> prevent data spoofing.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
>>> no
>>>>>>>> point
>>>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
>>> latest
>>>>>>>>>>>> versions:
>>>>>>>>>>>>> the endpoint itself can return them.
>>>>>>>>>>>>>
>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cheers.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cos, Raul,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
>>> Central
>>>>>>>>>>> being a
>>>>>>>>>>>>> 3rd
>>>>>>>>>>>>>> party repo (did not think about that initially). All your
>>>>>>>>>>> suggestions
>>>>>>>>>>>>> make
>>>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
>>> and so
>>>>>>>>> far
>>>>>>>>>>>>>> everything suggested required GIT dependencies and extra
>>> work.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
>>> Ignite
>>>>>>>>>>>> website
>>>>>>>>>>>>>> which will have only the latest version. Every time a node
>>>>>>>> starts,
>>>>>>>>>>> it
>>>>>>>>>>>>>> receives the latest version from the page and suggests that
>>>>>>>> users
>>>>>>>>>>>> upgrade
>>>>>>>>>>>>>> if needed.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
>>> track how
>>>>>>>>>>> many
>>>>>>>>>>>>> times
>>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>>> of
>>>>>>>>> starts.
>>>>>>>>>>>> This
>>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>>> the
>>>>>>>>> Ignite
>>>>>>>>>>>> PMC.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
>>> kept
>>>>>>>>> and
>>>>>>>>>>>>>> managed within Apache.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>>>>>>>> cos@apache.org <javascript:;>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I agree with Raul.
>>>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
>>> frown
>>>>>>>>>>> upon by
>>>>>>>>>>>>>> some.
>>>>>>>>>>>>>>> And might have a consequences like stats collection about
>>>>>>>>> users'
>>>>>>>>>>>>>>> infrastructure.
>>>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>>>>>>>> binary
>>>>>>>>>>>> data:
>>>>>>>>>>>>>>> everything is clear text and could be easily monitored by
>>>>>>>> any
>>>>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> network
>>>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>>>>>>>> bit
>>>>>>>>> of
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>> discipline.
>>>>>>>>>>>>>>> - the binary data download in the runtime is my main
>>> concern.
>>>>>>>>>>> This is
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>>>>>>>> repository
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
>>> a
>>>>>>>> call
>>>>>>>>>>> to
>>>>>>>>>>>> an
>>>>>>>>>>>>>>> external script to check upon the atest software version
>>>>>>>>> available
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
>>> party
>>>>>>>>>>>> server,
>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> believe the best way would be to put this script on ASF
>>> infra
>>>>>>>>> and
>>>>>>>>>>>> have
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> "update checker" running in a completely controlled
>>>>>>>> environment.
>>>>>>>>>>>>>> Actually,
>>>>>>>>>>>>>>> I
>>>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
>>> I
>>>>>>>> can
>>>>>>>>>>>>> probably
>>>>>>>>>>>>>>> dig
>>>>>>>>>>>>>>> out the corresponding thread.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>>> Cok
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>>>>>>>> Hey guys
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>>>>>>>> (Sonatype).
>>>>>>>>>>>>>> We
>>>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>>>>>>>>> extraction
>>>>>>>>>>>> they
>>>>>>>>>>>>>>> run.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
>>> maven.org
>>>>>>>>>>>>>> periodically
>>>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>>>>>>>> clear
>>>>>>>>>>>>>> indication
>>>>>>>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
>>> what
>>>>>>>>>>>>> corporation
>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>> is.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
>>> new
>>>>>>>>>>>>> dependency,
>>>>>>>>>>>>>>>> which I advise against.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
>>> Git
>>>>>>>>>>> tags:
>>>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>>>>>>>>> master
>>>>>>>>>>>>> branch
>>>>>>>>>>>>>>> (e.g
>>>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>>>>>>>> each
>>>>>>>>>>> major
>>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I would advocate this last option, but it requires
>>> somebody
>>>>>>>>>>>>> remembering
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> update the file with every release, unless we automate it
>>>>>>>>> with a
>>>>>>>>>>>>> Maven
>>>>>>>>>>>>>>>> plugin.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>>>>>>>> <javascript:;>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
>>> Is
>>>>>>>> it
>>>>>>>>>>> good
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> go?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
>>> ticket
>>>>>>>> on
>>>>>>>>>>> it?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>>>>>>>>> usage
>>>>>>>>>>>>> metrics
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
>>> resurrect
>>>>>>>>> the
>>>>>>>>>>>> topic
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
>>> collect
>>>>>>>>>>>> metrics,
>>>>>>>>>>>>>> but
>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>>>>>>>> maven.org
>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>>>>>>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
>>> at
>>>>>>>>> the
>>>>>>>>>>>> same
>>>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>>> urge our users to download and start using the latest
>>>>>>>>>>> version of
>>>>>>>>>>>>>>> Ignite.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
>>> in
>>>>>>>>> how
>>>>>>>>>>>> many
>>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
>>> cool
>>>>>>>>>>> metric
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
>>> think
>>>>>>>> of
>>>>>>>>>>> it
>>>>>>>>>>>> as
>>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>>> page
>>>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
>>> decide
>>>>>>>> to
>>>>>>>>>>>>> display
>>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>>>>>>>> release,
>>>>>>>>>>>>> e.g.
>>>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
>>> byte.
>>>>>>>>>>> Every
>>>>>>>>>>>>> time
>>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>>>>>>>> background.
>>>>>>>>>>>>>> Then
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>>>>>>>> for
>>>>>>>>>>> this
>>>>>>>>>>>>> JAR
>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
>>> starts
>>>>>>>> of
>>>>>>>>>>>> Ignite
>>>>>>>>>>>>>>> nodes.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>>>>>>>> Ignite
>>>>>>>>> to
>>>>>>>>>>>> send
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>>
>
Re: [DISCUSS] Ignite Update Checker
Posted by Denis Magda <dm...@apache.org>.
Hi Cos,
Sure, will see you around there.
Anyway, a short summary is the following.
Starting Ignite 2.3 the nodes will be connecting to a *static* file [1] deployed on Ignite site to obtain the most recent version. If the file has a later version than the nodes will print out a message encouraging a user to do an upgrade.
On top of that, some community members proposed to trigger GA once the file [1] is loaded. But to achieve this the file has to become dynamic being able to execute simple CGI scripts. The INFRA turned this request down.
[1] https://ignite.apache.org/latest
[2] https://issues.apache.org/jira/browse/INFRA-15182
—
Denis
> On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <co...@apache.org> wrote:
>
> Guys,
>
> I had a quick chat with Nikita earlier today and it seems that we have came across of a blocker of some kind. With my member's hat on I would love to help to find a resolution that let us (and potentially other communities) to move forward.
>
> Let's craft the message and circulate it with Infra and Greg. Denis, could you find me tomorrow at the IMCS and give me an insight into the technical side of it? I feel a bit at loss after going through this thread...
> --
> Regards,
> Cos
>
> On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org> wrote:
>> Dmitriy,
>>
>> That’s the rule. See replies in the ticket [1]
>>
>> *Background: the TLP server is already pretty darned busy just serving
>> *static* sites. Dynamic operation for near-200 PMCs would bury the
>> machine. Our policy of "static-only websites" has been in place since
>> the Foundation started*
>>
>> Download scripts seem to be the only exception and we as PMC don’t even
>> have access to them.
>>
>> If you want to keep pushing this direction let’s craft a message to
>> Greg and Daniel directly. I don’t know what else to ask for here rather
>> than a virtual machine that’s conceivably to much for a single script
>> like that.
>>
>> [1] https://issues.apache.org/jira/browse/INFRA-15182
>> <https://issues.apache.org/jira/browse/INFRA-15182>
>>
>> —
>> Denis
>>
>>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>>
>>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
>> <vo...@gridgain.com>
>>> wrote:
>>>
>>>> I am not sure it is good idea to send requests to 3rd-party
>> addresses from
>>>> Ignite node. Let's do not make the same mistakes again.
>>>>
>>>
>>> Agree with Vladimir.
>>>
>>> We obviously have CGI support on the website. Can someone explain why
>> CGI
>>> is not possible to use?
>>>
>>>
>>>>
>>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
>> <an...@gridgain.com>
>>>> wrote:
>>>>
>>>>> We may directly send request to GA from Ignite node:
>>>>> https://developers.google.com/analytics/devguides/
>>>> collection/protocol/v1/
>>>>> <https://developers.google.com/analytics/devguides/
>>>> collection/protocol/v1/
>>>>>>
>>>>> Latest version can be received from maven central:
>>>>> https://repo1.maven.org/maven2/org/apache/ignite/
>>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>>>>>
>>>>>
>>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>>>>> написал(а):
>>>>>>
>>>>>> Denis,
>>>>>>
>>>>>> I am not sure I understand. We already do have CGI enabled for
>>>>>> download.cgi. Is there something else we need?
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>>>> wrote:
>>>>>>
>>>>>>> There is an obstacle. There is no way to execute the script using
>> PHP
>>>> or
>>>>>>> similar sever side language and trigger GA as discussed earlier:
>>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>>>>>>>
>>>>>>> How else can we tackle this?
>>>>>>>
>>>>>>> Denis
>>>>>>>
>>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I think it is safe to assume at this point that everyone is in
>>>> general
>>>>>>>> agreement, since there are no active objections.
>>>>>>>>
>>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>>>> happen:
>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>>>>>>> dsetrakyan@apache.org
>>>>>>>> <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>>>> raul.asf@evosent.com
>>>>>>>> <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yeah, I guess that's doable as well and requires less
>> management
>>>>>>> effort
>>>>>>>>>> than my suggestion. We could use events [1] to store payload
>> data
>>>>>>> (e.g.
>>>>>>>>>> IP,
>>>>>>>>>> version, etc.)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Yes, we could use events or some other similar API provided by
>> GA.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> What the download page CGI developed in? PHP?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> To be honest, no clue. I guess someone in the community can
>> figure
>>>> it
>>>>>>>> out:
>>>>>>>>>
>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>>>>>>> (Google)
>>>>>>>> is
>>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>>>> there's
>>>>>>>>>> doubt
>>>>>>>>>> in the PMC, it's better to ask legal@.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>>>> website
>>>>>>> is
>>>>>>>>> GA enabled, and all we are doing is accessing a standard web
>> page
>>>> from
>>>>>>>> the
>>>>>>>>> Ignite web site. The information gathered from GA is available
>> only
>>>> to
>>>>>>>> the
>>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
>> this
>>>>>>>>> question.
>>>>>>>>>
>>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>>>>>>> foundation/policies/privacy.html
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Sure, would be nice to hear from Roman as well.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>> Cheers.
>>>>>>>>>>
>>>>>>>>>> [1]
>>>>>>>>>> https://developers.google.com/analytics/devguides/collection
>>>>>>>>>> /analyticsjs/events
>>>>>>>>>>
>>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Raul,
>>>>>>>>>>>
>>>>>>>>>>> Could point about Javascript, it will not work because it
>> executes
>>>>>>> in
>>>>>>>>>> the
>>>>>>>>>>> browser. This means we need a server-side script, like CGI we
>> are
>>>>>>>> using
>>>>>>>>>> on
>>>>>>>>>>> our download page.
>>>>>>>>>>>
>>>>>>>>>>> How about this approach. We create something like
>>>> ignite-version.cgi
>>>>>>>>>> script
>>>>>>>>>>> which will invoke a call to GA and then return the latest
>> version.
>>>>>>>>>>>
>>>>>>>>>>> This should work, right?
>>>>>>>>>>>
>>>>>>>>>>> D.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>>>>>>> raul.asf@evosent.com
>>>>>>>> <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hey Dmitriy and all
>>>>>>>>>>>>
>>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
>> how
>>>>>>>> many
>>>>>>>>>>> times
>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>> of
>>>>>>>> starts.
>>>>>>>>>>> This
>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>> the
>>>>>>>> Ignite
>>>>>>>>>>> PMC.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
>> Javascript
>>>>>>> on
>>>>>>>>>> the
>>>>>>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>>>>>>
>>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>>>>>>> endpoint
>>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>>>>>>> request.
>>>>>>>>>> The
>>>>>>>>>>>> simplest thing is to write each request in a log file, along
>> with
>>>>>>>> the
>>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
>> (not
>>>>>>>> sure
>>>>>>>>>>> about
>>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
>> if
>>>>>>> you
>>>>>>>>>> make
>>>>>>>>>>> it
>>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
>> node
>>>>>>>> creates
>>>>>>>>>> on
>>>>>>>>>>>> first startup or something.
>>>>>>>>>>>>
>>>>>>>>>>>> This endpoint would need some basic DDoS protection and
>>>>>>> blacklisting
>>>>>>>>>> to
>>>>>>>>>>>> prevent data spoofing.
>>>>>>>>>>>>
>>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
>> no
>>>>>>> point
>>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
>> latest
>>>>>>>>>>> versions:
>>>>>>>>>>>> the endpoint itself can return them.
>>>>>>>>>>>>
>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>
>>>>>>>>>>>> Cheers.
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Cos, Raul,
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
>> Central
>>>>>>>>>> being a
>>>>>>>>>>>> 3rd
>>>>>>>>>>>>> party repo (did not think about that initially). All your
>>>>>>>>>> suggestions
>>>>>>>>>>>> make
>>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
>> and so
>>>>>>>> far
>>>>>>>>>>>>> everything suggested required GIT dependencies and extra
>> work.
>>>>>>>>>>>>>
>>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
>> Ignite
>>>>>>>>>>> website
>>>>>>>>>>>>> which will have only the latest version. Every time a node
>>>>>>> starts,
>>>>>>>>>> it
>>>>>>>>>>>>> receives the latest version from the page and suggests that
>>>>>>> users
>>>>>>>>>>> upgrade
>>>>>>>>>>>>> if needed.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
>> track how
>>>>>>>>>> many
>>>>>>>>>>>> times
>>>>>>>>>>>>> this page was accessed, which will be equal to the number
>> of
>>>>>>>> starts.
>>>>>>>>>>> This
>>>>>>>>>>>>> way, the counter information is tracked and monitored by
>> the
>>>>>>>> Ignite
>>>>>>>>>>> PMC.
>>>>>>>>>>>>>
>>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
>> kept
>>>>>>>> and
>>>>>>>>>>>>> managed within Apache.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>
>>>>>>>>>>>>> D.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>>>>>>> cos@apache.org <javascript:;>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I agree with Raul.
>>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
>> frown
>>>>>>>>>> upon by
>>>>>>>>>>>>> some.
>>>>>>>>>>>>>> And might have a consequences like stats collection about
>>>>>>>> users'
>>>>>>>>>>>>>> infrastructure.
>>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>>>>>>> binary
>>>>>>>>>>> data:
>>>>>>>>>>>>>> everything is clear text and could be easily monitored by
>>>>>>> any
>>>>>>>> of
>>>>>>>>>>> the
>>>>>>>>>>>>>> network
>>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>>>>>>> bit
>>>>>>>> of
>>>>>>>>>>> the
>>>>>>>>>>>>>> release
>>>>>>>>>>>>>> discipline.
>>>>>>>>>>>>>> - the binary data download in the runtime is my main
>> concern.
>>>>>>>>>> This is
>>>>>>>>>>>> the
>>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>>>>>>> repository
>>>>>>>>>>>>> and
>>>>>>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
>> a
>>>>>>> call
>>>>>>>>>> to
>>>>>>>>>>> an
>>>>>>>>>>>>>> external script to check upon the atest software version
>>>>>>>> available
>>>>>>>>>>> for
>>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
>> party
>>>>>>>>>>> server,
>>>>>>>>>>>> I
>>>>>>>>>>>>>> believe the best way would be to put this script on ASF
>> infra
>>>>>>>> and
>>>>>>>>>>> have
>>>>>>>>>>>>> the
>>>>>>>>>>>>>> "update checker" running in a completely controlled
>>>>>>> environment.
>>>>>>>>>>>>> Actually,
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
>> I
>>>>>>> can
>>>>>>>>>>>> probably
>>>>>>>>>>>>>> dig
>>>>>>>>>>>>>> out the corresponding thread.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>>> Cok
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>>>>>>> Hey guys
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>>>>>>> (Sonatype).
>>>>>>>>>>>>> We
>>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>>>>>>>> extraction
>>>>>>>>>>> they
>>>>>>>>>>>>>> run.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
>> maven.org
>>>>>>>>>>>>> periodically
>>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>>>>>>> clear
>>>>>>>>>>>>> indication
>>>>>>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
>> what
>>>>>>>>>>>> corporation
>>>>>>>>>>>>> it
>>>>>>>>>>>>>>> is.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
>> new
>>>>>>>>>>>> dependency,
>>>>>>>>>>>>>>> which I advise against.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
>> Git
>>>>>>>>>> tags:
>>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>>>>>>>> master
>>>>>>>>>>>> branch
>>>>>>>>>>>>>> (e.g
>>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>>>>>>> each
>>>>>>>>>> major
>>>>>>>>>>>>>> version
>>>>>>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I would advocate this last option, but it requires
>> somebody
>>>>>>>>>>>> remembering
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>> update the file with every release, unless we automate it
>>>>>>>> with a
>>>>>>>>>>>> Maven
>>>>>>>>>>>>>>> plugin.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>>>>>>> <javascript:;>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
>> Is
>>>>>>> it
>>>>>>>>>> good
>>>>>>>>>>>> to
>>>>>>>>>>>>>> go?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> —
>>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
>> ticket
>>>>>>> on
>>>>>>>>>> it?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>>>>>>>> usage
>>>>>>>>>>>> metrics
>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
>> resurrect
>>>>>>>> the
>>>>>>>>>>> topic
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
>> collect
>>>>>>>>>>> metrics,
>>>>>>>>>>>>> but
>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>>>>>>> maven.org
>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>>>>>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
>> at
>>>>>>>> the
>>>>>>>>>>> same
>>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>>> urge our users to download and start using the latest
>>>>>>>>>> version of
>>>>>>>>>>>>>> Ignite.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
>> in
>>>>>>>> how
>>>>>>>>>>> many
>>>>>>>>>>>>>> times
>>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
>> cool
>>>>>>>>>> metric
>>>>>>>>>>>> for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
>> think
>>>>>>> of
>>>>>>>>>> it
>>>>>>>>>>> as
>>>>>>>>>>>>> of a
>>>>>>>>>>>>>>> page
>>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
>> decide
>>>>>>> to
>>>>>>>>>>>> display
>>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>>>>>>> release,
>>>>>>>>>>>> e.g.
>>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
>> byte.
>>>>>>>>>> Every
>>>>>>>>>>>> time
>>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>>>>>>> background.
>>>>>>>>>>>>> Then
>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>>>>>>> for
>>>>>>>>>> this
>>>>>>>>>>>> JAR
>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
>> starts
>>>>>>> of
>>>>>>>>>>> Ignite
>>>>>>>>>>>>>> nodes.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>>>>>>> Ignite
>>>>>>>> to
>>>>>>>>>>> send
>>>>>>>>>>>>> or
>>>>>>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>>>>>>> approach.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
Re: [DISCUSS] Ignite Update Checker
Posted by Konstantin Boudnik <co...@apache.org>.
Guys,
I had a quick chat with Nikita earlier today and it seems that we have came across of a blocker of some kind. With my member's hat on I would love to help to find a resolution that let us (and potentially other communities) to move forward.
Let's craft the message and circulate it with Infra and Greg. Denis, could you find me tomorrow at the IMCS and give me an insight into the technical side of it? I feel a bit at loss after going through this thread...
--
Regards,
Cos
On October 2, 2017 7:31:48 AM PDT, Denis Magda <dm...@apache.org> wrote:
>Dmitriy,
>
>That’s the rule. See replies in the ticket [1]
>
>*Background: the TLP server is already pretty darned busy just serving
>*static* sites. Dynamic operation for near-200 PMCs would bury the
>machine. Our policy of "static-only websites" has been in place since
>the Foundation started*
>
>Download scripts seem to be the only exception and we as PMC don’t even
>have access to them.
>
>If you want to keep pushing this direction let’s craft a message to
>Greg and Daniel directly. I don’t know what else to ask for here rather
>than a virtual machine that’s conceivably to much for a single script
>like that.
>
>[1] https://issues.apache.org/jira/browse/INFRA-15182
><https://issues.apache.org/jira/browse/INFRA-15182>
>
>—
>Denis
>
>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
>wrote:
>>
>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov
><vo...@gridgain.com>
>> wrote:
>>
>>> I am not sure it is good idea to send requests to 3rd-party
>addresses from
>>> Ignite node. Let's do not make the same mistakes again.
>>>
>>
>> Agree with Vladimir.
>>
>> We obviously have CGI support on the website. Can someone explain why
>CGI
>> is not possible to use?
>>
>>
>>>
>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov
><an...@gridgain.com>
>>> wrote:
>>>
>>>> We may directly send request to GA from Ignite node:
>>>> https://developers.google.com/analytics/devguides/
>>> collection/protocol/v1/
>>>> <https://developers.google.com/analytics/devguides/
>>> collection/protocol/v1/
>>>>>
>>>> Latest version can be received from maven central:
>>>> https://repo1.maven.org/maven2/org/apache/ignite/
>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>>>>
>>>>
>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>>>> написал(а):
>>>>>
>>>>> Denis,
>>>>>
>>>>> I am not sure I understand. We already do have CGI enabled for
>>>>> download.cgi. Is there something else we need?
>>>>>
>>>>> D.
>>>>>
>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>>> wrote:
>>>>>
>>>>>> There is an obstacle. There is no way to execute the script using
>PHP
>>> or
>>>>>> similar sever side language and trigger GA as discussed earlier:
>>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>>>>>>
>>>>>> How else can we tackle this?
>>>>>>
>>>>>> Denis
>>>>>>
>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>>>> dsetrakyan@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> I think it is safe to assume at this point that everyone is in
>>> general
>>>>>>> agreement, since there are no active objections.
>>>>>>>
>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>>> happen:
>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>>>>>
>>>>>>> D.
>>>>>>>
>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>>>>>> dsetrakyan@apache.org
>>>>>>> <javascript:;>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>>> raul.asf@evosent.com
>>>>>>> <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Yeah, I guess that's doable as well and requires less
>management
>>>>>> effort
>>>>>>>>> than my suggestion. We could use events [1] to store payload
>data
>>>>>> (e.g.
>>>>>>>>> IP,
>>>>>>>>> version, etc.)
>>>>>>>>
>>>>>>>>
>>>>>>>> Yes, we could use events or some other similar API provided by
>GA.
>>>>>>>>
>>>>>>>>
>>>>>>>>> What the download page CGI developed in? PHP?
>>>>>>>>>
>>>>>>>>
>>>>>>>> To be honest, no clue. I guess someone in the community can
>figure
>>> it
>>>>>>> out:
>>>>>>>>
>https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>>>>>
>>>>>>>>
>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>>>>>> (Google)
>>>>>>> is
>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>>> there's
>>>>>>>>> doubt
>>>>>>>>> in the PMC, it's better to ask legal@.
>>>>>>>>
>>>>>>>>
>>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>>> website
>>>>>> is
>>>>>>>> GA enabled, and all we are doing is accessing a standard web
>page
>>> from
>>>>>>> the
>>>>>>>> Ignite web site. The information gathered from GA is available
>only
>>> to
>>>>>>> the
>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
>this
>>>>>>>> question.
>>>>>>>>
>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>>>>>> foundation/policies/privacy.html
>>>>>>>>
>>>>>>>>
>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Sure, would be nice to hear from Roman as well.
>>>>>>>>
>>>>>>>>
>>>>>>>>> Cheers.
>>>>>>>>>
>>>>>>>>> [1]
>>>>>>>>> https://developers.google.com/analytics/devguides/collection
>>>>>>>>> /analyticsjs/events
>>>>>>>>>
>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Raul,
>>>>>>>>>>
>>>>>>>>>> Could point about Javascript, it will not work because it
>executes
>>>>>> in
>>>>>>>>> the
>>>>>>>>>> browser. This means we need a server-side script, like CGI we
>are
>>>>>>> using
>>>>>>>>> on
>>>>>>>>>> our download page.
>>>>>>>>>>
>>>>>>>>>> How about this approach. We create something like
>>> ignite-version.cgi
>>>>>>>>> script
>>>>>>>>>> which will invoke a call to GA and then return the latest
>version.
>>>>>>>>>>
>>>>>>>>>> This should work, right?
>>>>>>>>>>
>>>>>>>>>> D.
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>>>>>> raul.asf@evosent.com
>>>>>>> <javascript:;>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Dmitriy and all
>>>>>>>>>>>
>>>>>>>>>>> Also, since we have GA enabled for the website, we can track
>how
>>>>>>> many
>>>>>>>>>> times
>>>>>>>>>>>> this page was accessed, which will be equal to the number
>of
>>>>>>> starts.
>>>>>>>>>> This
>>>>>>>>>>>> way, the counter information is tracked and monitored by
>the
>>>>>>> Ignite
>>>>>>>>>> PMC.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Unfortunately this won't work because GA is loaded via
>Javascript
>>>>>> on
>>>>>>>>> the
>>>>>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>>>>>
>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>>>>>> endpoint
>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>>>>>> request.
>>>>>>>>> The
>>>>>>>>>>> simplest thing is to write each request in a log file, along
>with
>>>>>>> the
>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP
>(not
>>>>>>> sure
>>>>>>>>>> about
>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
>if
>>>>>> you
>>>>>>>>> make
>>>>>>>>>> it
>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the
>node
>>>>>>> creates
>>>>>>>>> on
>>>>>>>>>>> first startup or something.
>>>>>>>>>>>
>>>>>>>>>>> This endpoint would need some basic DDoS protection and
>>>>>> blacklisting
>>>>>>>>> to
>>>>>>>>>>> prevent data spoofing.
>>>>>>>>>>>
>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's
>no
>>>>>> point
>>>>>>>>>>> placing another file on Git or elsewhere for reporting the
>latest
>>>>>>>>>> versions:
>>>>>>>>>>> the endpoint itself can return them.
>>>>>>>>>>>
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Cheers.
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Cos, Raul,
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
>Central
>>>>>>>>> being a
>>>>>>>>>>> 3rd
>>>>>>>>>>>> party repo (did not think about that initially). All your
>>>>>>>>> suggestions
>>>>>>>>>>> make
>>>>>>>>>>>> sense, but I would like to keep it as simple as possible,
>and so
>>>>>>> far
>>>>>>>>>>>> everything suggested required GIT dependencies and extra
>work.
>>>>>>>>>>>>
>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
>Ignite
>>>>>>>>>> website
>>>>>>>>>>>> which will have only the latest version. Every time a node
>>>>>> starts,
>>>>>>>>> it
>>>>>>>>>>>> receives the latest version from the page and suggests that
>>>>>> users
>>>>>>>>>> upgrade
>>>>>>>>>>>> if needed.
>>>>>>>>>>>>
>>>>>>>>>>>> Also, since we have GA enabled for the website, we can
>track how
>>>>>>>>> many
>>>>>>>>>>> times
>>>>>>>>>>>> this page was accessed, which will be equal to the number
>of
>>>>>>> starts.
>>>>>>>>>> This
>>>>>>>>>>>> way, the counter information is tracked and monitored by
>the
>>>>>>> Ignite
>>>>>>>>>> PMC.
>>>>>>>>>>>>
>>>>>>>>>>>> This approach looks pretty innocent to me and everything is
>kept
>>>>>>> and
>>>>>>>>>>>> managed within Apache.
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>
>>>>>>>>>>>> D.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>>>>>> cos@apache.org <javascript:;>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> I agree with Raul.
>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
>frown
>>>>>>>>> upon by
>>>>>>>>>>>> some.
>>>>>>>>>>>>> And might have a consequences like stats collection about
>>>>>>> users'
>>>>>>>>>>>>> infrastructure.
>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>>>>>> binary
>>>>>>>>>> data:
>>>>>>>>>>>>> everything is clear text and could be easily monitored by
>>>>>> any
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>> network
>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>>>>>> bit
>>>>>>> of
>>>>>>>>>> the
>>>>>>>>>>>>> release
>>>>>>>>>>>>> discipline.
>>>>>>>>>>>>> - the binary data download in the runtime is my main
>concern.
>>>>>>>>> This is
>>>>>>>>>>> the
>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>>>>>> repository
>>>>>>>>>>>> and
>>>>>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>>>>>
>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make
>a
>>>>>> call
>>>>>>>>> to
>>>>>>>>>> an
>>>>>>>>>>>>> external script to check upon the atest software version
>>>>>>> available
>>>>>>>>>> for
>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
>party
>>>>>>>>>> server,
>>>>>>>>>>> I
>>>>>>>>>>>>> believe the best way would be to put this script on ASF
>infra
>>>>>>> and
>>>>>>>>>> have
>>>>>>>>>>>> the
>>>>>>>>>>>>> "update checker" running in a completely controlled
>>>>>> environment.
>>>>>>>>>>>> Actually,
>>>>>>>>>>>>> I
>>>>>>>>>>>>> recall we had this very discussion during the Incubation;
>I
>>>>>> can
>>>>>>>>>>> probably
>>>>>>>>>>>>> dig
>>>>>>>>>>>>> out the corresponding thread.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>>> Cok
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>>>>>> Hey guys
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>>>>>> (Sonatype).
>>>>>>>>>>>> We
>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>>>>>>> extraction
>>>>>>>>>> they
>>>>>>>>>>>>> run.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If Ignite servers all over the world were hitting
>maven.org
>>>>>>>>>>>> periodically
>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>>>>>> clear
>>>>>>>>>>>> indication
>>>>>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> They could reverse lookup the IP address and find out
>what
>>>>>>>>>>> corporation
>>>>>>>>>>>> it
>>>>>>>>>>>>>> is.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a
>new
>>>>>>>>>>> dependency,
>>>>>>>>>>>>>> which I advise against.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
>Git
>>>>>>>>> tags:
>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>>>>>>> master
>>>>>>>>>>> branch
>>>>>>>>>>>>> (e.g
>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>>>>>> each
>>>>>>>>> major
>>>>>>>>>>>>> version
>>>>>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I would advocate this last option, but it requires
>somebody
>>>>>>>>>>> remembering
>>>>>>>>>>>>> to
>>>>>>>>>>>>>> update the file with every release, unless we automate it
>>>>>>> with a
>>>>>>>>>>> Maven
>>>>>>>>>>>>>> plugin.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>>>>>> <javascript:;>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think?
>Is
>>>>>> it
>>>>>>>>> good
>>>>>>>>>>> to
>>>>>>>>>>>>> go?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> —
>>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a
>ticket
>>>>>> on
>>>>>>>>> it?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>>>>>>> usage
>>>>>>>>>>> metrics
>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to
>resurrect
>>>>>>> the
>>>>>>>>>> topic
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>>>>>> The main purpose of the update checker is not to
>collect
>>>>>>>>>> metrics,
>>>>>>>>>>>> but
>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>>>>>> maven.org
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>>>>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This way we do not send any information anywhere and,
>at
>>>>>>> the
>>>>>>>>>> same
>>>>>>>>>>>>> time,
>>>>>>>>>>>>>>>> urge our users to download and start using the latest
>>>>>>>>> version of
>>>>>>>>>>>>> Ignite.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight
>in
>>>>>>> how
>>>>>>>>>> many
>>>>>>>>>>>>> times
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a
>cool
>>>>>>>>> metric
>>>>>>>>>>> for
>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>> community to gauge the project popularity. You can
>think
>>>>>> of
>>>>>>>>> it
>>>>>>>>>> as
>>>>>>>>>>>> of a
>>>>>>>>>>>>>> page
>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even
>decide
>>>>>> to
>>>>>>>>>>> display
>>>>>>>>>>>>> this
>>>>>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>>>>>> release,
>>>>>>>>>>> e.g.
>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1
>byte.
>>>>>>>>> Every
>>>>>>>>>>> time
>>>>>>>>>>>>> an
>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>>>>>> background.
>>>>>>>>>>>> Then
>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>>>>>> for
>>>>>>>>> this
>>>>>>>>>>> JAR
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of
>starts
>>>>>> of
>>>>>>>>>> Ignite
>>>>>>>>>>>>> nodes.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>>>>>> Ignite
>>>>>>> to
>>>>>>>>>> send
>>>>>>>>>>>> or
>>>>>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>>>>>> approach.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>
Re: [DISCUSS] Ignite Update Checker
Posted by Denis Magda <dm...@gridgain.com>.
I can’t see it. Could you share a direct link to it?
If we can add one more script like that and it will work then the question
is closed.
Denis
On Monday, October 2, 2017, Dmitriy Setrakyan <ds...@apache.org> wrote:
> Denis, I can see the download.cgi script in SVN. Am I missing something?
>
> On Mon, Oct 2, 2017 at 5:31 PM, Denis Magda <dmagda@apache.org
> <javascript:;>> wrote:
>
> > Dmitriy,
> >
> > That’s the rule. See replies in the ticket [1]
> >
> > *Background: the TLP server is already pretty darned busy just serving
> > *static* sites. Dynamic operation for near-200 PMCs would bury the
> machine.
> > Our policy of "static-only websites" has been in place since the
> Foundation
> > started*
> >
> > Download scripts seem to be the only exception and we as PMC don’t even
> > have access to them.
> >
> > If you want to keep pushing this direction let’s craft a message to Greg
> > and Daniel directly. I don’t know what else to ask for here rather than a
> > virtual machine that’s conceivably to much for a single script like that.
> >
> > [1] https://issues.apache.org/jira/browse/INFRA-15182 <
> > https://issues.apache.org/jira/browse/INFRA-15182>
> >
> > —
> > Denis
> >
> > > On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <dsetrakyan@apache.org
> <javascript:;>>
> > wrote:
> > >
> > > On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <vozerov@gridgain.com
> <javascript:;>>
> > > wrote:
> > >
> > >> I am not sure it is good idea to send requests to 3rd-party addresses
> > from
> > >> Ignite node. Let's do not make the same mistakes again.
> > >>
> > >
> > > Agree with Vladimir.
> > >
> > > We obviously have CGI support on the website. Can someone explain why
> CGI
> > > is not possible to use?
> > >
> > >
> > >>
> > >> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <
> anovikov@gridgain.com <javascript:;>>
> > >> wrote:
> > >>
> > >>> We may directly send request to GA from Ignite node:
> > >>> https://developers.google.com/analytics/devguides/
> > >> collection/protocol/v1/
> > >>> <https://developers.google.com/analytics/devguides/
> > >> collection/protocol/v1/
> > >>>>
> > >>> Latest version can be received from maven central:
> > >>> https://repo1.maven.org/maven2/org/apache/ignite/
> > >>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
> > >>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> > >>>
> > >>>
> > >>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <dsetrakyan@apache.org
> <javascript:;>>
> > >>> написал(а):
> > >>>>
> > >>>> Denis,
> > >>>>
> > >>>> I am not sure I understand. We already do have CGI enabled for
> > >>>> download.cgi. Is there something else we need?
> > >>>>
> > >>>> D.
> > >>>>
> > >>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dmagda@gridgain.com
> <javascript:;>>
> > >> wrote:
> > >>>>
> > >>>>> There is an obstacle. There is no way to execute the script using
> PHP
> > >> or
> > >>>>> similar sever side language and trigger GA as discussed earlier:
> > >>>>> https://issues.apache.org/jira/browse/INFRA-15182
> > >>>>>
> > >>>>> How else can we tackle this?
> > >>>>>
> > >>>>> Denis
> > >>>>>
> > >>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
> > >>> dsetrakyan@apache.org <javascript:;>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> I think it is safe to assume at this point that everyone is in
> > >> general
> > >>>>>> agreement, since there are no active objections.
> > >>>>>>
> > >>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
> > >> happen:
> > >>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
> > >>>>>>
> > >>>>>> D.
> > >>>>>>
> > >>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> > >>>>> dsetrakyan@apache.org <javascript:;>
> > >>>>>> <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> > >> raul.asf@evosent.com <javascript:;>
> > >>>>>> <javascript:;>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Yeah, I guess that's doable as well and requires less management
> > >>>>> effort
> > >>>>>>>> than my suggestion. We could use events [1] to store payload
> data
> > >>>>> (e.g.
> > >>>>>>>> IP,
> > >>>>>>>> version, etc.)
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Yes, we could use events or some other similar API provided by
> GA.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> What the download page CGI developed in? PHP?
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> To be honest, no clue. I guess someone in the community can
> figure
> > >> it
> > >>>>>> out:
> > >>>>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> However, I'm not sure whether storing this data in a 3rd party
> > >>>>> (Google)
> > >>>>>> is
> > >>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
> > >> there's
> > >>>>>>>> doubt
> > >>>>>>>> in the PMC, it's better to ask legal@.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> I am not sure there is anything to ask about. The whole Ignite
> > >> website
> > >>>>> is
> > >>>>>>> GA enabled, and all we are doing is accessing a standard web page
> > >> from
> > >>>>>> the
> > >>>>>>> Ignite web site. The information gathered from GA is available
> only
> > >> to
> > >>>>>> the
> > >>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by
> this
> > >>>>>>> question.
> > >>>>>>>
> > >>>>>>> Even ASF website itself uses GA: https://www.apache.org/
> > >>>>>>> foundation/policies/privacy.html
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>> Sure, would be nice to hear from Roman as well.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>>> Cheers.
> > >>>>>>>>
> > >>>>>>>> [1]
> > >>>>>>>> https://developers.google.com/analytics/devguides/collection
> > >>>>>>>> /analyticsjs/events
> > >>>>>>>>
> > >>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > >>>>>>>> dsetrakyan@apache.org <javascript:;> <javascript:;>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> Raul,
> > >>>>>>>>>
> > >>>>>>>>> Could point about Javascript, it will not work because it
> > executes
> > >>>>> in
> > >>>>>>>> the
> > >>>>>>>>> browser. This means we need a server-side script, like CGI we
> are
> > >>>>>> using
> > >>>>>>>> on
> > >>>>>>>>> our download page.
> > >>>>>>>>>
> > >>>>>>>>> How about this approach. We create something like
> > >> ignite-version.cgi
> > >>>>>>>> script
> > >>>>>>>>> which will invoke a call to GA and then return the latest
> > version.
> > >>>>>>>>>
> > >>>>>>>>> This should work, right?
> > >>>>>>>>>
> > >>>>>>>>> D.
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> > >>>>> raul.asf@evosent.com <javascript:;>
> > >>>>>> <javascript:;>>
> > >>>>>>>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> Hey Dmitriy and all
> > >>>>>>>>>>
> > >>>>>>>>>> Also, since we have GA enabled for the website, we can track
> how
> > >>>>>> many
> > >>>>>>>>> times
> > >>>>>>>>>>> this page was accessed, which will be equal to the number of
> > >>>>>> starts.
> > >>>>>>>>> This
> > >>>>>>>>>>> way, the counter information is tracked and monitored by the
> > >>>>>> Ignite
> > >>>>>>>>> PMC.
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> Unfortunately this won't work because GA is loaded via
> > Javascript
> > >>>>> on
> > >>>>>>>> the
> > >>>>>>>>>> browser, so Google will never receive the page hit.
> > >>>>>>>>>>
> > >>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
> > >>>>> endpoint
> > >>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
> > >>>>> request.
> > >>>>>>>> The
> > >>>>>>>>>> simplest thing is to write each request in a log file, along
> > with
> > >>>>>> the
> > >>>>>>>>>> timestamp, the version reported by the client, maybe the IP
> (not
> > >>>>>> sure
> > >>>>>>>>> about
> > >>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK
> if
> > >>>>> you
> > >>>>>>>> make
> > >>>>>>>>> it
> > >>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
> > >>>>>> creates
> > >>>>>>>> on
> > >>>>>>>>>> first startup or something.
> > >>>>>>>>>>
> > >>>>>>>>>> This endpoint would need some basic DDoS protection and
> > >>>>> blacklisting
> > >>>>>>>> to
> > >>>>>>>>>> prevent data spoofing.
> > >>>>>>>>>>
> > >>>>>>>>>> If we'll be implementing this endpoint anyway, then there's no
> > >>>>> point
> > >>>>>>>>>> placing another file on Git or elsewhere for reporting the
> > latest
> > >>>>>>>>> versions:
> > >>>>>>>>>> the endpoint itself can return them.
> > >>>>>>>>>>
> > >>>>>>>>>> WDYT?
> > >>>>>>>>>>
> > >>>>>>>>>> Cheers.
> > >>>>>>>>>>
> > >>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > >>>>>>>>> dsetrakyan@apache.org <javascript:;> <javascript:;>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> Cos, Raul,
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thanks for the feedback. I completely agree about Maven
> Central
> > >>>>>>>> being a
> > >>>>>>>>>> 3rd
> > >>>>>>>>>>> party repo (did not think about that initially). All your
> > >>>>>>>> suggestions
> > >>>>>>>>>> make
> > >>>>>>>>>>> sense, but I would like to keep it as simple as possible, and
> > so
> > >>>>>> far
> > >>>>>>>>>>> everything suggested required GIT dependencies and extra
> work.
> > >>>>>>>>>>>
> > >>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
> > Ignite
> > >>>>>>>>> website
> > >>>>>>>>>>> which will have only the latest version. Every time a node
> > >>>>> starts,
> > >>>>>>>> it
> > >>>>>>>>>>> receives the latest version from the page and suggests that
> > >>>>> users
> > >>>>>>>>> upgrade
> > >>>>>>>>>>> if needed.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Also, since we have GA enabled for the website, we can track
> > how
> > >>>>>>>> many
> > >>>>>>>>>> times
> > >>>>>>>>>>> this page was accessed, which will be equal to the number of
> > >>>>>> starts.
> > >>>>>>>>> This
> > >>>>>>>>>>> way, the counter information is tracked and monitored by the
> > >>>>>> Ignite
> > >>>>>>>>> PMC.
> > >>>>>>>>>>>
> > >>>>>>>>>>> This approach looks pretty innocent to me and everything is
> > kept
> > >>>>>> and
> > >>>>>>>>>>> managed within Apache.
> > >>>>>>>>>>>
> > >>>>>>>>>>> Thoughts?
> > >>>>>>>>>>>
> > >>>>>>>>>>> D.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > >>>>>>>> cos@apache.org <javascript:;> <javascript:;>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> I agree with Raul.
> > >>>>>>>>>>>> - providing a ping-back address to a 3rd party might be
> frown
> > >>>>>>>> upon by
> > >>>>>>>>>>> some.
> > >>>>>>>>>>>> And might have a consequences like stats collection about
> > >>>>>> users'
> > >>>>>>>>>>>> infrastructure.
> > >>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
> > >>>>> binary
> > >>>>>>>>> data:
> > >>>>>>>>>>>> everything is clear text and could be easily monitored by
> > >>>>> any
> > >>>>>> of
> > >>>>>>>>> the
> > >>>>>>>>>>>> network
> > >>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
> > >>>>> bit
> > >>>>>> of
> > >>>>>>>>> the
> > >>>>>>>>>>>> release
> > >>>>>>>>>>>> discipline.
> > >>>>>>>>>>>> - the binary data download in the runtime is my main
> concern.
> > >>>>>>>> This is
> > >>>>>>>>>> the
> > >>>>>>>>>>>> vector of MMA. What if someone gains the control over the
> > >>>>>>>>> repository
> > >>>>>>>>>>> and
> > >>>>>>>>>>>> replaces the file with some malicious content.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
> > >>>>> call
> > >>>>>>>> to
> > >>>>>>>>> an
> > >>>>>>>>>>>> external script to check upon the atest software version
> > >>>>>> available
> > >>>>>>>>> for
> > >>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd
> party
> > >>>>>>>>> server,
> > >>>>>>>>>> I
> > >>>>>>>>>>>> believe the best way would be to put this script on ASF
> infra
> > >>>>>> and
> > >>>>>>>>> have
> > >>>>>>>>>>> the
> > >>>>>>>>>>>> "update checker" running in a completely controlled
> > >>>>> environment.
> > >>>>>>>>>>> Actually,
> > >>>>>>>>>>>> I
> > >>>>>>>>>>>> recall we had this very discussion during the Incubation; I
> > >>>>> can
> > >>>>>>>>>> probably
> > >>>>>>>>>>>> dig
> > >>>>>>>>>>>> out the corresponding thread.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Thoughts?
> > >>>>>>>>>>>> Cok
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > >>>>>>>>>>>>> Hey guys
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
> > >>>>>>>>> (Sonatype).
> > >>>>>>>>>>> We
> > >>>>>>>>>>>>> don't know what kind of data analysis or intelligence
> > >>>>>> extraction
> > >>>>>>>>> they
> > >>>>>>>>>>>> run.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> If Ignite servers all over the world were hitting
> maven.org
> > >>>>>>>>>>> periodically
> > >>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
> > >>>>> clear
> > >>>>>>>>>>> indication
> > >>>>>>>>>>>>> of who is running an Ignite server.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> They could reverse lookup the IP address and find out what
> > >>>>>>>>>> corporation
> > >>>>>>>>>>> it
> > >>>>>>>>>>>>> is.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> We could use the Git ssh API, but that would require a new
> > >>>>>>>>>> dependency,
> > >>>>>>>>>>>>> which I advise against.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new
> Git
> > >>>>>>>> tags:
> > >>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Another option is to place a txt file in the root of the
> > >>>>>> master
> > >>>>>>>>>> branch
> > >>>>>>>>>>>> (e.g
> > >>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
> > >>>>> each
> > >>>>>>>> major
> > >>>>>>>>>>>> version
> > >>>>>>>>>>>>> line (1.x, 2.x).
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I would advocate this last option, but it requires somebody
> > >>>>>>>>>> remembering
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>> update the file with every release, unless we automate it
> > >>>>>> with a
> > >>>>>>>>>> Maven
> > >>>>>>>>>>>>> plugin.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Hope that helps!
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> <javascript:;>
> > >>>>>> <javascript:;>> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> I see nothing wrong with this approach.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
> > >>>>> it
> > >>>>>>>> good
> > >>>>>>>>>> to
> > >>>>>>>>>>>> go?
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> —
> > >>>>>>>>>>>>> Denis
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > >>>>>>>>>>> dsetrakyan@apache.org <javascript:;> <javascript:;>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
> > >>>>> on
> > >>>>>>>> it?
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > >>>>>>>>>>>> dsetrakyan@apache.org <javascript:;> <javascript:;>>
> > >>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Igniters,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> There has been lots of talk of proposals about various
> > >>>>>> usage
> > >>>>>>>>>> metrics
> > >>>>>>>>>>>> for
> > >>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
> > >>>>>> the
> > >>>>>>>>> topic
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>>> propose something very simple and non-intrusive.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 1. Update Checker
> > >>>>>>>>>>>>>>> The main purpose of the update checker is not to collect
> > >>>>>>>>> metrics,
> > >>>>>>>>>>> but
> > >>>>>>>>>>>> to
> > >>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
> > >>>>>>>>> maven.org
> > >>>>>>>>>>> and
> > >>>>>>>>>>>>>>> getting the version out of the metadata file:
> > >>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
> > >>>>>> org/apache/ignite/ignite-core/
> > >>>>>>>>>>>>>>> maven-metadata.xml
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> This way we do not send any information anywhere and, at
> > >>>>>> the
> > >>>>>>>>> same
> > >>>>>>>>>>>> time,
> > >>>>>>>>>>>>>>> urge our users to download and start using the latest
> > >>>>>>>> version of
> > >>>>>>>>>>>> Ignite.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> 2. Startup Counter
> > >>>>>>>>>>>>>>> This piece is optional, but we can also get an insight in
> > >>>>>> how
> > >>>>>>>>> many
> > >>>>>>>>>>>> times
> > >>>>>>>>>>>>> a
> > >>>>>>>>>>>>>>> certain Ignite release gets started. This is just a cool
> > >>>>>>>> metric
> > >>>>>>>>>> for
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>> community to gauge the project popularity. You can think
> > >>>>> of
> > >>>>>>>> it
> > >>>>>>>>> as
> > >>>>>>>>>>> of a
> > >>>>>>>>>>>>> page
> > >>>>>>>>>>>>>>> visit counter shown on many websites. We can even decide
> > >>>>> to
> > >>>>>>>>>> display
> > >>>>>>>>>>>> this
> > >>>>>>>>>>>>>>> counter on the Ignite website as well.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
> > >>>>>>>> release,
> > >>>>>>>>>> e.g.
> > >>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
> > >>>>>>>> Every
> > >>>>>>>>>> time
> > >>>>>>>>>>>> an
> > >>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
> > >>>>>>>> background.
> > >>>>>>>>>>> Then
> > >>>>>>>>>>>> we
> > >>>>>>>>>>>>>>> will be able to view the number of the total downloads
> > >>>>> for
> > >>>>>>>> this
> > >>>>>>>>>> JAR
> > >>>>>>>>>>> in
> > >>>>>>>>>>>>>>> Maven Central, which is essentially the number of starts
> > >>>>> of
> > >>>>>>>>> Ignite
> > >>>>>>>>>>>> nodes.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> *Note that neither of the above suggestions require
> > >>>>> Ignite
> > >>>>>> to
> > >>>>>>>>> send
> > >>>>>>>>>>> or
> > >>>>>>>>>>>>>>> track any user information whatsoever.*
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
> > >>>>>>>> approach.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> D.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>
> > >>>
> > >>
> >
> >
>
Re: [DISCUSS] Ignite Update Checker
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Denis, I can see the download.cgi script in SVN. Am I missing something?
On Mon, Oct 2, 2017 at 5:31 PM, Denis Magda <dm...@apache.org> wrote:
> Dmitriy,
>
> That’s the rule. See replies in the ticket [1]
>
> *Background: the TLP server is already pretty darned busy just serving
> *static* sites. Dynamic operation for near-200 PMCs would bury the machine.
> Our policy of "static-only websites" has been in place since the Foundation
> started*
>
> Download scripts seem to be the only exception and we as PMC don’t even
> have access to them.
>
> If you want to keep pushing this direction let’s craft a message to Greg
> and Daniel directly. I don’t know what else to ask for here rather than a
> virtual machine that’s conceivably to much for a single script like that.
>
> [1] https://issues.apache.org/jira/browse/INFRA-15182 <
> https://issues.apache.org/jira/browse/INFRA-15182>
>
> —
> Denis
>
> > On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
> >
> > On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <vo...@gridgain.com>
> > wrote:
> >
> >> I am not sure it is good idea to send requests to 3rd-party addresses
> from
> >> Ignite node. Let's do not make the same mistakes again.
> >>
> >
> > Agree with Vladimir.
> >
> > We obviously have CGI support on the website. Can someone explain why CGI
> > is not possible to use?
> >
> >
> >>
> >> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <an...@gridgain.com>
> >> wrote:
> >>
> >>> We may directly send request to GA from Ignite node:
> >>> https://developers.google.com/analytics/devguides/
> >> collection/protocol/v1/
> >>> <https://developers.google.com/analytics/devguides/
> >> collection/protocol/v1/
> >>>>
> >>> Latest version can be received from maven central:
> >>> https://repo1.maven.org/maven2/org/apache/ignite/
> >>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
> >>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> >>>
> >>>
> >>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
> >>> написал(а):
> >>>>
> >>>> Denis,
> >>>>
> >>>> I am not sure I understand. We already do have CGI enabled for
> >>>> download.cgi. Is there something else we need?
> >>>>
> >>>> D.
> >>>>
> >>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
> >> wrote:
> >>>>
> >>>>> There is an obstacle. There is no way to execute the script using PHP
> >> or
> >>>>> similar sever side language and trigger GA as discussed earlier:
> >>>>> https://issues.apache.org/jira/browse/INFRA-15182
> >>>>>
> >>>>> How else can we tackle this?
> >>>>>
> >>>>> Denis
> >>>>>
> >>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
> >>> dsetrakyan@apache.org>
> >>>>> wrote:
> >>>>>
> >>>>>> I think it is safe to assume at this point that everyone is in
> >> general
> >>>>>> agreement, since there are no active objections.
> >>>>>>
> >>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
> >> happen:
> >>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org
> >>>>>> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> >> raul.asf@evosent.com
> >>>>>> <javascript:;>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Yeah, I guess that's doable as well and requires less management
> >>>>> effort
> >>>>>>>> than my suggestion. We could use events [1] to store payload data
> >>>>> (e.g.
> >>>>>>>> IP,
> >>>>>>>> version, etc.)
> >>>>>>>
> >>>>>>>
> >>>>>>> Yes, we could use events or some other similar API provided by GA.
> >>>>>>>
> >>>>>>>
> >>>>>>>> What the download page CGI developed in? PHP?
> >>>>>>>>
> >>>>>>>
> >>>>>>> To be honest, no clue. I guess someone in the community can figure
> >> it
> >>>>>> out:
> >>>>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >>>>>>>
> >>>>>>>
> >>>>>>>> However, I'm not sure whether storing this data in a 3rd party
> >>>>> (Google)
> >>>>>> is
> >>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
> >> there's
> >>>>>>>> doubt
> >>>>>>>> in the PMC, it's better to ask legal@.
> >>>>>>>
> >>>>>>>
> >>>>>>> I am not sure there is anything to ask about. The whole Ignite
> >> website
> >>>>> is
> >>>>>>> GA enabled, and all we are doing is accessing a standard web page
> >> from
> >>>>>> the
> >>>>>>> Ignite web site. The information gathered from GA is available only
> >> to
> >>>>>> the
> >>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by this
> >>>>>>> question.
> >>>>>>>
> >>>>>>> Even ASF website itself uses GA: https://www.apache.org/
> >>>>>>> foundation/policies/privacy.html
> >>>>>>>
> >>>>>>>
> >>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
> >>>>>>>>
> >>>>>>>
> >>>>>>> Sure, would be nice to hear from Roman as well.
> >>>>>>>
> >>>>>>>
> >>>>>>>> Cheers.
> >>>>>>>>
> >>>>>>>> [1]
> >>>>>>>> https://developers.google.com/analytics/devguides/collection
> >>>>>>>> /analyticsjs/events
> >>>>>>>>
> >>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> Raul,
> >>>>>>>>>
> >>>>>>>>> Could point about Javascript, it will not work because it
> executes
> >>>>> in
> >>>>>>>> the
> >>>>>>>>> browser. This means we need a server-side script, like CGI we are
> >>>>>> using
> >>>>>>>> on
> >>>>>>>>> our download page.
> >>>>>>>>>
> >>>>>>>>> How about this approach. We create something like
> >> ignite-version.cgi
> >>>>>>>> script
> >>>>>>>>> which will invoke a call to GA and then return the latest
> version.
> >>>>>>>>>
> >>>>>>>>> This should work, right?
> >>>>>>>>>
> >>>>>>>>> D.
> >>>>>>>>>
> >>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> >>>>> raul.asf@evosent.com
> >>>>>> <javascript:;>>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Hey Dmitriy and all
> >>>>>>>>>>
> >>>>>>>>>> Also, since we have GA enabled for the website, we can track how
> >>>>>> many
> >>>>>>>>> times
> >>>>>>>>>>> this page was accessed, which will be equal to the number of
> >>>>>> starts.
> >>>>>>>>> This
> >>>>>>>>>>> way, the counter information is tracked and monitored by the
> >>>>>> Ignite
> >>>>>>>>> PMC.
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> Unfortunately this won't work because GA is loaded via
> Javascript
> >>>>> on
> >>>>>>>> the
> >>>>>>>>>> browser, so Google will never receive the page hit.
> >>>>>>>>>>
> >>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
> >>>>> endpoint
> >>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
> >>>>> request.
> >>>>>>>> The
> >>>>>>>>>> simplest thing is to write each request in a log file, along
> with
> >>>>>> the
> >>>>>>>>>> timestamp, the version reported by the client, maybe the IP (not
> >>>>>> sure
> >>>>>>>>> about
> >>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK if
> >>>>> you
> >>>>>>>> make
> >>>>>>>>> it
> >>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
> >>>>>> creates
> >>>>>>>> on
> >>>>>>>>>> first startup or something.
> >>>>>>>>>>
> >>>>>>>>>> This endpoint would need some basic DDoS protection and
> >>>>> blacklisting
> >>>>>>>> to
> >>>>>>>>>> prevent data spoofing.
> >>>>>>>>>>
> >>>>>>>>>> If we'll be implementing this endpoint anyway, then there's no
> >>>>> point
> >>>>>>>>>> placing another file on Git or elsewhere for reporting the
> latest
> >>>>>>>>> versions:
> >>>>>>>>>> the endpoint itself can return them.
> >>>>>>>>>>
> >>>>>>>>>> WDYT?
> >>>>>>>>>>
> >>>>>>>>>> Cheers.
> >>>>>>>>>>
> >>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Cos, Raul,
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks for the feedback. I completely agree about Maven Central
> >>>>>>>> being a
> >>>>>>>>>> 3rd
> >>>>>>>>>>> party repo (did not think about that initially). All your
> >>>>>>>> suggestions
> >>>>>>>>>> make
> >>>>>>>>>>> sense, but I would like to keep it as simple as possible, and
> so
> >>>>>> far
> >>>>>>>>>>> everything suggested required GIT dependencies and extra work.
> >>>>>>>>>>>
> >>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the
> Ignite
> >>>>>>>>> website
> >>>>>>>>>>> which will have only the latest version. Every time a node
> >>>>> starts,
> >>>>>>>> it
> >>>>>>>>>>> receives the latest version from the page and suggests that
> >>>>> users
> >>>>>>>>> upgrade
> >>>>>>>>>>> if needed.
> >>>>>>>>>>>
> >>>>>>>>>>> Also, since we have GA enabled for the website, we can track
> how
> >>>>>>>> many
> >>>>>>>>>> times
> >>>>>>>>>>> this page was accessed, which will be equal to the number of
> >>>>>> starts.
> >>>>>>>>> This
> >>>>>>>>>>> way, the counter information is tracked and monitored by the
> >>>>>> Ignite
> >>>>>>>>> PMC.
> >>>>>>>>>>>
> >>>>>>>>>>> This approach looks pretty innocent to me and everything is
> kept
> >>>>>> and
> >>>>>>>>>>> managed within Apache.
> >>>>>>>>>>>
> >>>>>>>>>>> Thoughts?
> >>>>>>>>>>>
> >>>>>>>>>>> D.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >>>>>>>> cos@apache.org <javascript:;>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> I agree with Raul.
> >>>>>>>>>>>> - providing a ping-back address to a 3rd party might be frown
> >>>>>>>> upon by
> >>>>>>>>>>> some.
> >>>>>>>>>>>> And might have a consequences like stats collection about
> >>>>>> users'
> >>>>>>>>>>>> infrastructure.
> >>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
> >>>>> binary
> >>>>>>>>> data:
> >>>>>>>>>>>> everything is clear text and could be easily monitored by
> >>>>> any
> >>>>>> of
> >>>>>>>>> the
> >>>>>>>>>>>> network
> >>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
> >>>>> bit
> >>>>>> of
> >>>>>>>>> the
> >>>>>>>>>>>> release
> >>>>>>>>>>>> discipline.
> >>>>>>>>>>>> - the binary data download in the runtime is my main concern.
> >>>>>>>> This is
> >>>>>>>>>> the
> >>>>>>>>>>>> vector of MMA. What if someone gains the control over the
> >>>>>>>>> repository
> >>>>>>>>>>> and
> >>>>>>>>>>>> replaces the file with some malicious content.
> >>>>>>>>>>>>
> >>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
> >>>>> call
> >>>>>>>> to
> >>>>>>>>> an
> >>>>>>>>>>>> external script to check upon the atest software version
> >>>>>> available
> >>>>>>>>> for
> >>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd party
> >>>>>>>>> server,
> >>>>>>>>>> I
> >>>>>>>>>>>> believe the best way would be to put this script on ASF infra
> >>>>>> and
> >>>>>>>>> have
> >>>>>>>>>>> the
> >>>>>>>>>>>> "update checker" running in a completely controlled
> >>>>> environment.
> >>>>>>>>>>> Actually,
> >>>>>>>>>>>> I
> >>>>>>>>>>>> recall we had this very discussion during the Incubation; I
> >>>>> can
> >>>>>>>>>> probably
> >>>>>>>>>>>> dig
> >>>>>>>>>>>> out the corresponding thread.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thoughts?
> >>>>>>>>>>>> Cok
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >>>>>>>>>>>>> Hey guys
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
> >>>>>>>>> (Sonatype).
> >>>>>>>>>>> We
> >>>>>>>>>>>>> don't know what kind of data analysis or intelligence
> >>>>>> extraction
> >>>>>>>>> they
> >>>>>>>>>>>> run.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> If Ignite servers all over the world were hitting maven.org
> >>>>>>>>>>> periodically
> >>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
> >>>>> clear
> >>>>>>>>>>> indication
> >>>>>>>>>>>>> of who is running an Ignite server.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> They could reverse lookup the IP address and find out what
> >>>>>>>>>> corporation
> >>>>>>>>>>> it
> >>>>>>>>>>>>> is.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> We could use the Git ssh API, but that would require a new
> >>>>>>>>>> dependency,
> >>>>>>>>>>>>> which I advise against.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git
> >>>>>>>> tags:
> >>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Another option is to place a txt file in the root of the
> >>>>>> master
> >>>>>>>>>> branch
> >>>>>>>>>>>> (e.g
> >>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
> >>>>> each
> >>>>>>>> major
> >>>>>>>>>>>> version
> >>>>>>>>>>>>> line (1.x, 2.x).
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I would advocate this last option, but it requires somebody
> >>>>>>>>>> remembering
> >>>>>>>>>>>> to
> >>>>>>>>>>>>> update the file with every release, unless we automate it
> >>>>>> with a
> >>>>>>>>>> Maven
> >>>>>>>>>>>>> plugin.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hope that helps!
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> >>>>>> <javascript:;>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I see nothing wrong with this approach.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
> >>>>> it
> >>>>>>>> good
> >>>>>>>>>> to
> >>>>>>>>>>>> go?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> —
> >>>>>>>>>>>>> Denis
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
> >>>>> on
> >>>>>>>> it?
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Igniters,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> There has been lots of talk of proposals about various
> >>>>>> usage
> >>>>>>>>>> metrics
> >>>>>>>>>>>> for
> >>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
> >>>>>> the
> >>>>>>>>> topic
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> propose something very simple and non-intrusive.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 1. Update Checker
> >>>>>>>>>>>>>>> The main purpose of the update checker is not to collect
> >>>>>>>>> metrics,
> >>>>>>>>>>> but
> >>>>>>>>>>>> to
> >>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
> >>>>>>>>> maven.org
> >>>>>>>>>>> and
> >>>>>>>>>>>>>>> getting the version out of the metadata file:
> >>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
> >>>>>> org/apache/ignite/ignite-core/
> >>>>>>>>>>>>>>> maven-metadata.xml
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> This way we do not send any information anywhere and, at
> >>>>>> the
> >>>>>>>>> same
> >>>>>>>>>>>> time,
> >>>>>>>>>>>>>>> urge our users to download and start using the latest
> >>>>>>>> version of
> >>>>>>>>>>>> Ignite.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 2. Startup Counter
> >>>>>>>>>>>>>>> This piece is optional, but we can also get an insight in
> >>>>>> how
> >>>>>>>>> many
> >>>>>>>>>>>> times
> >>>>>>>>>>>>> a
> >>>>>>>>>>>>>>> certain Ignite release gets started. This is just a cool
> >>>>>>>> metric
> >>>>>>>>>> for
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>> community to gauge the project popularity. You can think
> >>>>> of
> >>>>>>>> it
> >>>>>>>>> as
> >>>>>>>>>>> of a
> >>>>>>>>>>>>> page
> >>>>>>>>>>>>>>> visit counter shown on many websites. We can even decide
> >>>>> to
> >>>>>>>>>> display
> >>>>>>>>>>>> this
> >>>>>>>>>>>>>>> counter on the Ignite website as well.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
> >>>>>>>> release,
> >>>>>>>>>> e.g.
> >>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
> >>>>>>>> Every
> >>>>>>>>>> time
> >>>>>>>>>>>> an
> >>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
> >>>>>>>> background.
> >>>>>>>>>>> Then
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>> will be able to view the number of the total downloads
> >>>>> for
> >>>>>>>> this
> >>>>>>>>>> JAR
> >>>>>>>>>>> in
> >>>>>>>>>>>>>>> Maven Central, which is essentially the number of starts
> >>>>> of
> >>>>>>>>> Ignite
> >>>>>>>>>>>> nodes.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> *Note that neither of the above suggestions require
> >>>>> Ignite
> >>>>>> to
> >>>>>>>>> send
> >>>>>>>>>>> or
> >>>>>>>>>>>>>>> track any user information whatsoever.*
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
> >>>>>>>> approach.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> D.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>>
> >>
>
>
Re: [DISCUSS] Ignite Update Checker
Posted by Denis Magda <dm...@apache.org>.
Dmitriy,
That’s the rule. See replies in the ticket [1]
*Background: the TLP server is already pretty darned busy just serving *static* sites. Dynamic operation for near-200 PMCs would bury the machine. Our policy of "static-only websites" has been in place since the Foundation started*
Download scripts seem to be the only exception and we as PMC don’t even have access to them.
If you want to keep pushing this direction let’s craft a message to Greg and Daniel directly. I don’t know what else to ask for here rather than a virtual machine that’s conceivably to much for a single script like that.
[1] https://issues.apache.org/jira/browse/INFRA-15182 <https://issues.apache.org/jira/browse/INFRA-15182>
—
Denis
> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <ds...@apache.org> wrote:
>
> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <vo...@gridgain.com>
> wrote:
>
>> I am not sure it is good idea to send requests to 3rd-party addresses from
>> Ignite node. Let's do not make the same mistakes again.
>>
>
> Agree with Vladimir.
>
> We obviously have CGI support on the website. Can someone explain why CGI
> is not possible to use?
>
>
>>
>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <an...@gridgain.com>
>> wrote:
>>
>>> We may directly send request to GA from Ignite node:
>>> https://developers.google.com/analytics/devguides/
>> collection/protocol/v1/
>>> <https://developers.google.com/analytics/devguides/
>> collection/protocol/v1/
>>>>
>>> Latest version can be received from maven central:
>>> https://repo1.maven.org/maven2/org/apache/ignite/
>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/
>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>>>
>>>
>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
>>> написал(а):
>>>>
>>>> Denis,
>>>>
>>>> I am not sure I understand. We already do have CGI enabled for
>>>> download.cgi. Is there something else we need?
>>>>
>>>> D.
>>>>
>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
>> wrote:
>>>>
>>>>> There is an obstacle. There is no way to execute the script using PHP
>> or
>>>>> similar sever side language and trigger GA as discussed earlier:
>>>>> https://issues.apache.org/jira/browse/INFRA-15182
>>>>>
>>>>> How else can we tackle this?
>>>>>
>>>>> Denis
>>>>>
>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan <
>>> dsetrakyan@apache.org>
>>>>> wrote:
>>>>>
>>>>>> I think it is safe to assume at this point that everyone is in
>> general
>>>>>> agreement, since there are no active objections.
>>>>>>
>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it
>> happen:
>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org
>>>>>> <javascript:;>>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
>> raul.asf@evosent.com
>>>>>> <javascript:;>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Yeah, I guess that's doable as well and requires less management
>>>>> effort
>>>>>>>> than my suggestion. We could use events [1] to store payload data
>>>>> (e.g.
>>>>>>>> IP,
>>>>>>>> version, etc.)
>>>>>>>
>>>>>>>
>>>>>>> Yes, we could use events or some other similar API provided by GA.
>>>>>>>
>>>>>>>
>>>>>>>> What the download page CGI developed in? PHP?
>>>>>>>>
>>>>>>>
>>>>>>> To be honest, no clue. I guess someone in the community can figure
>> it
>>>>>> out:
>>>>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>>>>
>>>>>>>
>>>>>>>> However, I'm not sure whether storing this data in a 3rd party
>>>>> (Google)
>>>>>> is
>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if
>> there's
>>>>>>>> doubt
>>>>>>>> in the PMC, it's better to ask legal@.
>>>>>>>
>>>>>>>
>>>>>>> I am not sure there is anything to ask about. The whole Ignite
>> website
>>>>> is
>>>>>>> GA enabled, and all we are doing is accessing a standard web page
>> from
>>>>>> the
>>>>>>> Ignite web site. The information gathered from GA is available only
>> to
>>>>>> the
>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by this
>>>>>>> question.
>>>>>>>
>>>>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>>>>> foundation/policies/privacy.html
>>>>>>>
>>>>>>>
>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>>>>
>>>>>>>
>>>>>>> Sure, would be nice to hear from Roman as well.
>>>>>>>
>>>>>>>
>>>>>>>> Cheers.
>>>>>>>>
>>>>>>>> [1]
>>>>>>>> https://developers.google.com/analytics/devguides/collection
>>>>>>>> /analyticsjs/events
>>>>>>>>
>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Raul,
>>>>>>>>>
>>>>>>>>> Could point about Javascript, it will not work because it executes
>>>>> in
>>>>>>>> the
>>>>>>>>> browser. This means we need a server-side script, like CGI we are
>>>>>> using
>>>>>>>> on
>>>>>>>>> our download page.
>>>>>>>>>
>>>>>>>>> How about this approach. We create something like
>> ignite-version.cgi
>>>>>>>> script
>>>>>>>>> which will invoke a call to GA and then return the latest version.
>>>>>>>>>
>>>>>>>>> This should work, right?
>>>>>>>>>
>>>>>>>>> D.
>>>>>>>>>
>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>>>>> raul.asf@evosent.com
>>>>>> <javascript:;>>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hey Dmitriy and all
>>>>>>>>>>
>>>>>>>>>> Also, since we have GA enabled for the website, we can track how
>>>>>> many
>>>>>>>>> times
>>>>>>>>>>> this page was accessed, which will be equal to the number of
>>>>>> starts.
>>>>>>>>> This
>>>>>>>>>>> way, the counter information is tracked and monitored by the
>>>>>> Ignite
>>>>>>>>> PMC.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Unfortunately this won't work because GA is loaded via Javascript
>>>>> on
>>>>>>>> the
>>>>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>>>>
>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS
>>>>> endpoint
>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>>>>> request.
>>>>>>>> The
>>>>>>>>>> simplest thing is to write each request in a log file, along with
>>>>>> the
>>>>>>>>>> timestamp, the version reported by the client, maybe the IP (not
>>>>>> sure
>>>>>>>>> about
>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK if
>>>>> you
>>>>>>>> make
>>>>>>>>> it
>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
>>>>>> creates
>>>>>>>> on
>>>>>>>>>> first startup or something.
>>>>>>>>>>
>>>>>>>>>> This endpoint would need some basic DDoS protection and
>>>>> blacklisting
>>>>>>>> to
>>>>>>>>>> prevent data spoofing.
>>>>>>>>>>
>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's no
>>>>> point
>>>>>>>>>> placing another file on Git or elsewhere for reporting the latest
>>>>>>>>> versions:
>>>>>>>>>> the endpoint itself can return them.
>>>>>>>>>>
>>>>>>>>>> WDYT?
>>>>>>>>>>
>>>>>>>>>> Cheers.
>>>>>>>>>>
>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Cos, Raul,
>>>>>>>>>>>
>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven Central
>>>>>>>> being a
>>>>>>>>>> 3rd
>>>>>>>>>>> party repo (did not think about that initially). All your
>>>>>>>> suggestions
>>>>>>>>>> make
>>>>>>>>>>> sense, but I would like to keep it as simple as possible, and so
>>>>>> far
>>>>>>>>>>> everything suggested required GIT dependencies and extra work.
>>>>>>>>>>>
>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite
>>>>>>>>> website
>>>>>>>>>>> which will have only the latest version. Every time a node
>>>>> starts,
>>>>>>>> it
>>>>>>>>>>> receives the latest version from the page and suggests that
>>>>> users
>>>>>>>>> upgrade
>>>>>>>>>>> if needed.
>>>>>>>>>>>
>>>>>>>>>>> Also, since we have GA enabled for the website, we can track how
>>>>>>>> many
>>>>>>>>>> times
>>>>>>>>>>> this page was accessed, which will be equal to the number of
>>>>>> starts.
>>>>>>>>> This
>>>>>>>>>>> way, the counter information is tracked and monitored by the
>>>>>> Ignite
>>>>>>>>> PMC.
>>>>>>>>>>>
>>>>>>>>>>> This approach looks pretty innocent to me and everything is kept
>>>>>> and
>>>>>>>>>>> managed within Apache.
>>>>>>>>>>>
>>>>>>>>>>> Thoughts?
>>>>>>>>>>>
>>>>>>>>>>> D.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>>>>> cos@apache.org <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> I agree with Raul.
>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be frown
>>>>>>>> upon by
>>>>>>>>>>> some.
>>>>>>>>>>>> And might have a consequences like stats collection about
>>>>>> users'
>>>>>>>>>>>> infrastructure.
>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>>>>> binary
>>>>>>>>> data:
>>>>>>>>>>>> everything is clear text and could be easily monitored by
>>>>> any
>>>>>> of
>>>>>>>>> the
>>>>>>>>>>>> network
>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>>>>> bit
>>>>>> of
>>>>>>>>> the
>>>>>>>>>>>> release
>>>>>>>>>>>> discipline.
>>>>>>>>>>>> - the binary data download in the runtime is my main concern.
>>>>>>>> This is
>>>>>>>>>> the
>>>>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>>>>> repository
>>>>>>>>>>> and
>>>>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>>>>
>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
>>>>> call
>>>>>>>> to
>>>>>>>>> an
>>>>>>>>>>>> external script to check upon the atest software version
>>>>>> available
>>>>>>>>> for
>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd party
>>>>>>>>> server,
>>>>>>>>>> I
>>>>>>>>>>>> believe the best way would be to put this script on ASF infra
>>>>>> and
>>>>>>>>> have
>>>>>>>>>>> the
>>>>>>>>>>>> "update checker" running in a completely controlled
>>>>> environment.
>>>>>>>>>>> Actually,
>>>>>>>>>>>> I
>>>>>>>>>>>> recall we had this very discussion during the Incubation; I
>>>>> can
>>>>>>>>>> probably
>>>>>>>>>>>> dig
>>>>>>>>>>>> out the corresponding thread.
>>>>>>>>>>>>
>>>>>>>>>>>> Thoughts?
>>>>>>>>>>>> Cok
>>>>>>>>>>>>
>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>>>>> Hey guys
>>>>>>>>>>>>>
>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>>>>> (Sonatype).
>>>>>>>>>>> We
>>>>>>>>>>>>> don't know what kind of data analysis or intelligence
>>>>>> extraction
>>>>>>>>> they
>>>>>>>>>>>> run.
>>>>>>>>>>>>>
>>>>>>>>>>>>> If Ignite servers all over the world were hitting maven.org
>>>>>>>>>>> periodically
>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>>>>> clear
>>>>>>>>>>> indication
>>>>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>>>>
>>>>>>>>>>>>> They could reverse lookup the IP address and find out what
>>>>>>>>>> corporation
>>>>>>>>>>> it
>>>>>>>>>>>>> is.
>>>>>>>>>>>>>
>>>>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>>>>
>>>>>>>>>>>>> We could use the Git ssh API, but that would require a new
>>>>>>>>>> dependency,
>>>>>>>>>>>>> which I advise against.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git
>>>>>>>> tags:
>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>>>>
>>>>>>>>>>>>> Another option is to place a txt file in the root of the
>>>>>> master
>>>>>>>>>> branch
>>>>>>>>>>>> (e.g
>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>>>>> each
>>>>>>>> major
>>>>>>>>>>>> version
>>>>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>>>>
>>>>>>>>>>>>> I would advocate this last option, but it requires somebody
>>>>>>>>>> remembering
>>>>>>>>>>>> to
>>>>>>>>>>>>> update the file with every release, unless we automate it
>>>>>> with a
>>>>>>>>>> Maven
>>>>>>>>>>>>> plugin.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hope that helps!
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>>>>> <javascript:;>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
>>>>> it
>>>>>>>> good
>>>>>>>>>> to
>>>>>>>>>>>> go?
>>>>>>>>>>>>>
>>>>>>>>>>>>> —
>>>>>>>>>>>>> Denis
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>>>>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
>>>>> on
>>>>>>>> it?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There has been lots of talk of proposals about various
>>>>>> usage
>>>>>>>>>> metrics
>>>>>>>>>>>> for
>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
>>>>>> the
>>>>>>>>> topic
>>>>>>>>>>> and
>>>>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>>>>> The main purpose of the update checker is not to collect
>>>>>>>>> metrics,
>>>>>>>>>>> but
>>>>>>>>>>>> to
>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>>>>> maven.org
>>>>>>>>>>> and
>>>>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>>>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> This way we do not send any information anywhere and, at
>>>>>> the
>>>>>>>>> same
>>>>>>>>>>>> time,
>>>>>>>>>>>>>>> urge our users to download and start using the latest
>>>>>>>> version of
>>>>>>>>>>>> Ignite.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight in
>>>>>> how
>>>>>>>>> many
>>>>>>>>>>>> times
>>>>>>>>>>>>> a
>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a cool
>>>>>>>> metric
>>>>>>>>>> for
>>>>>>>>>>>> the
>>>>>>>>>>>>>>> community to gauge the project popularity. You can think
>>>>> of
>>>>>>>> it
>>>>>>>>> as
>>>>>>>>>>> of a
>>>>>>>>>>>>> page
>>>>>>>>>>>>>>> visit counter shown on many websites. We can even decide
>>>>> to
>>>>>>>>>> display
>>>>>>>>>>>> this
>>>>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>>>>> release,
>>>>>>>>>> e.g.
>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
>>>>>>>> Every
>>>>>>>>>> time
>>>>>>>>>>>> an
>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>>>>> background.
>>>>>>>>>>> Then
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> will be able to view the number of the total downloads
>>>>> for
>>>>>>>> this
>>>>>>>>>> JAR
>>>>>>>>>>> in
>>>>>>>>>>>>>>> Maven Central, which is essentially the number of starts
>>>>> of
>>>>>>>>> Ignite
>>>>>>>>>>>> nodes.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> *Note that neither of the above suggestions require
>>>>> Ignite
>>>>>> to
>>>>>>>>> send
>>>>>>>>>>> or
>>>>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>>>>> approach.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> D.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>
Re: [DISCUSS] Ignite Update Checker
Posted by Dmitriy Setrakyan <ds...@apache.org>.
On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <vo...@gridgain.com>
wrote:
> I am not sure it is good idea to send requests to 3rd-party addresses from
> Ignite node. Let's do not make the same mistakes again.
>
Agree with Vladimir.
We obviously have CGI support on the website. Can someone explain why CGI
is not possible to use?
>
> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <an...@gridgain.com>
> wrote:
>
> > We may directly send request to GA from Ignite node:
> > https://developers.google.com/analytics/devguides/
> collection/protocol/v1/
> > <https://developers.google.com/analytics/devguides/
> collection/protocol/v1/
> > >
> > Latest version can be received from maven central:
> > https://repo1.maven.org/maven2/org/apache/ignite/
> > ignite-core/maven-metadata.xml <https://repo1.maven.org/
> > maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> >
> >
> > > 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
> > написал(а):
> > >
> > > Denis,
> > >
> > > I am not sure I understand. We already do have CGI enabled for
> > > download.cgi. Is there something else we need?
> > >
> > > D.
> > >
> > > On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com>
> wrote:
> > >
> > >> There is an obstacle. There is no way to execute the script using PHP
> or
> > >> similar sever side language and trigger GA as discussed earlier:
> > >> https://issues.apache.org/jira/browse/INFRA-15182
> > >>
> > >> How else can we tackle this?
> > >>
> > >> Denis
> > >>
> > >> On Thursday, September 7, 2017, Dmitriy Setrakyan <
> > dsetrakyan@apache.org>
> > >> wrote:
> > >>
> > >>> I think it is safe to assume at this point that everyone is in
> general
> > >>> agreement, since there are no active objections.
> > >>>
> > >>> I have filed a ticket for the 2.3 release. Let's try to make it
> happen:
> > >>> https://issues.apache.org/jira/browse/IGNITE-6305
> > >>>
> > >>> D.
> > >>>
> > >>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> > >> dsetrakyan@apache.org
> > >>> <javascript:;>>
> > >>> wrote:
> > >>>
> > >>>>
> > >>>>
> > >>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <
> raul.asf@evosent.com
> > >>> <javascript:;>>
> > >>>> wrote:
> > >>>>
> > >>>>> Yeah, I guess that's doable as well and requires less management
> > >> effort
> > >>>>> than my suggestion. We could use events [1] to store payload data
> > >> (e.g.
> > >>>>> IP,
> > >>>>> version, etc.)
> > >>>>
> > >>>>
> > >>>> Yes, we could use events or some other similar API provided by GA.
> > >>>>
> > >>>>
> > >>>>> What the download page CGI developed in? PHP?
> > >>>>>
> > >>>>
> > >>>> To be honest, no clue. I guess someone in the community can figure
> it
> > >>> out:
> > >>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > >>>>
> > >>>>
> > >>>>> However, I'm not sure whether storing this data in a 3rd party
> > >> (Google)
> > >>> is
> > >>>>> compliant with the ASF policy. I guess it's no biggie, but if
> there's
> > >>>>> doubt
> > >>>>> in the PMC, it's better to ask legal@.
> > >>>>
> > >>>>
> > >>>> I am not sure there is anything to ask about. The whole Ignite
> website
> > >> is
> > >>>> GA enabled, and all we are doing is accessing a standard web page
> from
> > >>> the
> > >>>> Ignite web site. The information gathered from GA is available only
> to
> > >>> the
> > >>>> Ignite PMC. Frankly, I think legal@ will be very confused by this
> > >>>> question.
> > >>>>
> > >>>> Even ASF website itself uses GA: https://www.apache.org/
> > >>>> foundation/policies/privacy.html
> > >>>>
> > >>>>
> > >>>>> I think Cos said it's OK; maybe Roman can pitch in.
> > >>>>>
> > >>>>
> > >>>> Sure, would be nice to hear from Roman as well.
> > >>>>
> > >>>>
> > >>>>> Cheers.
> > >>>>>
> > >>>>> [1]
> > >>>>> https://developers.google.com/analytics/devguides/collection
> > >>>>> /analyticsjs/events
> > >>>>>
> > >>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > >>>>> dsetrakyan@apache.org <javascript:;>>
> > >>>>> wrote:
> > >>>>>
> > >>>>>> Raul,
> > >>>>>>
> > >>>>>> Could point about Javascript, it will not work because it executes
> > >> in
> > >>>>> the
> > >>>>>> browser. This means we need a server-side script, like CGI we are
> > >>> using
> > >>>>> on
> > >>>>>> our download page.
> > >>>>>>
> > >>>>>> How about this approach. We create something like
> ignite-version.cgi
> > >>>>> script
> > >>>>>> which will invoke a call to GA and then return the latest version.
> > >>>>>>
> > >>>>>> This should work, right?
> > >>>>>>
> > >>>>>> D.
> > >>>>>>
> > >>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> > >> raul.asf@evosent.com
> > >>> <javascript:;>>
> > >>>>>> wrote:
> > >>>>>>
> > >>>>>>> Hey Dmitriy and all
> > >>>>>>>
> > >>>>>>> Also, since we have GA enabled for the website, we can track how
> > >>> many
> > >>>>>> times
> > >>>>>>>> this page was accessed, which will be equal to the number of
> > >>> starts.
> > >>>>>> This
> > >>>>>>>> way, the counter information is tracked and monitored by the
> > >>> Ignite
> > >>>>>> PMC.
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> Unfortunately this won't work because GA is loaded via Javascript
> > >> on
> > >>>>> the
> > >>>>>>> browser, so Google will never receive the page hit.
> > >>>>>>>
> > >>>>>>> Given the constraints, the most viable solution is an HTTPS
> > >> endpoint
> > >>>>>>> running on ASF infra that Ignite invokes via a GET or POST
> > >> request.
> > >>>>> The
> > >>>>>>> simplest thing is to write each request in a log file, along with
> > >>> the
> > >>>>>>> timestamp, the version reported by the client, maybe the IP (not
> > >>> sure
> > >>>>>> about
> > >>>>>>> the ASF rules about this concerning privacy, I guess it's OK if
> > >> you
> > >>>>> make
> > >>>>>> it
> > >>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
> > >>> creates
> > >>>>> on
> > >>>>>>> first startup or something.
> > >>>>>>>
> > >>>>>>> This endpoint would need some basic DDoS protection and
> > >> blacklisting
> > >>>>> to
> > >>>>>>> prevent data spoofing.
> > >>>>>>>
> > >>>>>>> If we'll be implementing this endpoint anyway, then there's no
> > >> point
> > >>>>>>> placing another file on Git or elsewhere for reporting the latest
> > >>>>>> versions:
> > >>>>>>> the endpoint itself can return them.
> > >>>>>>>
> > >>>>>>> WDYT?
> > >>>>>>>
> > >>>>>>> Cheers.
> > >>>>>>>
> > >>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > >>>>>> dsetrakyan@apache.org <javascript:;>>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>>> Cos, Raul,
> > >>>>>>>>
> > >>>>>>>> Thanks for the feedback. I completely agree about Maven Central
> > >>>>> being a
> > >>>>>>> 3rd
> > >>>>>>>> party repo (did not think about that initially). All your
> > >>>>> suggestions
> > >>>>>>> make
> > >>>>>>>> sense, but I would like to keep it as simple as possible, and so
> > >>> far
> > >>>>>>>> everything suggested required GIT dependencies and extra work.
> > >>>>>>>>
> > >>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite
> > >>>>>> website
> > >>>>>>>> which will have only the latest version. Every time a node
> > >> starts,
> > >>>>> it
> > >>>>>>>> receives the latest version from the page and suggests that
> > >> users
> > >>>>>> upgrade
> > >>>>>>>> if needed.
> > >>>>>>>>
> > >>>>>>>> Also, since we have GA enabled for the website, we can track how
> > >>>>> many
> > >>>>>>> times
> > >>>>>>>> this page was accessed, which will be equal to the number of
> > >>> starts.
> > >>>>>> This
> > >>>>>>>> way, the counter information is tracked and monitored by the
> > >>> Ignite
> > >>>>>> PMC.
> > >>>>>>>>
> > >>>>>>>> This approach looks pretty innocent to me and everything is kept
> > >>> and
> > >>>>>>>> managed within Apache.
> > >>>>>>>>
> > >>>>>>>> Thoughts?
> > >>>>>>>>
> > >>>>>>>> D.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > >>>>> cos@apache.org <javascript:;>>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>> I agree with Raul.
> > >>>>>>>>> - providing a ping-back address to a 3rd party might be frown
> > >>>>> upon by
> > >>>>>>>> some.
> > >>>>>>>>> And might have a consequences like stats collection about
> > >>> users'
> > >>>>>>>>> infrastructure.
> > >>>>>>>>> - checking an ASF git-repo is easy and won't download any
> > >> binary
> > >>>>>> data:
> > >>>>>>>>> everything is clear text and could be easily monitored by
> > >> any
> > >>> of
> > >>>>>> the
> > >>>>>>>>> network
> > >>>>>>>>> diagnostic tools, shall it be required. But it involves a
> > >> bit
> > >>> of
> > >>>>>> the
> > >>>>>>>>> release
> > >>>>>>>>> discipline.
> > >>>>>>>>> - the binary data download in the runtime is my main concern.
> > >>>>> This is
> > >>>>>>> the
> > >>>>>>>>> vector of MMA. What if someone gains the control over the
> > >>>>>> repository
> > >>>>>>>> and
> > >>>>>>>>> replaces the file with some malicious content.
> > >>>>>>>>>
> > >>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
> > >> call
> > >>>>> to
> > >>>>>> an
> > >>>>>>>>> external script to check upon the atest software version
> > >>> available
> > >>>>>> for
> > >>>>>>>>> download. In the past, the endpoint was running on a 3rd party
> > >>>>>> server,
> > >>>>>>> I
> > >>>>>>>>> believe the best way would be to put this script on ASF infra
> > >>> and
> > >>>>>> have
> > >>>>>>>> the
> > >>>>>>>>> "update checker" running in a completely controlled
> > >> environment.
> > >>>>>>>> Actually,
> > >>>>>>>>> I
> > >>>>>>>>> recall we had this very discussion during the Incubation; I
> > >> can
> > >>>>>>> probably
> > >>>>>>>>> dig
> > >>>>>>>>> out the corresponding thread.
> > >>>>>>>>>
> > >>>>>>>>> Thoughts?
> > >>>>>>>>> Cok
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > >>>>>>>>>> Hey guys
> > >>>>>>>>>>
> > >>>>>>>>>> In my opinion, maven.org is still owned by a third party
> > >>>>>> (Sonatype).
> > >>>>>>>> We
> > >>>>>>>>>> don't know what kind of data analysis or intelligence
> > >>> extraction
> > >>>>>> they
> > >>>>>>>>> run.
> > >>>>>>>>>>
> > >>>>>>>>>> If Ignite servers all over the world were hitting maven.org
> > >>>>>>>> periodically
> > >>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
> > >> clear
> > >>>>>>>> indication
> > >>>>>>>>>> of who is running an Ignite server.
> > >>>>>>>>>>
> > >>>>>>>>>> They could reverse lookup the IP address and find out what
> > >>>>>>> corporation
> > >>>>>>>> it
> > >>>>>>>>>> is.
> > >>>>>>>>>>
> > >>>>>>>>>> How about having Ignite check the ASF Git directly?
> > >>>>>>>>>>
> > >>>>>>>>>> We could use the Git ssh API, but that would require a new
> > >>>>>>> dependency,
> > >>>>>>>>>> which I advise against.
> > >>>>>>>>>>
> > >>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git
> > >>>>> tags:
> > >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >>>>>>>>>>
> > >>>>>>>>>> Another option is to place a txt file in the root of the
> > >>> master
> > >>>>>>> branch
> > >>>>>>>>> (e.g
> > >>>>>>>>>> LATEST), containing a list of the latest GA versions for
> > >> each
> > >>>>> major
> > >>>>>>>>> version
> > >>>>>>>>>> line (1.x, 2.x).
> > >>>>>>>>>>
> > >>>>>>>>>> I would advocate this last option, but it requires somebody
> > >>>>>>> remembering
> > >>>>>>>>> to
> > >>>>>>>>>> update the file with every release, unless we automate it
> > >>> with a
> > >>>>>>> Maven
> > >>>>>>>>>> plugin.
> > >>>>>>>>>>
> > >>>>>>>>>> Hope that helps!
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> > >>> <javascript:;>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> I see nothing wrong with this approach.
> > >>>>>>>>>>
> > >>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
> > >> it
> > >>>>> good
> > >>>>>>> to
> > >>>>>>>>> go?
> > >>>>>>>>>>
> > >>>>>>>>>> —
> > >>>>>>>>>> Denis
> > >>>>>>>>>>
> > >>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > >>>>>>>> dsetrakyan@apache.org <javascript:;>
> > >>>>>>>>>>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
> > >> on
> > >>>>> it?
> > >>>>>>>>>>>
> > >>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > >>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> > >>>>>>>>>>> wrote:
> > >>>>>>>>>>>
> > >>>>>>>>>>>> Igniters,
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> There has been lots of talk of proposals about various
> > >>> usage
> > >>>>>>> metrics
> > >>>>>>>>> for
> > >>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
> > >>> the
> > >>>>>> topic
> > >>>>>>>> and
> > >>>>>>>>>>>> propose something very simple and non-intrusive.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 1. Update Checker
> > >>>>>>>>>>>> The main purpose of the update checker is not to collect
> > >>>>>> metrics,
> > >>>>>>>> but
> > >>>>>>>>> to
> > >>>>>>>>>>>> notify users about a new version of Ignite by accessing
> > >>>>>> maven.org
> > >>>>>>>> and
> > >>>>>>>>>>>> getting the version out of the metadata file:
> > >>>>>>>>>>>> http://repo2.maven.org/maven2/
> > >>> org/apache/ignite/ignite-core/
> > >>>>>>>>>>>> maven-metadata.xml
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> This way we do not send any information anywhere and, at
> > >>> the
> > >>>>>> same
> > >>>>>>>>> time,
> > >>>>>>>>>>>> urge our users to download and start using the latest
> > >>>>> version of
> > >>>>>>>>> Ignite.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> 2. Startup Counter
> > >>>>>>>>>>>> This piece is optional, but we can also get an insight in
> > >>> how
> > >>>>>> many
> > >>>>>>>>> times
> > >>>>>>>>>> a
> > >>>>>>>>>>>> certain Ignite release gets started. This is just a cool
> > >>>>> metric
> > >>>>>>> for
> > >>>>>>>>> the
> > >>>>>>>>>>>> community to gauge the project popularity. You can think
> > >> of
> > >>>>> it
> > >>>>>> as
> > >>>>>>>> of a
> > >>>>>>>>>> page
> > >>>>>>>>>>>> visit counter shown on many websites. We can even decide
> > >> to
> > >>>>>>> display
> > >>>>>>>>> this
> > >>>>>>>>>>>> counter on the Ignite website as well.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
> > >>>>> release,
> > >>>>>>> e.g.
> > >>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
> > >>>>> Every
> > >>>>>>> time
> > >>>>>>>>> an
> > >>>>>>>>>>>> Ignite node starts, it will download this JAR in the
> > >>>>> background.
> > >>>>>>>> Then
> > >>>>>>>>> we
> > >>>>>>>>>>>> will be able to view the number of the total downloads
> > >> for
> > >>>>> this
> > >>>>>>> JAR
> > >>>>>>>> in
> > >>>>>>>>>>>> Maven Central, which is essentially the number of starts
> > >> of
> > >>>>>> Ignite
> > >>>>>>>>> nodes.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> *Note that neither of the above suggestions require
> > >> Ignite
> > >>> to
> > >>>>>> send
> > >>>>>>>> or
> > >>>>>>>>>>>> track any user information whatsoever.*
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> Please reply suggesting weather you are OK with this
> > >>>>> approach.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> D.
> > >>>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> >
> >
>
Re: [DISCUSS] Ignite Update Checker
Posted by Vladimir Ozerov <vo...@gridgain.com>.
I am not sure it is good idea to send requests to 3rd-party addresses from
Ignite node. Let's do not make the same mistakes again.
On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov <an...@gridgain.com>
wrote:
> We may directly send request to GA from Ignite node:
> https://developers.google.com/analytics/devguides/collection/protocol/v1/
> <https://developers.google.com/analytics/devguides/collection/protocol/v1/
> >
> Latest version can be received from maven central:
> https://repo1.maven.org/maven2/org/apache/ignite/
> ignite-core/maven-metadata.xml <https://repo1.maven.org/
> maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
>
>
> > 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org>
> написал(а):
> >
> > Denis,
> >
> > I am not sure I understand. We already do have CGI enabled for
> > download.cgi. Is there something else we need?
> >
> > D.
> >
> > On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com> wrote:
> >
> >> There is an obstacle. There is no way to execute the script using PHP or
> >> similar sever side language and trigger GA as discussed earlier:
> >> https://issues.apache.org/jira/browse/INFRA-15182
> >>
> >> How else can we tackle this?
> >>
> >> Denis
> >>
> >> On Thursday, September 7, 2017, Dmitriy Setrakyan <
> dsetrakyan@apache.org>
> >> wrote:
> >>
> >>> I think it is safe to assume at this point that everyone is in general
> >>> agreement, since there are no active objections.
> >>>
> >>> I have filed a ticket for the 2.3 release. Let's try to make it happen:
> >>> https://issues.apache.org/jira/browse/IGNITE-6305
> >>>
> >>> D.
> >>>
> >>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> >> dsetrakyan@apache.org
> >>> <javascript:;>>
> >>> wrote:
> >>>
> >>>>
> >>>>
> >>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com
> >>> <javascript:;>>
> >>>> wrote:
> >>>>
> >>>>> Yeah, I guess that's doable as well and requires less management
> >> effort
> >>>>> than my suggestion. We could use events [1] to store payload data
> >> (e.g.
> >>>>> IP,
> >>>>> version, etc.)
> >>>>
> >>>>
> >>>> Yes, we could use events or some other similar API provided by GA.
> >>>>
> >>>>
> >>>>> What the download page CGI developed in? PHP?
> >>>>>
> >>>>
> >>>> To be honest, no clue. I guess someone in the community can figure it
> >>> out:
> >>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> >>>>
> >>>>
> >>>>> However, I'm not sure whether storing this data in a 3rd party
> >> (Google)
> >>> is
> >>>>> compliant with the ASF policy. I guess it's no biggie, but if there's
> >>>>> doubt
> >>>>> in the PMC, it's better to ask legal@.
> >>>>
> >>>>
> >>>> I am not sure there is anything to ask about. The whole Ignite website
> >> is
> >>>> GA enabled, and all we are doing is accessing a standard web page from
> >>> the
> >>>> Ignite web site. The information gathered from GA is available only to
> >>> the
> >>>> Ignite PMC. Frankly, I think legal@ will be very confused by this
> >>>> question.
> >>>>
> >>>> Even ASF website itself uses GA: https://www.apache.org/
> >>>> foundation/policies/privacy.html
> >>>>
> >>>>
> >>>>> I think Cos said it's OK; maybe Roman can pitch in.
> >>>>>
> >>>>
> >>>> Sure, would be nice to hear from Roman as well.
> >>>>
> >>>>
> >>>>> Cheers.
> >>>>>
> >>>>> [1]
> >>>>> https://developers.google.com/analytics/devguides/collection
> >>>>> /analyticsjs/events
> >>>>>
> >>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> >>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>> wrote:
> >>>>>
> >>>>>> Raul,
> >>>>>>
> >>>>>> Could point about Javascript, it will not work because it executes
> >> in
> >>>>> the
> >>>>>> browser. This means we need a server-side script, like CGI we are
> >>> using
> >>>>> on
> >>>>>> our download page.
> >>>>>>
> >>>>>> How about this approach. We create something like ignite-version.cgi
> >>>>> script
> >>>>>> which will invoke a call to GA and then return the latest version.
> >>>>>>
> >>>>>> This should work, right?
> >>>>>>
> >>>>>> D.
> >>>>>>
> >>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> >> raul.asf@evosent.com
> >>> <javascript:;>>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hey Dmitriy and all
> >>>>>>>
> >>>>>>> Also, since we have GA enabled for the website, we can track how
> >>> many
> >>>>>> times
> >>>>>>>> this page was accessed, which will be equal to the number of
> >>> starts.
> >>>>>> This
> >>>>>>>> way, the counter information is tracked and monitored by the
> >>> Ignite
> >>>>>> PMC.
> >>>>>>>
> >>>>>>>
> >>>>>>> Unfortunately this won't work because GA is loaded via Javascript
> >> on
> >>>>> the
> >>>>>>> browser, so Google will never receive the page hit.
> >>>>>>>
> >>>>>>> Given the constraints, the most viable solution is an HTTPS
> >> endpoint
> >>>>>>> running on ASF infra that Ignite invokes via a GET or POST
> >> request.
> >>>>> The
> >>>>>>> simplest thing is to write each request in a log file, along with
> >>> the
> >>>>>>> timestamp, the version reported by the client, maybe the IP (not
> >>> sure
> >>>>>> about
> >>>>>>> the ASF rules about this concerning privacy, I guess it's OK if
> >> you
> >>>>> make
> >>>>>> it
> >>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
> >>> creates
> >>>>> on
> >>>>>>> first startup or something.
> >>>>>>>
> >>>>>>> This endpoint would need some basic DDoS protection and
> >> blacklisting
> >>>>> to
> >>>>>>> prevent data spoofing.
> >>>>>>>
> >>>>>>> If we'll be implementing this endpoint anyway, then there's no
> >> point
> >>>>>>> placing another file on Git or elsewhere for reporting the latest
> >>>>>> versions:
> >>>>>>> the endpoint itself can return them.
> >>>>>>>
> >>>>>>> WDYT?
> >>>>>>>
> >>>>>>> Cheers.
> >>>>>>>
> >>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> >>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>>> Cos, Raul,
> >>>>>>>>
> >>>>>>>> Thanks for the feedback. I completely agree about Maven Central
> >>>>> being a
> >>>>>>> 3rd
> >>>>>>>> party repo (did not think about that initially). All your
> >>>>> suggestions
> >>>>>>> make
> >>>>>>>> sense, but I would like to keep it as simple as possible, and so
> >>> far
> >>>>>>>> everything suggested required GIT dependencies and extra work.
> >>>>>>>>
> >>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite
> >>>>>> website
> >>>>>>>> which will have only the latest version. Every time a node
> >> starts,
> >>>>> it
> >>>>>>>> receives the latest version from the page and suggests that
> >> users
> >>>>>> upgrade
> >>>>>>>> if needed.
> >>>>>>>>
> >>>>>>>> Also, since we have GA enabled for the website, we can track how
> >>>>> many
> >>>>>>> times
> >>>>>>>> this page was accessed, which will be equal to the number of
> >>> starts.
> >>>>>> This
> >>>>>>>> way, the counter information is tracked and monitored by the
> >>> Ignite
> >>>>>> PMC.
> >>>>>>>>
> >>>>>>>> This approach looks pretty innocent to me and everything is kept
> >>> and
> >>>>>>>> managed within Apache.
> >>>>>>>>
> >>>>>>>> Thoughts?
> >>>>>>>>
> >>>>>>>> D.
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> >>>>> cos@apache.org <javascript:;>>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> I agree with Raul.
> >>>>>>>>> - providing a ping-back address to a 3rd party might be frown
> >>>>> upon by
> >>>>>>>> some.
> >>>>>>>>> And might have a consequences like stats collection about
> >>> users'
> >>>>>>>>> infrastructure.
> >>>>>>>>> - checking an ASF git-repo is easy and won't download any
> >> binary
> >>>>>> data:
> >>>>>>>>> everything is clear text and could be easily monitored by
> >> any
> >>> of
> >>>>>> the
> >>>>>>>>> network
> >>>>>>>>> diagnostic tools, shall it be required. But it involves a
> >> bit
> >>> of
> >>>>>> the
> >>>>>>>>> release
> >>>>>>>>> discipline.
> >>>>>>>>> - the binary data download in the runtime is my main concern.
> >>>>> This is
> >>>>>>> the
> >>>>>>>>> vector of MMA. What if someone gains the control over the
> >>>>>> repository
> >>>>>>>> and
> >>>>>>>>> replaces the file with some malicious content.
> >>>>>>>>>
> >>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
> >> call
> >>>>> to
> >>>>>> an
> >>>>>>>>> external script to check upon the atest software version
> >>> available
> >>>>>> for
> >>>>>>>>> download. In the past, the endpoint was running on a 3rd party
> >>>>>> server,
> >>>>>>> I
> >>>>>>>>> believe the best way would be to put this script on ASF infra
> >>> and
> >>>>>> have
> >>>>>>>> the
> >>>>>>>>> "update checker" running in a completely controlled
> >> environment.
> >>>>>>>> Actually,
> >>>>>>>>> I
> >>>>>>>>> recall we had this very discussion during the Incubation; I
> >> can
> >>>>>>> probably
> >>>>>>>>> dig
> >>>>>>>>> out the corresponding thread.
> >>>>>>>>>
> >>>>>>>>> Thoughts?
> >>>>>>>>> Cok
> >>>>>>>>>
> >>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> >>>>>>>>>> Hey guys
> >>>>>>>>>>
> >>>>>>>>>> In my opinion, maven.org is still owned by a third party
> >>>>>> (Sonatype).
> >>>>>>>> We
> >>>>>>>>>> don't know what kind of data analysis or intelligence
> >>> extraction
> >>>>>> they
> >>>>>>>>> run.
> >>>>>>>>>>
> >>>>>>>>>> If Ignite servers all over the world were hitting maven.org
> >>>>>>>> periodically
> >>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
> >> clear
> >>>>>>>> indication
> >>>>>>>>>> of who is running an Ignite server.
> >>>>>>>>>>
> >>>>>>>>>> They could reverse lookup the IP address and find out what
> >>>>>>> corporation
> >>>>>>>> it
> >>>>>>>>>> is.
> >>>>>>>>>>
> >>>>>>>>>> How about having Ignite check the ASF Git directly?
> >>>>>>>>>>
> >>>>>>>>>> We could use the Git ssh API, but that would require a new
> >>>>>>> dependency,
> >>>>>>>>>> which I advise against.
> >>>>>>>>>>
> >>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git
> >>>>> tags:
> >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
> >>>>>>>>>>
> >>>>>>>>>> Another option is to place a txt file in the root of the
> >>> master
> >>>>>>> branch
> >>>>>>>>> (e.g
> >>>>>>>>>> LATEST), containing a list of the latest GA versions for
> >> each
> >>>>> major
> >>>>>>>>> version
> >>>>>>>>>> line (1.x, 2.x).
> >>>>>>>>>>
> >>>>>>>>>> I would advocate this last option, but it requires somebody
> >>>>>>> remembering
> >>>>>>>>> to
> >>>>>>>>>> update the file with every release, unless we automate it
> >>> with a
> >>>>>>> Maven
> >>>>>>>>>> plugin.
> >>>>>>>>>>
> >>>>>>>>>> Hope that helps!
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> >>> <javascript:;>> wrote:
> >>>>>>>>>>
> >>>>>>>>>> I see nothing wrong with this approach.
> >>>>>>>>>>
> >>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
> >> it
> >>>>> good
> >>>>>>> to
> >>>>>>>>> go?
> >>>>>>>>>>
> >>>>>>>>>> —
> >>>>>>>>>> Denis
> >>>>>>>>>>
> >>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> >>>>>>>> dsetrakyan@apache.org <javascript:;>
> >>>>>>>>>>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
> >> on
> >>>>> it?
> >>>>>>>>>>>
> >>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> >>>>>>>>> dsetrakyan@apache.org <javascript:;>>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>> Igniters,
> >>>>>>>>>>>>
> >>>>>>>>>>>> There has been lots of talk of proposals about various
> >>> usage
> >>>>>>> metrics
> >>>>>>>>> for
> >>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
> >>> the
> >>>>>> topic
> >>>>>>>> and
> >>>>>>>>>>>> propose something very simple and non-intrusive.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1. Update Checker
> >>>>>>>>>>>> The main purpose of the update checker is not to collect
> >>>>>> metrics,
> >>>>>>>> but
> >>>>>>>>> to
> >>>>>>>>>>>> notify users about a new version of Ignite by accessing
> >>>>>> maven.org
> >>>>>>>> and
> >>>>>>>>>>>> getting the version out of the metadata file:
> >>>>>>>>>>>> http://repo2.maven.org/maven2/
> >>> org/apache/ignite/ignite-core/
> >>>>>>>>>>>> maven-metadata.xml
> >>>>>>>>>>>>
> >>>>>>>>>>>> This way we do not send any information anywhere and, at
> >>> the
> >>>>>> same
> >>>>>>>>> time,
> >>>>>>>>>>>> urge our users to download and start using the latest
> >>>>> version of
> >>>>>>>>> Ignite.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2. Startup Counter
> >>>>>>>>>>>> This piece is optional, but we can also get an insight in
> >>> how
> >>>>>> many
> >>>>>>>>> times
> >>>>>>>>>> a
> >>>>>>>>>>>> certain Ignite release gets started. This is just a cool
> >>>>> metric
> >>>>>>> for
> >>>>>>>>> the
> >>>>>>>>>>>> community to gauge the project popularity. You can think
> >> of
> >>>>> it
> >>>>>> as
> >>>>>>>> of a
> >>>>>>>>>> page
> >>>>>>>>>>>> visit counter shown on many websites. We can even decide
> >> to
> >>>>>>> display
> >>>>>>>>> this
> >>>>>>>>>>>> counter on the Ignite website as well.
> >>>>>>>>>>>>
> >>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
> >>>>> release,
> >>>>>>> e.g.
> >>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
> >>>>> Every
> >>>>>>> time
> >>>>>>>>> an
> >>>>>>>>>>>> Ignite node starts, it will download this JAR in the
> >>>>> background.
> >>>>>>>> Then
> >>>>>>>>> we
> >>>>>>>>>>>> will be able to view the number of the total downloads
> >> for
> >>>>> this
> >>>>>>> JAR
> >>>>>>>> in
> >>>>>>>>>>>> Maven Central, which is essentially the number of starts
> >> of
> >>>>>> Ignite
> >>>>>>>>> nodes.
> >>>>>>>>>>>>
> >>>>>>>>>>>> *Note that neither of the above suggestions require
> >> Ignite
> >>> to
> >>>>>> send
> >>>>>>>> or
> >>>>>>>>>>>> track any user information whatsoever.*
> >>>>>>>>>>>>
> >>>>>>>>>>>> Please reply suggesting weather you are OK with this
> >>>>> approach.
> >>>>>>>>>>>>
> >>>>>>>>>>>> D.
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>>
> >>
>
>
Re: [DISCUSS] Ignite Update Checker
Posted by Andrey Novikov <an...@gridgain.com>.
We may directly send request to GA from Ignite node: https://developers.google.com/analytics/devguides/collection/protocol/v1/ <https://developers.google.com/analytics/devguides/collection/protocol/v1/>
Latest version can be received from maven central: https://repo1.maven.org/maven2/org/apache/ignite/ignite-core/maven-metadata.xml <https://repo1.maven.org/maven2/org/apache/ignite/ignite-core/maven-metadata.xml>
> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <ds...@apache.org> написал(а):
>
> Denis,
>
> I am not sure I understand. We already do have CGI enabled for
> download.cgi. Is there something else we need?
>
> D.
>
> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com> wrote:
>
>> There is an obstacle. There is no way to execute the script using PHP or
>> similar sever side language and trigger GA as discussed earlier:
>> https://issues.apache.org/jira/browse/INFRA-15182
>>
>> How else can we tackle this?
>>
>> Denis
>>
>> On Thursday, September 7, 2017, Dmitriy Setrakyan <ds...@apache.org>
>> wrote:
>>
>>> I think it is safe to assume at this point that everyone is in general
>>> agreement, since there are no active objections.
>>>
>>> I have filed a ticket for the 2.3 release. Let's try to make it happen:
>>> https://issues.apache.org/jira/browse/IGNITE-6305
>>>
>>> D.
>>>
>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
>> dsetrakyan@apache.org
>>> <javascript:;>>
>>> wrote:
>>>
>>>>
>>>>
>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com
>>> <javascript:;>>
>>>> wrote:
>>>>
>>>>> Yeah, I guess that's doable as well and requires less management
>> effort
>>>>> than my suggestion. We could use events [1] to store payload data
>> (e.g.
>>>>> IP,
>>>>> version, etc.)
>>>>
>>>>
>>>> Yes, we could use events or some other similar API provided by GA.
>>>>
>>>>
>>>>> What the download page CGI developed in? PHP?
>>>>>
>>>>
>>>> To be honest, no clue. I guess someone in the community can figure it
>>> out:
>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
>>>>
>>>>
>>>>> However, I'm not sure whether storing this data in a 3rd party
>> (Google)
>>> is
>>>>> compliant with the ASF policy. I guess it's no biggie, but if there's
>>>>> doubt
>>>>> in the PMC, it's better to ask legal@.
>>>>
>>>>
>>>> I am not sure there is anything to ask about. The whole Ignite website
>> is
>>>> GA enabled, and all we are doing is accessing a standard web page from
>>> the
>>>> Ignite web site. The information gathered from GA is available only to
>>> the
>>>> Ignite PMC. Frankly, I think legal@ will be very confused by this
>>>> question.
>>>>
>>>> Even ASF website itself uses GA: https://www.apache.org/
>>>> foundation/policies/privacy.html
>>>>
>>>>
>>>>> I think Cos said it's OK; maybe Roman can pitch in.
>>>>>
>>>>
>>>> Sure, would be nice to hear from Roman as well.
>>>>
>>>>
>>>>> Cheers.
>>>>>
>>>>> [1]
>>>>> https://developers.google.com/analytics/devguides/collection
>>>>> /analyticsjs/events
>>>>>
>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>> wrote:
>>>>>
>>>>>> Raul,
>>>>>>
>>>>>> Could point about Javascript, it will not work because it executes
>> in
>>>>> the
>>>>>> browser. This means we need a server-side script, like CGI we are
>>> using
>>>>> on
>>>>>> our download page.
>>>>>>
>>>>>> How about this approach. We create something like ignite-version.cgi
>>>>> script
>>>>>> which will invoke a call to GA and then return the latest version.
>>>>>>
>>>>>> This should work, right?
>>>>>>
>>>>>> D.
>>>>>>
>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
>> raul.asf@evosent.com
>>> <javascript:;>>
>>>>>> wrote:
>>>>>>
>>>>>>> Hey Dmitriy and all
>>>>>>>
>>>>>>> Also, since we have GA enabled for the website, we can track how
>>> many
>>>>>> times
>>>>>>>> this page was accessed, which will be equal to the number of
>>> starts.
>>>>>> This
>>>>>>>> way, the counter information is tracked and monitored by the
>>> Ignite
>>>>>> PMC.
>>>>>>>
>>>>>>>
>>>>>>> Unfortunately this won't work because GA is loaded via Javascript
>> on
>>>>> the
>>>>>>> browser, so Google will never receive the page hit.
>>>>>>>
>>>>>>> Given the constraints, the most viable solution is an HTTPS
>> endpoint
>>>>>>> running on ASF infra that Ignite invokes via a GET or POST
>> request.
>>>>> The
>>>>>>> simplest thing is to write each request in a log file, along with
>>> the
>>>>>>> timestamp, the version reported by the client, maybe the IP (not
>>> sure
>>>>>> about
>>>>>>> the ASF rules about this concerning privacy, I guess it's OK if
>> you
>>>>> make
>>>>>> it
>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node
>>> creates
>>>>> on
>>>>>>> first startup or something.
>>>>>>>
>>>>>>> This endpoint would need some basic DDoS protection and
>> blacklisting
>>>>> to
>>>>>>> prevent data spoofing.
>>>>>>>
>>>>>>> If we'll be implementing this endpoint anyway, then there's no
>> point
>>>>>>> placing another file on Git or elsewhere for reporting the latest
>>>>>> versions:
>>>>>>> the endpoint itself can return them.
>>>>>>>
>>>>>>> WDYT?
>>>>>>>
>>>>>>> Cheers.
>>>>>>>
>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Cos, Raul,
>>>>>>>>
>>>>>>>> Thanks for the feedback. I completely agree about Maven Central
>>>>> being a
>>>>>>> 3rd
>>>>>>>> party repo (did not think about that initially). All your
>>>>> suggestions
>>>>>>> make
>>>>>>>> sense, but I would like to keep it as simple as possible, and so
>>> far
>>>>>>>> everything suggested required GIT dependencies and extra work.
>>>>>>>>
>>>>>>>> How about Yakov's suggestion. We simply add a page to the Ignite
>>>>>> website
>>>>>>>> which will have only the latest version. Every time a node
>> starts,
>>>>> it
>>>>>>>> receives the latest version from the page and suggests that
>> users
>>>>>> upgrade
>>>>>>>> if needed.
>>>>>>>>
>>>>>>>> Also, since we have GA enabled for the website, we can track how
>>>>> many
>>>>>>> times
>>>>>>>> this page was accessed, which will be equal to the number of
>>> starts.
>>>>>> This
>>>>>>>> way, the counter information is tracked and monitored by the
>>> Ignite
>>>>>> PMC.
>>>>>>>>
>>>>>>>> This approach looks pretty innocent to me and everything is kept
>>> and
>>>>>>>> managed within Apache.
>>>>>>>>
>>>>>>>> Thoughts?
>>>>>>>>
>>>>>>>> D.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
>>>>> cos@apache.org <javascript:;>>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I agree with Raul.
>>>>>>>>> - providing a ping-back address to a 3rd party might be frown
>>>>> upon by
>>>>>>>> some.
>>>>>>>>> And might have a consequences like stats collection about
>>> users'
>>>>>>>>> infrastructure.
>>>>>>>>> - checking an ASF git-repo is easy and won't download any
>> binary
>>>>>> data:
>>>>>>>>> everything is clear text and could be easily monitored by
>> any
>>> of
>>>>>> the
>>>>>>>>> network
>>>>>>>>> diagnostic tools, shall it be required. But it involves a
>> bit
>>> of
>>>>>> the
>>>>>>>>> release
>>>>>>>>> discipline.
>>>>>>>>> - the binary data download in the runtime is my main concern.
>>>>> This is
>>>>>>> the
>>>>>>>>> vector of MMA. What if someone gains the control over the
>>>>>> repository
>>>>>>>> and
>>>>>>>>> replaces the file with some malicious content.
>>>>>>>>>
>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a
>> call
>>>>> to
>>>>>> an
>>>>>>>>> external script to check upon the atest software version
>>> available
>>>>>> for
>>>>>>>>> download. In the past, the endpoint was running on a 3rd party
>>>>>> server,
>>>>>>> I
>>>>>>>>> believe the best way would be to put this script on ASF infra
>>> and
>>>>>> have
>>>>>>>> the
>>>>>>>>> "update checker" running in a completely controlled
>> environment.
>>>>>>>> Actually,
>>>>>>>>> I
>>>>>>>>> recall we had this very discussion during the Incubation; I
>> can
>>>>>>> probably
>>>>>>>>> dig
>>>>>>>>> out the corresponding thread.
>>>>>>>>>
>>>>>>>>> Thoughts?
>>>>>>>>> Cok
>>>>>>>>>
>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
>>>>>>>>>> Hey guys
>>>>>>>>>>
>>>>>>>>>> In my opinion, maven.org is still owned by a third party
>>>>>> (Sonatype).
>>>>>>>> We
>>>>>>>>>> don't know what kind of data analysis or intelligence
>>> extraction
>>>>>> they
>>>>>>>>> run.
>>>>>>>>>>
>>>>>>>>>> If Ignite servers all over the world were hitting maven.org
>>>>>>>> periodically
>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a
>> clear
>>>>>>>> indication
>>>>>>>>>> of who is running an Ignite server.
>>>>>>>>>>
>>>>>>>>>> They could reverse lookup the IP address and find out what
>>>>>>> corporation
>>>>>>>> it
>>>>>>>>>> is.
>>>>>>>>>>
>>>>>>>>>> How about having Ignite check the ASF Git directly?
>>>>>>>>>>
>>>>>>>>>> We could use the Git ssh API, but that would require a new
>>>>>>> dependency,
>>>>>>>>>> which I advise against.
>>>>>>>>>>
>>>>>>>>>> Alternatively, we could have it scrape this HTML for new Git
>>>>> tags:
>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git
>>>>>>>>>>
>>>>>>>>>> Another option is to place a txt file in the root of the
>>> master
>>>>>>> branch
>>>>>>>>> (e.g
>>>>>>>>>> LATEST), containing a list of the latest GA versions for
>> each
>>>>> major
>>>>>>>>> version
>>>>>>>>>> line (1.x, 2.x).
>>>>>>>>>>
>>>>>>>>>> I would advocate this last option, but it requires somebody
>>>>>>> remembering
>>>>>>>>> to
>>>>>>>>>> update the file with every release, unless we automate it
>>> with a
>>>>>>> Maven
>>>>>>>>>> plugin.
>>>>>>>>>>
>>>>>>>>>> Hope that helps!
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
>>> <javascript:;>> wrote:
>>>>>>>>>>
>>>>>>>>>> I see nothing wrong with this approach.
>>>>>>>>>>
>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is
>> it
>>>>> good
>>>>>>> to
>>>>>>>>> go?
>>>>>>>>>>
>>>>>>>>>> —
>>>>>>>>>> Denis
>>>>>>>>>>
>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
>>>>>>>> dsetrakyan@apache.org <javascript:;>
>>>>>>>>>>
>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket
>> on
>>>>> it?
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
>>>>>>>>> dsetrakyan@apache.org <javascript:;>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Igniters,
>>>>>>>>>>>>
>>>>>>>>>>>> There has been lots of talk of proposals about various
>>> usage
>>>>>>> metrics
>>>>>>>>> for
>>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect
>>> the
>>>>>> topic
>>>>>>>> and
>>>>>>>>>>>> propose something very simple and non-intrusive.
>>>>>>>>>>>>
>>>>>>>>>>>> 1. Update Checker
>>>>>>>>>>>> The main purpose of the update checker is not to collect
>>>>>> metrics,
>>>>>>>> but
>>>>>>>>> to
>>>>>>>>>>>> notify users about a new version of Ignite by accessing
>>>>>> maven.org
>>>>>>>> and
>>>>>>>>>>>> getting the version out of the metadata file:
>>>>>>>>>>>> http://repo2.maven.org/maven2/
>>> org/apache/ignite/ignite-core/
>>>>>>>>>>>> maven-metadata.xml
>>>>>>>>>>>>
>>>>>>>>>>>> This way we do not send any information anywhere and, at
>>> the
>>>>>> same
>>>>>>>>> time,
>>>>>>>>>>>> urge our users to download and start using the latest
>>>>> version of
>>>>>>>>> Ignite.
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Startup Counter
>>>>>>>>>>>> This piece is optional, but we can also get an insight in
>>> how
>>>>>> many
>>>>>>>>> times
>>>>>>>>>> a
>>>>>>>>>>>> certain Ignite release gets started. This is just a cool
>>>>> metric
>>>>>>> for
>>>>>>>>> the
>>>>>>>>>>>> community to gauge the project popularity. You can think
>> of
>>>>> it
>>>>>> as
>>>>>>>> of a
>>>>>>>>>> page
>>>>>>>>>>>> visit counter shown on many websites. We can even decide
>> to
>>>>>>> display
>>>>>>>>> this
>>>>>>>>>>>> counter on the Ignite website as well.
>>>>>>>>>>>>
>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every
>>>>> release,
>>>>>>> e.g.
>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte.
>>>>> Every
>>>>>>> time
>>>>>>>>> an
>>>>>>>>>>>> Ignite node starts, it will download this JAR in the
>>>>> background.
>>>>>>>> Then
>>>>>>>>> we
>>>>>>>>>>>> will be able to view the number of the total downloads
>> for
>>>>> this
>>>>>>> JAR
>>>>>>>> in
>>>>>>>>>>>> Maven Central, which is essentially the number of starts
>> of
>>>>>> Ignite
>>>>>>>>> nodes.
>>>>>>>>>>>>
>>>>>>>>>>>> *Note that neither of the above suggestions require
>> Ignite
>>> to
>>>>>> send
>>>>>>>> or
>>>>>>>>>>>> track any user information whatsoever.*
>>>>>>>>>>>>
>>>>>>>>>>>> Please reply suggesting weather you are OK with this
>>>>> approach.
>>>>>>>>>>>>
>>>>>>>>>>>> D.
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
Re: [DISCUSS] Ignite Update Checker
Posted by Dmitriy Setrakyan <ds...@apache.org>.
Denis,
I am not sure I understand. We already do have CGI enabled for
download.cgi. Is there something else we need?
D.
On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dm...@gridgain.com> wrote:
> There is an obstacle. There is no way to execute the script using PHP or
> similar sever side language and trigger GA as discussed earlier:
> https://issues.apache.org/jira/browse/INFRA-15182
>
> How else can we tackle this?
>
> Denis
>
> On Thursday, September 7, 2017, Dmitriy Setrakyan <ds...@apache.org>
> wrote:
>
> > I think it is safe to assume at this point that everyone is in general
> > agreement, since there are no active objections.
> >
> > I have filed a ticket for the 2.3 release. Let's try to make it happen:
> > https://issues.apache.org/jira/browse/IGNITE-6305
> >
> > D.
> >
> > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan <
> dsetrakyan@apache.org
> > <javascript:;>>
> > wrote:
> >
> > >
> > >
> > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul.asf@evosent.com
> > <javascript:;>>
> > > wrote:
> > >
> > >> Yeah, I guess that's doable as well and requires less management
> effort
> > >> than my suggestion. We could use events [1] to store payload data
> (e.g.
> > >> IP,
> > >> version, etc.)
> > >
> > >
> > > Yes, we could use events or some other similar API provided by GA.
> > >
> > >
> > >> What the download page CGI developed in? PHP?
> > >>
> > >
> > > To be honest, no clue. I guess someone in the community can figure it
> > out:
> > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html
> > >
> > >
> > >> However, I'm not sure whether storing this data in a 3rd party
> (Google)
> > is
> > >> compliant with the ASF policy. I guess it's no biggie, but if there's
> > >> doubt
> > >> in the PMC, it's better to ask legal@.
> > >
> > >
> > > I am not sure there is anything to ask about. The whole Ignite website
> is
> > > GA enabled, and all we are doing is accessing a standard web page from
> > the
> > > Ignite web site. The information gathered from GA is available only to
> > the
> > > Ignite PMC. Frankly, I think legal@ will be very confused by this
> > > question.
> > >
> > > Even ASF website itself uses GA: https://www.apache.org/
> > > foundation/policies/privacy.html
> > >
> > >
> > >> I think Cos said it's OK; maybe Roman can pitch in.
> > >>
> > >
> > > Sure, would be nice to hear from Roman as well.
> > >
> > >
> > >> Cheers.
> > >>
> > >> [1]
> > >> https://developers.google.com/analytics/devguides/collection
> > >> /analyticsjs/events
> > >>
> > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan <
> > >> dsetrakyan@apache.org <javascript:;>>
> > >> wrote:
> > >>
> > >> > Raul,
> > >> >
> > >> > Could point about Javascript, it will not work because it executes
> in
> > >> the
> > >> > browser. This means we need a server-side script, like CGI we are
> > using
> > >> on
> > >> > our download page.
> > >> >
> > >> > How about this approach. We create something like ignite-version.cgi
> > >> script
> > >> > which will invoke a call to GA and then return the latest version.
> > >> >
> > >> > This should work, right?
> > >> >
> > >> > D.
> > >> >
> > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani <
> raul.asf@evosent.com
> > <javascript:;>>
> > >> > wrote:
> > >> >
> > >> > > Hey Dmitriy and all
> > >> > >
> > >> > > Also, since we have GA enabled for the website, we can track how
> > many
> > >> > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > >
> > >> > >
> > >> > > Unfortunately this won't work because GA is loaded via Javascript
> on
> > >> the
> > >> > > browser, so Google will never receive the page hit.
> > >> > >
> > >> > > Given the constraints, the most viable solution is an HTTPS
> endpoint
> > >> > > running on ASF infra that Ignite invokes via a GET or POST
> request.
> > >> The
> > >> > > simplest thing is to write each request in a log file, along with
> > the
> > >> > > timestamp, the version reported by the client, maybe the IP (not
> > sure
> > >> > about
> > >> > > the ASF rules about this concerning privacy, I guess it's OK if
> you
> > >> make
> > >> > it
> > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node
> > creates
> > >> on
> > >> > > first startup or something.
> > >> > >
> > >> > > This endpoint would need some basic DDoS protection and
> blacklisting
> > >> to
> > >> > > prevent data spoofing.
> > >> > >
> > >> > > If we'll be implementing this endpoint anyway, then there's no
> point
> > >> > > placing another file on Git or elsewhere for reporting the latest
> > >> > versions:
> > >> > > the endpoint itself can return them.
> > >> > >
> > >> > > WDYT?
> > >> > >
> > >> > > Cheers.
> > >> > >
> > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan <
> > >> > dsetrakyan@apache.org <javascript:;>>
> > >> > > wrote:
> > >> > >
> > >> > > > Cos, Raul,
> > >> > > >
> > >> > > > Thanks for the feedback. I completely agree about Maven Central
> > >> being a
> > >> > > 3rd
> > >> > > > party repo (did not think about that initially). All your
> > >> suggestions
> > >> > > make
> > >> > > > sense, but I would like to keep it as simple as possible, and so
> > far
> > >> > > > everything suggested required GIT dependencies and extra work.
> > >> > > >
> > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite
> > >> > website
> > >> > > > which will have only the latest version. Every time a node
> starts,
> > >> it
> > >> > > > receives the latest version from the page and suggests that
> users
> > >> > upgrade
> > >> > > > if needed.
> > >> > > >
> > >> > > > Also, since we have GA enabled for the website, we can track how
> > >> many
> > >> > > times
> > >> > > > this page was accessed, which will be equal to the number of
> > starts.
> > >> > This
> > >> > > > way, the counter information is tracked and monitored by the
> > Ignite
> > >> > PMC.
> > >> > > >
> > >> > > > This approach looks pretty innocent to me and everything is kept
> > and
> > >> > > > managed within Apache.
> > >> > > >
> > >> > > > Thoughts?
> > >> > > >
> > >> > > > D.
> > >> > > >
> > >> > > >
> > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik <
> > >> cos@apache.org <javascript:;>>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > I agree with Raul.
> > >> > > > > - providing a ping-back address to a 3rd party might be frown
> > >> upon by
> > >> > > > some.
> > >> > > > > And might have a consequences like stats collection about
> > users'
> > >> > > > > infrastructure.
> > >> > > > > - checking an ASF git-repo is easy and won't download any
> binary
> > >> > data:
> > >> > > > > everything is clear text and could be easily monitored by
> any
> > of
> > >> > the
> > >> > > > > network
> > >> > > > > diagnostic tools, shall it be required. But it involves a
> bit
> > of
> > >> > the
> > >> > > > > release
> > >> > > > > discipline.
> > >> > > > > - the binary data download in the runtime is my main concern.
> > >> This is
> > >> > > the
> > >> > > > > vector of MMA. What if someone gains the control over the
> > >> > repository
> > >> > > > and
> > >> > > > > replaces the file with some malicious content.
> > >> > > > >
> > >> > > > > As for the particular mechanism: IIRC Ignite used to make a
> call
> > >> to
> > >> > an
> > >> > > > > external script to check upon the atest software version
> > available
> > >> > for
> > >> > > > > download. In the past, the endpoint was running on a 3rd party
> > >> > server,
> > >> > > I
> > >> > > > > believe the best way would be to put this script on ASF infra
> > and
> > >> > have
> > >> > > > the
> > >> > > > > "update checker" running in a completely controlled
> environment.
> > >> > > > Actually,
> > >> > > > > I
> > >> > > > > recall we had this very discussion during the Incubation; I
> can
> > >> > > probably
> > >> > > > > dig
> > >> > > > > out the corresponding thread.
> > >> > > > >
> > >> > > > > Thoughts?
> > >> > > > > Cok
> > >> > > > >
> > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote:
> > >> > > > > > Hey guys
> > >> > > > > >
> > >> > > > > > In my opinion, maven.org is still owned by a third party
> > >> > (Sonatype).
> > >> > > > We
> > >> > > > > > don't know what kind of data analysis or intelligence
> > extraction
> > >> > they
> > >> > > > > run.
> > >> > > > > >
> > >> > > > > > If Ignite servers all over the world were hitting maven.org
> > >> > > > periodically
> > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a
> clear
> > >> > > > indication
> > >> > > > > > of who is running an Ignite server.
> > >> > > > > >
> > >> > > > > > They could reverse lookup the IP address and find out what
> > >> > > corporation
> > >> > > > it
> > >> > > > > > is.
> > >> > > > > >
> > >> > > > > > How about having Ignite check the ASF Git directly?
> > >> > > > > >
> > >> > > > > > We could use the Git ssh API, but that would require a new
> > >> > > dependency,
> > >> > > > > > which I advise against.
> > >> > > > > >
> > >> > > > > > Alternatively, we could have it scrape this HTML for new Git
> > >> tags:
> > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git
> > >> > > > > >
> > >> > > > > > Another option is to place a txt file in the root of the
> > master
> > >> > > branch
> > >> > > > > (e.g
> > >> > > > > > LATEST), containing a list of the latest GA versions for
> each
> > >> major
> > >> > > > > version
> > >> > > > > > line (1.x, 2.x).
> > >> > > > > >
> > >> > > > > > I would advocate this last option, but it requires somebody
> > >> > > remembering
> > >> > > > > to
> > >> > > > > > update the file with every release, unless we automate it
> > with a
> > >> > > Maven
> > >> > > > > > plugin.
> > >> > > > > >
> > >> > > > > > Hope that helps!
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <dmagda@apache.org
> > <javascript:;>> wrote:
> > >> > > > > >
> > >> > > > > > I see nothing wrong with this approach.
> > >> > > > > >
> > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is
> it
> > >> good
> > >> > > to
> > >> > > > > go?
> > >> > > > > >
> > >> > > > > > —
> > >> > > > > > Denis
> > >> > > > > >
> > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan <
> > >> > > > dsetrakyan@apache.org <javascript:;>
> > >> > > > > >
> > >> > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > Is everyone OK with this approach? Should I file a ticket
> on
> > >> it?
> > >> > > > > > >
> > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan <
> > >> > > > > dsetrakyan@apache.org <javascript:;>>
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > >> Igniters,
> > >> > > > > > >>
> > >> > > > > > >> There has been lots of talk of proposals about various
> > usage
> > >> > > metrics
> > >> > > > > for
> > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect
> > the
> > >> > topic
> > >> > > > and
> > >> > > > > > >> propose something very simple and non-intrusive.
> > >> > > > > > >>
> > >> > > > > > >> 1. Update Checker
> > >> > > > > > >> The main purpose of the update checker is not to collect
> > >> > metrics,
> > >> > > > but
> > >> > > > > to
> > >> > > > > > >> notify users about a new version of Ignite by accessing
> > >> > maven.org
> > >> > > > and
> > >> > > > > > >> getting the version out of the metadata file:
> > >> > > > > > >> http://repo2.maven.org/maven2/
> > org/apache/ignite/ignite-core/
> > >> > > > > > >> maven-metadata.xml
> > >> > > > > > >>
> > >> > > > > > >> This way we do not send any information anywhere and, at
> > the
> > >> > same
> > >> > > > > time,
> > >> > > > > > >> urge our users to download and start using the latest
> > >> version of
> > >> > > > > Ignite.
> > >> > > > > > >>
> > >> > > > > > >> 2. Startup Counter
> > >> > > > > > >> This piece is optional, but we can also get an insight in
> > how
> > >> > many
> > >> > > > > times
> > >> > > > > > a
> > >> > > > > > >> certain Ignite release gets started. This is just a cool
> > >> metric
> > >> > > for
> > >> > > > > the
> > >> > > > > > >> community to gauge the project popularity. You can think
> of
> > >> it
> > >> > as
> > >> > > > of a
> > >> > > > > > page
> > >> > > > > > >> visit counter shown on many websites. We can even decide
> to
> > >> > > display
> > >> > > > > this
> > >> > > > > > >> counter on the Ignite website as well.
> > >> > > > > > >>
> > >> > > > > > >> To do this, we can simply add a JAR in maven for every
> > >> release,
> > >> > > e.g.
> > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte.
> > >> Every
> > >> > > time
> > >> > > > > an
> > >> > > > > > >> Ignite node starts, it will download this JAR in the
> > >> background.
> > >> > > > Then
> > >> > > > > we
> > >> > > > > > >> will be able to view the number of the total downloads
> for
> > >> this
> > >> > > JAR
> > >> > > > in
> > >> > > > > > >> Maven Central, which is essentially the number of starts
> of
> > >> > Ignite
> > >> > > > > nodes.
> > >> > > > > > >>
> > >> > > > > > >> *Note that neither of the above suggestions require
> Ignite
> > to
> > >> > send
> > >> > > > or
> > >> > > > > > >> track any user information whatsoever.*
> > >> > > > > > >>
> > >> > > > > > >> Please reply suggesting weather you are OK with this
> > >> approach.
> > >> > > > > > >>
> > >> > > > > > >> D.
> > >> > > > > > >>
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> > >
> > >
> >
>