You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ignite.apache.org by Alex Plehanov <pl...@gmail.com> on 2020/06/03 11:45:40 UTC
Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Hello Igniters,
AI 2.8.1 is finally released and as we discussed here [1] its time to start
the discussion about 2.9 release.
I want to propose myself to be the release manager of the 2.9 release.
What about release time, I agree with Maxim that we should deliver features
as frequently as possible. If some feature doesn't fit into release dates
we should better include it into the next release and schedule the next
release earlier then postpone the current release.
I propose the following dates for 2.9 release:
Scope Freeze: June 26, 2020
Code Freeze: July 10, 2020
Voting Date: July 31, 2020
Release Date: August 7, 2019
WDYT?
[1] :
http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
> Is that OK?
We shouldn’t add any code that is broke some existing, widely used feature.
Let’s fix it before the merge.
> 10 июня 2020 г., в 13:54, Ivan Bessonov <be...@gmail.com> написал(а):
>
> Hello,
>
> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
> replied
> to the main conversation about "communication via discovery" [1]. We work
> on it
> together and recently have found one hard-to-fix scenario, detailed
> description is
> provided in Sergey's reply.
>
> In short, July 10th looks realistic only if we introduce new behavior in
> its current
> implementation, with new setting and IgniteExperimental status. Blocker
> here is
> current implementation of Continuos Query protocol that in some cases
> (described
> at the end) initiates TCP connection right from discovery thread which
> obviously
> leads to deadlock. We haven't estimated efforts needed to redesign of CQ
> protocol
> but it is definitely a risk and fixing it isn't feasible with a code freeze
> at 10th of July.
> So my verdict: we can include this new feature in 2.9 scope as experimental
> and with
> highlighted limitation on CQ usage. Is that OK?
>
> CQ limitation: server needs to open a communication connection to the
> client if during
> CQ registration client tries to p2p deploy new class not available on
> server classpath.
> In other cases registration of CQ should be fine.
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>
> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
>
>> Hi,
>>
>> Indeed, the tracing feature is almost ready. Discovery, communication and
>> transactions tracing will be introduced, as well as an option to configure
>> tracing in runtime. Right now we are working on final performance
>> optimizations, but it's very likely that we'll complete this activity
>> before the code freeze date.
>> Let's include tracing to the 2.9 release scope.
>>
>> More info:
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>> https://issues.apache.org/jira/browse/IGNITE-13060
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
>>
>>> Hi folks,
>>>
>>> The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd
>> like
>>> only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is about
>> to
>>> finish with the tracing support, and @Ivan Bessonov
>>> <ib...@gridgain.com>, who is fixing a serious limitation for K8
>>> deployments [1]. Most likely, both features will be ready by the code
>>> freeze date (July 10), but the guys should know it better.
>>>
>>> [1]
>>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
>>> wrote:
>>>
>>>> Hello Igniters,
>>>>
>>>> AI 2.8.1 is finally released and as we discussed here [1] its time to
>>>> start
>>>> the discussion about 2.9 release.
>>>>
>>>> I want to propose myself to be the release manager of the 2.9 release.
>>>>
>>>> What about release time, I agree with Maxim that we should deliver
>>>> features
>>>> as frequently as possible. If some feature doesn't fit into release
>> dates
>>>> we should better include it into the next release and schedule the next
>>>> release earlier then postpone the current release.
>>>>
>>>> I propose the following dates for 2.9 release:
>>>>
>>>> Scope Freeze: June 26, 2020
>>>> Code Freeze: July 10, 2020
>>>> Voting Date: July 31, 2020
>>>> Release Date: August 7, 2019
>>>>
>>>> WDYT?
>>>>
>>>> [1] :
>>>>
>>>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>>>
>>>
>>
>
>
> --
> Sincerely yours,
> Ivan Bessonov
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Vladislav Pyatkov <vl...@gmail.com>.
This patch [1] interesting only a deployment where clients often
reconnecting (stopping/starting) to cluster.
In these cases Coordinator node could fail by OOM, even the cluster did not
use MVCC caches.
On Mon, Oct 12, 2020 at 1:41 PM Zhenya Stanilovsky
<ar...@mail.ru.invalid> wrote:
>
>
> without [2] and [3] we obtain unexpected fields in index creation and as a
> consequence buggy sql execution and pla of course.
>
>
>
> >Guys,
> >
> >I've found 3 more tickets which were targeted to 2.9 but merged only to
> the
> >master branch ([1], [2], [3]).
> >Do we need these tickets in 2.9? Are they critical?
> >
> >[1]: https://issues.apache.org/jira/browse/IGNITE-12350
> >[2]: https://issues.apache.org/jira/browse/IGNITE-13376
> >[3]: https://issues.apache.org/jira/browse/IGNITE-13280
> >
> >вс, 11 окт. 2020 г. в 16:11, Nikolay Izhikov < nizhikov@apache.org >:
> >
> >> Igniters,
> >>
> >> IGNITE-13553 fixed and cherry-picked to 2.9.
> >> We can continue with the release.
> >>
> >> > 10 окт. 2020 г., в 00:00, Denis Magda < dmagda@apache.org >
> написал(а):
> >> >
> >> > Nikolay,
> >> >
> >> > re: the minor improvements. I'm not against of including those if we
> can
> >> > prepare the docs before the vote starts. Presently, the docs are
> "frozen"
> >> > for the 2.9 release, but I can scratch some time and take part in the
> >> docs
> >> > review the next week.
> >> >
> >> > -
> >> > Denis
> >> >
> >> >
> >> > On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov < nizhikov@apache.org
> >
> >> wrote:
> >> >
> >> >> Hello, Igniters.
> >> >>
> >> >> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
> >> >> stops in case wrong datatype put in indexed field»
> >> >> Can someone, please, help me with the review?
> >> >>
> >> >> [1] https://github.com/apache/ignite/pull/8330
> >> >> [2] https://issues.apache.org/jira/browse/IGNITE-13553
> >> >>
> >> >> ==================
> >> >>
> >> >> I propose to include several minor tickets to the scope of the 2.9
> that
> >> >> increase Ignite User Experience
> >> >>
> >> >> CMD tools improvements:
> >> >>
> >> >> IGNITE-13488 - Command to print metric value
> >> >> IGNITE-13426 - Command to print system view content
> >> >> IGNITE-13422 - Parameter to explicitly enable experimental commands
> >> >>
> >> >> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
> >> >>
> >> >> New system views:
> >> >>
> >> >> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> >> >> IGNITE-13408 BinaryMetadata view.
> >> >>
> >> >>> 9 окт. 2020 г., в 04:04, Denis Magda < dmagda@apache.org >
> написал(а):
> >> >>>
> >> >>> @Alex Plehanov < plehanov.alex@gmail.com >,
> >> >>>
> >> >>> If it still makes sense and not too late, you can cherry-pick the
> >> commit
> >> >>> with the new docs into the 2.9 branch:
> >> >>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
> >> >>>
> >> >>> Anyway, it's not crucial as long as we agreed to make an exception
> for
> >> >> this
> >> >>> release. The docs were already published to the website and
> assembled
> >> >> from
> >> >>> the master. Once the vote passes, I'll make them visible and
> traceable
> >> >> from
> >> >>> the website menus.
> >> >>>
> >> >>> -
> >> >>> Denis
> >> >>>
> >> >>>
> >> >>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> >> >> alexey.goncharuk@gmail.com >
> >> >>> wrote:
> >> >>>
> >> >>>> Saikat,
> >> >>>>
> >> >>>> The plan sounds good to me! Do you have an idea for the timeline of
> >> the
> >> >>>> module releases? Let me know if I can help in any way.
> >> >>>>
> >> >>>> Also, I was looking into the modularization IEP and noticed that
> OSGI
> >> >>>> module is discussed to be moved to the extensions, but there is no
> >> >>>> corresponding ticket. Should I create one? I will be happy to help
> >> with
> >> >>>> moving this module to extensions.
> >> >>>>
> >> >>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <
> saikat.maitra@gmail.com
> >> >:
> >> >>>>
> >> >>>>> Hi Alexey,
> >> >>>>>
> >> >>>>> All the modules as planned in IEP-36
> >> >>>>>
> >> >>>>
> >> >>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> >> >>>>> are migrated to ignite-extensions repository.
> >> >>>>>
> >> >>>>> We can definitely plan to release ignite-extensions modules.
> >> >>>>>
> >> >>>>> I have a pending PR related to the kafka module and the PR is in
> >> review
> >> >>>>> https://github.com/apache/ignite/pull/8222 but its
> corresponding PR
> >> >> has
> >> >>>>> been merged in ignite-extensions repository.
> >> >>>>>
> >> >>>>> My thoughts / action items for the migration efforts and releases
> >> were
> >> >>>>> as follows:
> >> >>>>>
> >> >>>>> 1. Merge the changes for
> https://github.com/apache/ignite/pull/8222
> >> >>>>> 2. Fix the upload nightly packages task for ignite-core to help
> fix
> >> the
> >> >>>>> travis build failure in ignite-extensions repository.
> >> >>>>> 3. Review and update the README.md files for the ignite-extensions
> >> >>>> modules
> >> >>>>> as required.
> >> >>>>> 4. Update the docs for ignite-extensions modules in
> ignite-website.
> >> >>>>> 5. Release each module separately and share updates.
> >> >>>>>
> >> >>>>> Please let me know if you have feedback.
> >> >>>>>
> >> >>>>> Regards,
> >> >>>>> Saikat
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov < mr.weider@gmail.com
> >
> >> >> wrote:
> >> >>>>>
> >> >>>>>> It seems that gitbox.apache.org is not available on CI/CD.
> >> >>>>>>
> >> >>>>>> I will try to update VCS to GitHub.
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <
> plehanov.alex@gmail.com >
> >> >>>>>> wrote:
> >> >>>>>>>
> >> >>>>>>> Guys,
> >> >>>>>>>
> >> >>>>>>> I've tried to build release candidate using TC [1]. But:
> >> >>>>>>> 1. I can't choose a branch in this TC task (there is no such
> >> field).
> >> >>>>>>> 2. On the "default" branch I got an error: Failed to collect
> >> changes,
> >> >>>>>>> error: List remote refs failed:
> >> >>>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
> >> >>>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> >> >>>> connect
> >> >>>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300,
> parent
> >> >>>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
> >> >>>>>>>
> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master "}
> >> >>>>>>>
> >> >>>>>>> Is there some problem with this TC task? What am I doing wrong?
> >> >>>>>>>
> >> >>>>>>> [1] :
> >> >>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >> >>>>>>>
> >> >>>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> >> >>>>>> alexey.goncharuk@gmail.com >:
> >> >>>>>>>
> >> >>>>>>>> Saikat, Nikolay,
> >> >>>>>>>>
> >> >>>>>>>> We have migrated a bunch of modules to ignite-extensions
> recently.
> >> >>>>>> Given
> >> >>>>>>>> that these modules will not be available in Ignite 2.9 anymore
> >> (will
> >> >>>>>>>> they?), should we also plan to release the extensions?
> >> >>>>>>>>
> >> >>>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
> >> >> plehanov.alex@gmail.com
> >> >>>>> :
> >> >>>>>>>>
> >> >>>>>>>>> Igniters,
> >> >>>>>>>>>
> >> >>>>>>>>> I've prepared release notes for the 2.9 release [1]. Can
> anyone
> >> >>>> review
> >> >>>>>>>> it,
> >> >>>>>>>>> please?
> >> >>>>>>>>>
> >> >>>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
> >> >>>>>>>>>
> >> >>>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> >> >>>> plehanov.alex@gmail.com
> >> >>>>>>> :
> >> >>>>>>>>>
> >> >>>>>>>>>> Guys,
> >> >>>>>>>>>>
> >> >>>>>>>>>> I've filled the ticket with reproducer [1] for the discovery
> >> bug.
> >> >>>>>> This
> >> >>>>>>>>> bug
> >> >>>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin
> >> privately
> >> >>>>>> and
> >> >>>>>>>>>> decided to revert this ticket. I will do it today (after TC
> bot
> >> >>>> visa)
> >> >>>>>>>> if
> >> >>>>>>>>>> there are no objections.
> >> >>>>>>>>>>
> >> >>>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >> >>>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >> >>>>>>>>>>
> >> >>>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> >> >>>> plehanov.alex@gmail.com
> >> >>>>>>> :
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Guys,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> During internal testing, we've found a critical bug with
> >> >>>>>>>>>>> discovery (cluster falls apart if two nodes segmented
> >> >>>> sequentially).
> >> >>>>>>>>> This
> >> >>>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >> >>>>>>>>>>> before release. Under investigation now. I'll let you know
> when
> >> >> we
> >> >>>>>> get
> >> >>>>>>>>>>> something.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <
> agura@apache.org >:
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> >> version
> >> >>>> of
> >> >>>>>>>>> the
> >> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> >> generation
> >> >>>> in
> >> >>>>>>>>> later
> >> >>>>>>>>>>>> release, or keep the code clean and fix the regression in
> the
> >> >>>> next
> >> >>>>>>>>> releases?
> >> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> >> fairly
> >> >>>>>>>>>>>> straightforward and does not change the semantics, just
> skips
> >> >> the
> >> >>>>>>>>> factory
> >> >>>>>>>>>>>> closures for certain messages.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual
> >> for
> >> >>>> all
> >> >>>>>>>>>>>> workloads (I didn't get any significant drops during
> >> >>>> benchmarking).
> >> >>>>>>>> So
> >> >>>>>>>>>>>> I prefer the runtime generation in later releases.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >> >>>>>>>>>>>> < alexey.goncharuk@gmail.com > wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Alexey, Andrey, Igniters,
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> >> version
> >> >>>> of
> >> >>>>>>>>> the
> >> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> >> generation
> >> >>>> in
> >> >>>>>>>>> later
> >> >>>>>>>>>>>> release, or keep the code clean and fix the regression in
> the
> >> >>>> next
> >> >>>>>>>>> releases?
> >> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> >> fairly
> >> >>>>>>>>>>>> straightforward and does not change the semantics, just
> skips
> >> >> the
> >> >>>>>>>>> factory
> >> >>>>>>>>>>>> closures for certain messages.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Personally, I would prefer fixing the regression given
> that
> >> we
> >> >>>>>> also
> >> >>>>>>>>>>>> introduced tracing in this release.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >> >>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>> :
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> >> >>>> ed52559eb95
> >> >>>>>>>>> and
> >> >>>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
> >> >>>> average
> >> >>>>>> on
> >> >>>>>>>>> our
> >> >>>>>>>>>>>> environment (about the same results we got benchmarking
> >> 65c30ec6
> >> >>>> vs
> >> >>>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
> >> >>>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9
> on
> >> >> this
> >> >>>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> >> >>>> servers, 5
> >> >>>>>>>>> clients
> >> >>>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
> >> >>>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464
> ns
> >> >>>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908
> ns
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >> >>>>>>>>>>>> a.budnikov.ignite@gmail.com >:
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Hi Everyone,
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
> >> >>>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite
> 2.9,
> >> >> you
> >> >>>>>> can
> >> >>>>>>>>>>>> update the docs by following the instruction.
> Unfortunately, I
> >> >>>>>> won't
> >> >>>>>>>> be
> >> >>>>>>>>>>>> able to spend any time on this project any longer. You can
> >> send
> >> >>>>>> your
> >> >>>>>>>>> pull
> >> >>>>>>>>>>>> requests and questions about the documentation to Denis
> Magda.
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> -Artem
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> [1] :
> >> >>>>>>>>>>>>
> >> >>>>
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com > wrote:
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> I've tried to play with message factories locally, but
> >> >>>>>>>>>>>> unfortunately, I
> >> >>>>>>>>>>>>>>>> cannot spot the difference between old and new
> >> >> implementation
> >> >>>>>> in
> >> >>>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >> >>>>>>>>>>>> MessageFactoryImpl
> >> >>>>>>>>>>>>>>>> with the old switch statement to the
> >> ignite-2.9-revert-12568
> >> >>>>>>>>> branch
> >> >>>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >> >>>>>>>> compatible
> >> >>>>>>>>>>>> with the
> >> >>>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Can you check if this change makes any difference
> >> >>>>>>>> performance-wise
> >> >>>>>>>>>>>> in your
> >> >>>>>>>>>>>>>>>> environment? If yes, we can go with runtime code
> >> generation
> >> >>>> in
> >> >>>>>>>> the
> >> >>>>>>>>>>>> long
> >> >>>>>>>>>>>>>>>> term: register classes and generate a dynamic message
> >> >> factory
> >> >>>>>>>> with
> >> >>>>>>>>>>>> a switch
> >> >>>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
> >> >>>> though,
> >> >>>>>>>>>>>> obviously).
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >> >>>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> Hello guys,
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket
> >> [1]),
> >> >>>> it
> >> >>>>>>>>>>>> reduced the
> >> >>>>>>>>>>>>>>>>> drop, but not completely removed it.
> >> >>>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> >> >>>> patch?
> >> >>>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> >> >>>> against
> >> >>>>>>>>>>>> 2.8.1
> >> >>>>>>>>>>>>>>>>> release on your environment?
> >> >>>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3%
> drop
> >> >>>> left,
> >> >>>>>>>>>>>> it's close
> >> >>>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
> >> >>>>>>>>>>>> showstopper. Guys,
> >> >>>>>>>>>>>>>>>>> WDYT?
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC
> thin
> >> >>>>>>>> driver
> >> >>>>>>>>>>>> between 2.8
> >> >>>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker
> and
> >> >>>>>>>> should
> >> >>>>>>>>>>>> be
> >> >>>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >> >>>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4]
> >> (NIO
> >> >>>>>>>>> message
> >> >>>>>>>>>>>>>>>>> send problem in some circumstances). I will
> cherry-pick
> >> it
> >> >>>> if
> >> >>>>>>>>>>>> there is no
> >> >>>>>>>>>>>>>>>>> objection.
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> [1]
> https://issues.apache.org/jira/browse/IGNITE-13411
> >> >>>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >> >>>>>>>>>>>>>>>>> [3]
> https://issues.apache.org/jira/browse/IGNITE-13414
> >> >>>>>>>>>>>>>>>>> [4]
> https://issues.apache.org/jira/browse/IGNITE-13361
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >> >>>>>>>> mmuzaf@apache.org
> >> >>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release.
> Since
> >> >>>>>>>> this
> >> >>>>>>>>>>>> issue is
> >> >>>>>>>>>>>>>>>>>> related to the new master key change functionality
> which
> >> >>>>>>>>>>>> haven't been
> >> >>>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick
> >> commit
> >> >>>>>>>> to
> >> >>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>> release branch.
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> [1]
> https://issues.apache.org/jira/browse/IGNITE-13390
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >> >>>>>>>>>>>> nizhikov@apache.org >
> >> >>>>>>>>>>>>>>>>> wrote:
> >> >>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>> Hello, Igniters.
> >> >>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client
> fix
> >> >>>>>>>> [1]
> >> >>>>>>>>>>>> into the
> >> >>>>>>>>>>>>>>>>> 2.9
> >> >>>>>>>>>>>>>>>>>> release
> >> >>>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
> >> >> fields
> >> >>>>>>>>> in
> >> >>>>>>>>>>>> wrong
> >> >>>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
> >> >>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>> [1]
> https://issues.apache.org/jira/browse/IGNITE-12809
> >> >>>>>>>>>>>>>>>>>>> [2]
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >> >>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >> >>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com >
> >> >>>>>>>>>>>>>>>>>> написал(а):
> >> >>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can
> optimize
> >> >>>>>>>>>>>> anything out of
> >> >>>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no
> >> ideas
> >> >>>>>>>>>>>> right now),
> >> >>>>>>>>>>>>>>>>> so
> >> >>>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to
> >> the
> >> >>>>>>>>>>>> switch
> >> >>>>>>>>>>>>>>>>> approach
> >> >>>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably,
> >> inlining
> >> >>>>>>>>> the
> >> >>>>>>>>>>>> Ignite
> >> >>>>>>>>>>>>>>>>>> messages
> >> >>>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the
> trick.
> >> Let
> >> >>>>>>>>> me
> >> >>>>>>>>>>>> explore
> >> >>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>>>> code a bit.
> >> >>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes
> for
> >> >>>>>>>> the
> >> >>>>>>>>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
> >> single
> >> >>>>>>>>>>>> virtual call
> >> >>>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
> >> >>>>>>>> amount
> >> >>>>>>>>>>>> of other
> >> >>>>>>>>>>>>>>>>>> work
> >> >>>>>>>>>>>>>>>>>>>> happening during the message processing.
> >> >>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >> >>>>>>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >> >>>>>>>>> results.
> >> >>>>>>>>>>>>>>>>> Actually,
> >> >>>>>>>>>>>>>>>>>> we
> >> >>>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >> >>>>>>>>> between
> >> >>>>>>>>>>>> e6a7f93
> >> >>>>>>>>>>>>>>>>>> (first
> >> >>>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >> >>>>>>>>>>>> 6592dfa5 (last
> >> >>>>>>>>>>>>>>>>>> commit in
> >> >>>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two
> problematic
> >> >>>>>>>>>>>> commits.
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible
> >> for
> >> >>>>>>>> a
> >> >>>>>>>>>>>> drop
> >> >>>>>>>>>>>>>>>>> between
> >> >>>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9
> with
> >> >>>>>>>>>>>> reverted
> >> >>>>>>>>>>>>>>>>>> IGNITE-13060
> >> >>>>>>>>>>>>>>>>>>>>> now and performance looks the same)
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring)
> is
> >> not
> >> >>>>>>>>>>>> related to
> >> >>>>>>>>>>>>>>>>>> drop
> >> >>>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some
> performance
> >> >>>>>>>>>>>> problem, and
> >> >>>>>>>>>>>>>>>>> we
> >> >>>>>>>>>>>>>>>>>> can
> >> >>>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or
> we
> >> >>>>>>>> leave
> >> >>>>>>>>>>>> it as is?
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568
> (MessageFactory
> >> >>>>>>>>>>>> refactoring)?
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568
> has
> >> an
> >> >>>>>>>>>>>> incorrect fix
> >> >>>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
> >> branch
> >> >>>>>>>>>>>> [1], so it
> >> >>>>>>>>>>>>>>>>>> cannot be
> >> >>>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more
> accurate
> >> >>>>>>>> work
> >> >>>>>>>>>>>> with fix
> >> >>>>>>>>>>>>>>>>>> versions
> >> >>>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >> >>>>>>>>>>>> versions.
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> --AG
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> [1]
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>
> >> >>>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >> >>>>>>>>>>>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>> Guys,
> >> >>>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060
> and
> >> >>>>>>>>>>>> IGNITE-12568
> >> >>>>>>>>>>>>>>>>>> (reverted
> >> >>>>>>>>>>>>>>>>>>>>>>>> it
> >> >>>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on
> 2.8.1
> >> >>>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to
> >> hot
> >> >>>>>>>>>>>> paths, to
> >> >>>>>>>>>>>>>>>>> trace
> >> >>>>>>>>>>>>>>>>>>>>>>>> these
> >> >>>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance
> drop
> >> >>>>>>>>> here.
> >> >>>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >> >>>>>>>>> switch/case
> >> >>>>>>>>>>>> block was
> >> >>>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers.
> The
> >> >>>>>>>>>>>> message factory
> >> >>>>>>>>>>>>>>>>>> is on
> >> >>>>>>>>>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >> >>>>>>>> impact
> >> >>>>>>>>>>>> on total
> >> >>>>>>>>>>>>>>>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >> >>>>>>>>>>>> microbenchmarks,
> >> >>>>>>>>>>>>>>>>>> and
> >> >>>>>>>>>>>>>>>>>>>>>>>> found
> >> >>>>>>>>>>>>>>>>>>>>>>>> that old implementation of
> MessageFactory.create()
> >> >>>>>>>>>>>> about 30-35%
> >> >>>>>>>>>>>>>>>>>> faster
> >> >>>>>>>>>>>>>>>>>>>>>>>> than
> >> >>>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with
> >> switch/case
> >> >>>>>>>>> can
> >> >>>>>>>>>>>>>>>>> effectively
> >> >>>>>>>>>>>>>>>>>>>>>>>> inline
> >> >>>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >> >>>>>>>> suppliers
> >> >>>>>>>>>>>> relatively
> >> >>>>>>>>>>>>>>>>>> heavy
> >> >>>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've
> tried to
> >> >>>>>>>>>>>> rewrite the
> >> >>>>>>>>>>>>>>>>> code
> >> >>>>>>>>>>>>>>>>>>>>>>>> using
> >> >>>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >> >>>>>>>>> interface
> >> >>>>>>>>>>>> (to
> >> >>>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
> >> "invokevirtual"),
> >> >>>>>>>>>>>> but it gives
> >> >>>>>>>>>>>>>>>>>> back
> >> >>>>>>>>>>>>>>>>>>>>>>>> only
> >> >>>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case,
> code
> >> >>>>>>>> looks
> >> >>>>>>>>>>>> ugly
> >> >>>>>>>>>>>>>>>>>> (lambdas
> >> >>>>>>>>>>>>>>>>>>>>>>>> can't
> >> >>>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more
> ways to
> >> >>>>>>>>>>>> optimize the
> >> >>>>>>>>>>>>>>>>>> current
> >> >>>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case
> block).
> >> >>>>>>>>>>>> Andrey Gura,
> >> >>>>>>>>>>>>>>>>> as
> >> >>>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some
> ideas
> >> >>>>>>>>> about
> >> >>>>>>>>>>>>>>>>>> optimization?
> >> >>>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but
> there
> >> are
> >> >>>>>>>>>>>> some metrics
> >> >>>>>>>>>>>>>>>>>>>>>>>> already
> >> >>>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old
> >> message
> >> >>>>>>>>>>>> factory
> >> >>>>>>>>>>>>>>>>>>>>>>>> implementation
> >> >>>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >> >>>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements)
> is
> >> >>>>>>>>>>>> already released
> >> >>>>>>>>>>>>>>>>>> in
> >> >>>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message
> factory)
> >> is
> >> >>>>>>>>>>>> only present
> >> >>>>>>>>>>>>>>>>>> in Ignite
> >> >>>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and
> whichever
> >> new
> >> >>>>>>>>>>>> metrics
> >> >>>>>>>>>>>>>>>>>> created for
> >> >>>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
> >> unblock
> >> >>>>>>>>>>>> the release
> >> >>>>>>>>>>>>>>>>>> and deal
> >> >>>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >> >>>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>
> >> >>
> >> >>
> >>
> >>
>
--
Vladislav Pyatkov
Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Zhenya Stanilovsky <ar...@mail.ru.INVALID>.
without [2] and [3] we obtain unexpected fields in index creation and as a consequence buggy sql execution and pla of course.
>Guys,
>
>I've found 3 more tickets which were targeted to 2.9 but merged only to the
>master branch ([1], [2], [3]).
>Do we need these tickets in 2.9? Are they critical?
>
>[1]: https://issues.apache.org/jira/browse/IGNITE-12350
>[2]: https://issues.apache.org/jira/browse/IGNITE-13376
>[3]: https://issues.apache.org/jira/browse/IGNITE-13280
>
>вс, 11 окт. 2020 г. в 16:11, Nikolay Izhikov < nizhikov@apache.org >:
>
>> Igniters,
>>
>> IGNITE-13553 fixed and cherry-picked to 2.9.
>> We can continue with the release.
>>
>> > 10 окт. 2020 г., в 00:00, Denis Magda < dmagda@apache.org > написал(а):
>> >
>> > Nikolay,
>> >
>> > re: the minor improvements. I'm not against of including those if we can
>> > prepare the docs before the vote starts. Presently, the docs are "frozen"
>> > for the 2.9 release, but I can scratch some time and take part in the
>> docs
>> > review the next week.
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov < nizhikov@apache.org >
>> wrote:
>> >
>> >> Hello, Igniters.
>> >>
>> >> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
>> >> stops in case wrong datatype put in indexed field»
>> >> Can someone, please, help me with the review?
>> >>
>> >> [1] https://github.com/apache/ignite/pull/8330
>> >> [2] https://issues.apache.org/jira/browse/IGNITE-13553
>> >>
>> >> ==================
>> >>
>> >> I propose to include several minor tickets to the scope of the 2.9 that
>> >> increase Ignite User Experience
>> >>
>> >> CMD tools improvements:
>> >>
>> >> IGNITE-13488 - Command to print metric value
>> >> IGNITE-13426 - Command to print system view content
>> >> IGNITE-13422 - Parameter to explicitly enable experimental commands
>> >>
>> >> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
>> >>
>> >> New system views:
>> >>
>> >> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
>> >> IGNITE-13408 BinaryMetadata view.
>> >>
>> >>> 9 окт. 2020 г., в 04:04, Denis Magda < dmagda@apache.org > написал(а):
>> >>>
>> >>> @Alex Plehanov < plehanov.alex@gmail.com >,
>> >>>
>> >>> If it still makes sense and not too late, you can cherry-pick the
>> commit
>> >>> with the new docs into the 2.9 branch:
>> >>>
>> >>
>> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
>> >>>
>> >>> Anyway, it's not crucial as long as we agreed to make an exception for
>> >> this
>> >>> release. The docs were already published to the website and assembled
>> >> from
>> >>> the master. Once the vote passes, I'll make them visible and traceable
>> >> from
>> >>> the website menus.
>> >>>
>> >>> -
>> >>> Denis
>> >>>
>> >>>
>> >>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
>> >> alexey.goncharuk@gmail.com >
>> >>> wrote:
>> >>>
>> >>>> Saikat,
>> >>>>
>> >>>> The plan sounds good to me! Do you have an idea for the timeline of
>> the
>> >>>> module releases? Let me know if I can help in any way.
>> >>>>
>> >>>> Also, I was looking into the modularization IEP and noticed that OSGI
>> >>>> module is discussed to be moved to the extensions, but there is no
>> >>>> corresponding ticket. Should I create one? I will be happy to help
>> with
>> >>>> moving this module to extensions.
>> >>>>
>> >>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra < saikat.maitra@gmail.com
>> >:
>> >>>>
>> >>>>> Hi Alexey,
>> >>>>>
>> >>>>> All the modules as planned in IEP-36
>> >>>>>
>> >>>>
>> >>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>> >>>>> are migrated to ignite-extensions repository.
>> >>>>>
>> >>>>> We can definitely plan to release ignite-extensions modules.
>> >>>>>
>> >>>>> I have a pending PR related to the kafka module and the PR is in
>> review
>> >>>>> https://github.com/apache/ignite/pull/8222 but its corresponding PR
>> >> has
>> >>>>> been merged in ignite-extensions repository.
>> >>>>>
>> >>>>> My thoughts / action items for the migration efforts and releases
>> were
>> >>>>> as follows:
>> >>>>>
>> >>>>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>> >>>>> 2. Fix the upload nightly packages task for ignite-core to help fix
>> the
>> >>>>> travis build failure in ignite-extensions repository.
>> >>>>> 3. Review and update the README.md files for the ignite-extensions
>> >>>> modules
>> >>>>> as required.
>> >>>>> 4. Update the docs for ignite-extensions modules in ignite-website.
>> >>>>> 5. Release each module separately and share updates.
>> >>>>>
>> >>>>> Please let me know if you have feedback.
>> >>>>>
>> >>>>> Regards,
>> >>>>> Saikat
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov < mr.weider@gmail.com >
>> >> wrote:
>> >>>>>
>> >>>>>> It seems that gitbox.apache.org is not available on CI/CD.
>> >>>>>>
>> >>>>>> I will try to update VCS to GitHub.
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov < plehanov.alex@gmail.com >
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>> Guys,
>> >>>>>>>
>> >>>>>>> I've tried to build release candidate using TC [1]. But:
>> >>>>>>> 1. I can't choose a branch in this TC task (there is no such
>> field).
>> >>>>>>> 2. On the "default" branch I got an error: Failed to collect
>> changes,
>> >>>>>>> error: List remote refs failed:
>> >>>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
>> >>>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>> >>>> connect
>> >>>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>> >>>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
>> >>>>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master "}
>> >>>>>>>
>> >>>>>>> Is there some problem with this TC task? What am I doing wrong?
>> >>>>>>>
>> >>>>>>> [1] :
>> >>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>> >>>>>>>
>> >>>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>> >>>>>> alexey.goncharuk@gmail.com >:
>> >>>>>>>
>> >>>>>>>> Saikat, Nikolay,
>> >>>>>>>>
>> >>>>>>>> We have migrated a bunch of modules to ignite-extensions recently.
>> >>>>>> Given
>> >>>>>>>> that these modules will not be available in Ignite 2.9 anymore
>> (will
>> >>>>>>>> they?), should we also plan to release the extensions?
>> >>>>>>>>
>> >>>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>> >> plehanov.alex@gmail.com
>> >>>>> :
>> >>>>>>>>
>> >>>>>>>>> Igniters,
>> >>>>>>>>>
>> >>>>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
>> >>>> review
>> >>>>>>>> it,
>> >>>>>>>>> please?
>> >>>>>>>>>
>> >>>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
>> >>>>>>>>>
>> >>>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>> >>>> plehanov.alex@gmail.com
>> >>>>>>> :
>> >>>>>>>>>
>> >>>>>>>>>> Guys,
>> >>>>>>>>>>
>> >>>>>>>>>> I've filled the ticket with reproducer [1] for the discovery
>> bug.
>> >>>>>> This
>> >>>>>>>>> bug
>> >>>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin
>> privately
>> >>>>>> and
>> >>>>>>>>>> decided to revert this ticket. I will do it today (after TC bot
>> >>>> visa)
>> >>>>>>>> if
>> >>>>>>>>>> there are no objections.
>> >>>>>>>>>>
>> >>>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>> >>>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>> >>>>>>>>>>
>> >>>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>> >>>> plehanov.alex@gmail.com
>> >>>>>>> :
>> >>>>>>>>>>
>> >>>>>>>>>>> Guys,
>> >>>>>>>>>>>
>> >>>>>>>>>>> During internal testing, we've found a critical bug with
>> >>>>>>>>>>> discovery (cluster falls apart if two nodes segmented
>> >>>> sequentially).
>> >>>>>>>>> This
>> >>>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
>> >>>>>>>>>>> before release. Under investigation now. I'll let you know when
>> >> we
>> >>>>>> get
>> >>>>>>>>>>> something.
>> >>>>>>>>>>>
>> >>>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura < agura@apache.org >:
>> >>>>>>>>>>>
>> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version
>> >>>> of
>> >>>>>>>>> the
>> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
>> generation
>> >>>> in
>> >>>>>>>>> later
>> >>>>>>>>>>>> release, or keep the code clean and fix the regression in the
>> >>>> next
>> >>>>>>>>> releases?
>> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
>> fairly
>> >>>>>>>>>>>> straightforward and does not change the semantics, just skips
>> >> the
>> >>>>>>>>> factory
>> >>>>>>>>>>>> closures for certain messages.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual
>> for
>> >>>> all
>> >>>>>>>>>>>> workloads (I didn't get any significant drops during
>> >>>> benchmarking).
>> >>>>>>>> So
>> >>>>>>>>>>>> I prefer the runtime generation in later releases.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>> >>>>>>>>>>>> < alexey.goncharuk@gmail.com > wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Alexey, Andrey, Igniters,
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version
>> >>>> of
>> >>>>>>>>> the
>> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
>> generation
>> >>>> in
>> >>>>>>>>> later
>> >>>>>>>>>>>> release, or keep the code clean and fix the regression in the
>> >>>> next
>> >>>>>>>>> releases?
>> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
>> fairly
>> >>>>>>>>>>>> straightforward and does not change the semantics, just skips
>> >> the
>> >>>>>>>>> factory
>> >>>>>>>>>>>> closures for certain messages.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Personally, I would prefer fixing the regression given that
>> we
>> >>>>>> also
>> >>>>>>>>>>>> introduced tracing in this release.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> >>>>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>> :
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>> >>>> ed52559eb95
>> >>>>>>>>> and
>> >>>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
>> >>>> average
>> >>>>>> on
>> >>>>>>>>> our
>> >>>>>>>>>>>> environment (about the same results we got benchmarking
>> 65c30ec6
>> >>>> vs
>> >>>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
>> >>>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>> >> this
>> >>>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>> >>>> servers, 5
>> >>>>>>>>> clients
>> >>>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
>> >>>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> >>>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>> >>>>>>>>>>>> a.budnikov.ignite@gmail.com >:
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Hi Everyone,
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
>> >>>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>> >> you
>> >>>>>> can
>> >>>>>>>>>>>> update the docs by following the instruction. Unfortunately, I
>> >>>>>> won't
>> >>>>>>>> be
>> >>>>>>>>>>>> able to spend any time on this project any longer. You can
>> send
>> >>>>>> your
>> >>>>>>>>> pull
>> >>>>>>>>>>>> requests and questions about the documentation to Denis Magda.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> -Artem
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> [1] :
>> >>>>>>>>>>>>
>> >>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> >>>>>>>>>>>> alexey.goncharuk@gmail.com > wrote:
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> I've tried to play with message factories locally, but
>> >>>>>>>>>>>> unfortunately, I
>> >>>>>>>>>>>>>>>> cannot spot the difference between old and new
>> >> implementation
>> >>>>>> in
>> >>>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
>> >>>>>>>>>>>> MessageFactoryImpl
>> >>>>>>>>>>>>>>>> with the old switch statement to the
>> ignite-2.9-revert-12568
>> >>>>>>>>> branch
>> >>>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> >>>>>>>> compatible
>> >>>>>>>>>>>> with the
>> >>>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Can you check if this change makes any difference
>> >>>>>>>> performance-wise
>> >>>>>>>>>>>> in your
>> >>>>>>>>>>>>>>>> environment? If yes, we can go with runtime code
>> generation
>> >>>> in
>> >>>>>>>> the
>> >>>>>>>>>>>> long
>> >>>>>>>>>>>>>>>> term: register classes and generate a dynamic message
>> >> factory
>> >>>>>>>> with
>> >>>>>>>>>>>> a switch
>> >>>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
>> >>>> though,
>> >>>>>>>>>>>> obviously).
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>> >>>>>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Hello guys,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket
>> [1]),
>> >>>> it
>> >>>>>>>>>>>> reduced the
>> >>>>>>>>>>>>>>>>> drop, but not completely removed it.
>> >>>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>> >>>> patch?
>> >>>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>> >>>> against
>> >>>>>>>>>>>> 2.8.1
>> >>>>>>>>>>>>>>>>> release on your environment?
>> >>>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>> >>>> left,
>> >>>>>>>>>>>> it's close
>> >>>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
>> >>>>>>>>>>>> showstopper. Guys,
>> >>>>>>>>>>>>>>>>> WDYT?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> >>>>>>>> driver
>> >>>>>>>>>>>> between 2.8
>> >>>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> >>>>>>>> should
>> >>>>>>>>>>>> be
>> >>>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>> >>>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4]
>> (NIO
>> >>>>>>>>> message
>> >>>>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick
>> it
>> >>>> if
>> >>>>>>>>>>>> there is no
>> >>>>>>>>>>>>>>>>> objection.
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> >>>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>> >>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> >>>>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> >>>>>>>> mmuzaf@apache.org
>> >>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> >>>>>>>> this
>> >>>>>>>>>>>> issue is
>> >>>>>>>>>>>>>>>>>> related to the new master key change functionality which
>> >>>>>>>>>>>> haven't been
>> >>>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick
>> commit
>> >>>>>>>> to
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> release branch.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>> >>>>>>>>>>>> nizhikov@apache.org >
>> >>>>>>>>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Hello, Igniters.
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> >>>>>>>> [1]
>> >>>>>>>>>>>> into the
>> >>>>>>>>>>>>>>>>> 2.9
>> >>>>>>>>>>>>>>>>>> release
>> >>>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>> >> fields
>> >>>>>>>>> in
>> >>>>>>>>>>>> wrong
>> >>>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> >>>>>>>>>>>>>>>>>>> [2]
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> >>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com >
>> >>>>>>>>>>>>>>>>>> написал(а):
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>> >>>>>>>>>>>> anything out of
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no
>> ideas
>> >>>>>>>>>>>> right now),
>> >>>>>>>>>>>>>>>>> so
>> >>>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to
>> the
>> >>>>>>>>>>>> switch
>> >>>>>>>>>>>>>>>>> approach
>> >>>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably,
>> inlining
>> >>>>>>>>> the
>> >>>>>>>>>>>> Ignite
>> >>>>>>>>>>>>>>>>>> messages
>> >>>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick.
>> Let
>> >>>>>>>>> me
>> >>>>>>>>>>>> explore
>> >>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>> code a bit.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> >>>>>>>> the
>> >>>>>>>>>>>>>>>>> performance.
>> >>>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
>> single
>> >>>>>>>>>>>> virtual call
>> >>>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
>> >>>>>>>> amount
>> >>>>>>>>>>>> of other
>> >>>>>>>>>>>>>>>>>> work
>> >>>>>>>>>>>>>>>>>>>> happening during the message processing.
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> >>>>>>>>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>> >>>>>>>>> results.
>> >>>>>>>>>>>>>>>>> Actually,
>> >>>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>> >>>>>>>>> between
>> >>>>>>>>>>>> e6a7f93
>> >>>>>>>>>>>>>>>>>> (first
>> >>>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>> >>>>>>>>>>>> 6592dfa5 (last
>> >>>>>>>>>>>>>>>>>> commit in
>> >>>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>> >>>>>>>>>>>> commits.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible
>> for
>> >>>>>>>> a
>> >>>>>>>>>>>> drop
>> >>>>>>>>>>>>>>>>> between
>> >>>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>> >>>>>>>>>>>> reverted
>> >>>>>>>>>>>>>>>>>> IGNITE-13060
>> >>>>>>>>>>>>>>>>>>>>> now and performance looks the same)
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is
>> not
>> >>>>>>>>>>>> related to
>> >>>>>>>>>>>>>>>>>> drop
>> >>>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>> >>>>>>>>>>>> problem, and
>> >>>>>>>>>>>>>>>>> we
>> >>>>>>>>>>>>>>>>>> can
>> >>>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> >>>>>>>> leave
>> >>>>>>>>>>>> it as is?
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>> >>>>>>>>>>>> refactoring)?
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has
>> an
>> >>>>>>>>>>>> incorrect fix
>> >>>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
>> branch
>> >>>>>>>>>>>> [1], so it
>> >>>>>>>>>>>>>>>>>> cannot be
>> >>>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> >>>>>>>> work
>> >>>>>>>>>>>> with fix
>> >>>>>>>>>>>>>>>>>> versions
>> >>>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>> >>>>>>>>>>>> versions.
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> --AG
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>
>> >>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >>>>>>>>>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> >>>>>>>>>>>>>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Guys,
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>> >>>>>>>>>>>> IGNITE-12568
>> >>>>>>>>>>>>>>>>>> (reverted
>> >>>>>>>>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to
>> hot
>> >>>>>>>>>>>> paths, to
>> >>>>>>>>>>>>>>>>> trace
>> >>>>>>>>>>>>>>>>>>>>>>>> these
>> >>>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>> >>>>>>>>> here.
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>> >>>>>>>>> switch/case
>> >>>>>>>>>>>> block was
>> >>>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>> >>>>>>>>>>>> message factory
>> >>>>>>>>>>>>>>>>>> is on
>> >>>>>>>>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> >>>>>>>> impact
>> >>>>>>>>>>>> on total
>> >>>>>>>>>>>>>>>>>>>>>>>> performance.
>> >>>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>> >>>>>>>>>>>> microbenchmarks,
>> >>>>>>>>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>>>>>>>> found
>> >>>>>>>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>> >>>>>>>>>>>> about 30-35%
>> >>>>>>>>>>>>>>>>>> faster
>> >>>>>>>>>>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with
>> switch/case
>> >>>>>>>>> can
>> >>>>>>>>>>>>>>>>> effectively
>> >>>>>>>>>>>>>>>>>>>>>>>> inline
>> >>>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> >>>>>>>> suppliers
>> >>>>>>>>>>>> relatively
>> >>>>>>>>>>>>>>>>>> heavy
>> >>>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>> >>>>>>>>>>>> rewrite the
>> >>>>>>>>>>>>>>>>> code
>> >>>>>>>>>>>>>>>>>>>>>>>> using
>> >>>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>> >>>>>>>>> interface
>> >>>>>>>>>>>> (to
>> >>>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
>> "invokevirtual"),
>> >>>>>>>>>>>> but it gives
>> >>>>>>>>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> >>>>>>>> looks
>> >>>>>>>>>>>> ugly
>> >>>>>>>>>>>>>>>>>> (lambdas
>> >>>>>>>>>>>>>>>>>>>>>>>> can't
>> >>>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>> >>>>>>>>>>>> optimize the
>> >>>>>>>>>>>>>>>>>> current
>> >>>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>> >>>>>>>>>>>> Andrey Gura,
>> >>>>>>>>>>>>>>>>> as
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>> >>>>>>>>> about
>> >>>>>>>>>>>>>>>>>> optimization?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there
>> are
>> >>>>>>>>>>>> some metrics
>> >>>>>>>>>>>>>>>>>>>>>>>> already
>> >>>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old
>> message
>> >>>>>>>>>>>> factory
>> >>>>>>>>>>>>>>>>>>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>> >>>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>> >>>>>>>>>>>> already released
>> >>>>>>>>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory)
>> is
>> >>>>>>>>>>>> only present
>> >>>>>>>>>>>>>>>>>> in Ignite
>> >>>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever
>> new
>> >>>>>>>>>>>> metrics
>> >>>>>>>>>>>>>>>>>> created for
>> >>>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
>> unblock
>> >>>>>>>>>>>> the release
>> >>>>>>>>>>>>>>>>>> and deal
>> >>>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>> >>>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>
>> >>
>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
Hi Ivan,
The documentation for Ignite 2.9 is kept here:
https://github.com/apache/ignite/tree/IGNITE-7595/docs
There is a readme file with instructions. You can make a pull request to
this branch. However, the installation instruction for Ignite C++ hasn't
been created yet. If you want, you can create the "Setting up Ignite for
C++" page by copying this [1] from readme.io and updating it with your
changes.
-Artem
[1]
https://apacheignite-cpp.readme.io/docs/getting-started-1#building-from-source
On 31.08.2020 09:43, Ivan Daschinsky wrote:
> Artem, in ignite 2.9 a way to build C++ for linux/mac os x was changed
> (autotools to cmake). As an author of this change, I want to
> contribute in documentation.
> As far as I understand, now it should be done through PR to specific
> repository. Could you please help me with this?
>
> пт, 28 авг. 2020 г. в 16:33, Anton Kalashnikov <kaa.dev@yandex.ru
> <ma...@yandex.ru>>:
>
> Hi Guys,
>
> As I understand we will be merging some tickets to release. May I
> suggest also add ticket [1] to 2.9 release.
>
> There are not a lot of changes in code but It's a critical fix for
> the ability to launch ignite in lamba on Azure(There are not any
> workaround).
>
> So if nobody minds let's merge it to 2.9.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13013
> <https://issues.apache.org/jira/browse/IGNITE-13013>
>
> --
> Best regards,
> Anton Kalashnikov
>
>
>
> 28.08.2020, 11:16, "Alex Plehanov" <plehanov.alex@gmail.com
> <ma...@gmail.com>>:
> > Guys,
> >
> > We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
> (reverted it
> > locally) and got the same performance as on 2.8.1
> >
> > IGNITE-13060 (Tracing) - some code was added to hot paths, to
> trace these
> > hot paths, it's clear why we have performance drop here.
> >
> > IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > refactored to an array of message suppliers. The message factory
> is on the
> > hot path, which explains why this commit has an impact on total
> > performance.
> > I've checked JIT assembly output, done some JMH microbenchmarks,
> and found
> > that old implementation of MessageFactory.create() about 30-35%
> faster than
> > the new one. The reason - approach with switch/case can
> effectively inline
> > message creation code, but with an array of suppliers relatively
> heavy
> > "invokeinterface" cannot be skipped. I've tried to rewrite the
> code using
> > an abstract class for suppliers instead of an interface (to
> > replace "invokeinterface" with the "invokevirtual"), but it
> gives back only
> > 10% of method performance and in this case, code looks ugly
> (lambdas can't
> > be used). Currently, I can't find any more ways to optimize the
> current
> > approach (except return to the switch/case block). Andrey Gura,
> as the
> > author of IGNITE-12568, maybe you have some ideas about
> optimization?
> >
> > Perhaps we should revert IGNITE-12568, but there are some
> metrics already
> > created, which can't be rewritten using old message factory
> implementation
> > (IGNITE-12756). Guys, WDYT?
> >
> > пт, 28 авг. 2020 г. в 01:52, Denis Magda <dmagda@apache.org
> <ma...@apache.org>>:
> >
> >> Looks beautiful and easy to use, thanks, Artem! Could you
> please add the
> >> following copyright to the footer of the pages?
> >>
> >> *© 2020 The Apache Software Foundation.*
> >> *Apache, Apache Ignite, the Apache feather and the Apache
> Ignite logo are
> >> either registered trademarks or trademarks of The Apache Software
> >> Foundation. *
> >> *Privacy Policy*
> >>
> >> -
> >> Denis
> >>
> >> On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov <
> >> a.budnikov.ignite@gmail.com
> <ma...@gmail.com>> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> We published the draft of Ignite 2.9 documentation on the
> Apache Ignite
> >>> web-site. The docs are available via the following link:
> >>>
> >>>
> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
> <https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker>
> >>>
> >>> Alex,
> >>>
> >>> Is there an estimate for the release date?
> >>>
> >>> -Artem
> >>>
> >>> On 26.08.2020 17:47, Alex Plehanov wrote:
> >>> > Denis,
> >>> >
> >>> > Currently, we are running mostly
> IgnitePutTxImplicitBenchmark without
> >>> > persistence. For other benchmarks drop is lower and it's
> harder to find
> >>> > problematic commit.
> >>> >
> >>> > ср, 26 авг. 2020 г. в 17:34, Denis Magda <dmagda@apache.org
> <ma...@apache.org>>:
> >>> >
> >>> >> Alex,
> >>> >>
> >>> >> Thanks for sending an update. The drop is quite big. What
> are the
> >>> types of
> >>> >> benchmarks you are observing the degradation for (atomic puts,
> >>> >> transactions, sql, etc.)?
> >>> >>
> >>> >> Let us know if any help by particular committers is required.
> >>> >>
> >>> >> -
> >>> >> Denis
> >>> >>
> >>> >>
> >>> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <
> >>> plehanov.alex@gmail.com <ma...@gmail.com>>
> >>> >> wrote:
> >>> >>
> >>> >>> Hello, guys!
> >>> >>>
> >>> >>> We finally have some benchmark results. Looks like there
> is more than
> >>> one
> >>> >>> commit with a performance drop. Detected drops for those
> commits only
> >>> >>> slightly higher than measurement error, so it was hard to
> find them
> >>> and
> >>> >> we
> >>> >>> are not completely sure we found them all and found them
> right.
> >>> >>>
> >>> >>> Drops detected:
> >>> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing:
> initial
> >>> >>> implementation)
> >>> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568
> MessageFactory is
> >>> >> refactored
> >>> >>> in order to detect registration of message with the same
> direct type)
> >>> >>>
> >>> >>> The total drop we have on our environment - 7-8% and
> perhaps there is
> >>> >>> something else here (benchmarks still in progress, I will
> write if we
> >>> >> find
> >>> >>> more suspected commits).
> >>> >>>
> >>> >>> Ivan Artiukhov, can you please recheck mentioned above
> commits on your
> >>> >>> environment?
> >>> >>>
> >>> >>>
> >>> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
> >>> ilya.kasnacheev@gmail.com <ma...@gmail.com>
> >>> >>> :
> >>> >>>
> >>> >>>> Hello!
> >>> >>>>
> >>> >>>> Readme.io uses blue book :)
> >>> >>>>
> >>> >>>> https://apacheignite.readme.io/docs/performance-tips
> <https://apacheignite.readme.io/docs/performance-tips>
> >>> >>>>
> >>> >>>> I was thinking of something along a blue circle with `i'
> in it, for
> >>> >>>> information items.
> >>> >>>>
> >>> >>>> Regards,
> >>> >>>> --
> >>> >>>> Ilya Kasnacheev
> >>> >>>>
> >>> >>>>
> >>> >>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
> >>> >> a.budnikov.ignite@gmail.com
> <ma...@gmail.com>
> >>> >>>> :
> >>> >>>>
> >>> >>>>>> Search does not seem to work.
> >>> >>>>> It uses mockups right now, but it should be ready when
> the docs are
> >>> >>>>> released.
> >>> >>>>>
> >>> >>>>>> I can see that note blocks are just annotated with
> "Note." Can we
> >>> >>> have
> >>> >>>>> some
> >>> >>>>>> image there?
> >>> >>>>> Do you have a preference as to which image you would
> like to see
> >>> >> there?
> >>> >>>>> -Artem
> >>> >>>>>
> >>> >>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> >>> >>>>>> Hello!
> >>> >>>>>>
> >>> >>>>>> Search does not seem to work. Are we going to have a
> proper search
> >>> >>>>> results
> >>> >>>>>> page? It is often the case that there's none.
> >>> >>>>>>
> >>> >>>>>> I can see that note blocks are just annotated with
> "Note." Can we
> >>> >>> have
> >>> >>>>> some
> >>> >>>>>> image there? Example is
> >>> >>>>>>
> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> <http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning>
> >>> >>>>>>
> >>> >>>>>> Regards,
>
>
>
> --
> Sincerely yours, Ivan Daschinskiy
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
Let’s include in the 2.9 release IGNITE-12718 [1]
This is a closed scope feature of the python thin client.
It was requested by some of our users.
[1] https://github.com/apache/ignite/commit/baf8c673c41a1790ef0a244862e6abfbd4eadbf5
> 31 авг. 2020 г., в 13:25, Alexey Goncharuk <al...@gmail.com> написал(а):
>
> Alexey,
>
> While investigating, I found that IGNITE-12568 has an incorrect fix version
> and is actually present in ignite-2.8.1 branch [1], so it cannot be the
> source of the drop against 2.8.1.
>
> P.S. Looks like we need to enforce a more accurate work with fix versions
> or develop some sort of tooling to verify the fix versions.
>
> --AG
>
> [1]
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>
> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <al...@gmail.com>:
>
>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
>>
>>> Guys,
>>>
>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted it
>>> locally) and got the same performance as on 2.8.1
>>>
>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
>>> hot paths, it's clear why we have performance drop here.
>>>
>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>>> refactored to an array of message suppliers. The message factory is on the
>>> hot path, which explains why this commit has an impact on total
>>> performance.
>>> I've checked JIT assembly output, done some JMH microbenchmarks, and found
>>> that old implementation of MessageFactory.create() about 30-35% faster
>>> than
>>> the new one. The reason - approach with switch/case can effectively inline
>>> message creation code, but with an array of suppliers relatively heavy
>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code using
>>> an abstract class for suppliers instead of an interface (to
>>> replace "invokeinterface" with the "invokevirtual"), but it gives back
>>> only
>>> 10% of method performance and in this case, code looks ugly (lambdas can't
>>> be used). Currently, I can't find any more ways to optimize the current
>>> approach (except return to the switch/case block). Andrey Gura, as the
>>> author of IGNITE-12568, maybe you have some ideas about optimization?
>>>
>>> Perhaps we should revert IGNITE-12568, but there are some metrics already
>>> created, which can't be rewritten using old message factory implementation
>>> (IGNITE-12756). Guys, WDYT?
>>>
>>
>> Alexey,
>>
>> I see that IGNITE-12756 (metrics improvements) is already released in
>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
>> 2.9 that depend on the new message factory to unblock the release and deal
>> with the optimizations in 2.10?
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexey,
For me, all three look quite critical as they: address a memory leak,
address a usability/performance issue with no workaround, and address a
regression in 2.8. If we want to limit the scope, I would still include the
memory leak fix, though.
Thoughts?
пн, 12 окт. 2020 г. в 13:07, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> I've found 3 more tickets which were targeted to 2.9 but merged only to the
> master branch ([1], [2], [3]).
> Do we need these tickets in 2.9? Are they critical?
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-12350
> [2]: https://issues.apache.org/jira/browse/IGNITE-13376
> [3]: https://issues.apache.org/jira/browse/IGNITE-13280
>
> вс, 11 окт. 2020 г. в 16:11, Nikolay Izhikov <ni...@apache.org>:
>
> > Igniters,
> >
> > IGNITE-13553 fixed and cherry-picked to 2.9.
> > We can continue with the release.
> >
> > > 10 окт. 2020 г., в 00:00, Denis Magda <dm...@apache.org> написал(а):
> > >
> > > Nikolay,
> > >
> > > re: the minor improvements. I'm not against of including those if we
> can
> > > prepare the docs before the vote starts. Presently, the docs are
> "frozen"
> > > for the 2.9 release, but I can scratch some time and take part in the
> > docs
> > > review the next week.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov <ni...@apache.org>
> > wrote:
> > >
> > >> Hello, Igniters.
> > >>
> > >> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
> > >> stops in case wrong datatype put in indexed field»
> > >> Can someone, please, help me with the review?
> > >>
> > >> [1] https://github.com/apache/ignite/pull/8330
> > >> [2] https://issues.apache.org/jira/browse/IGNITE-13553
> > >>
> > >> ==================
> > >>
> > >> I propose to include several minor tickets to the scope of the 2.9
> that
> > >> increase Ignite User Experience
> > >>
> > >> CMD tools improvements:
> > >>
> > >> IGNITE-13488 - Command to print metric value
> > >> IGNITE-13426 - Command to print system view content
> > >> IGNITE-13422 - Parameter to explicitly enable experimental commands
> > >>
> > >> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
> > >>
> > >> New system views:
> > >>
> > >> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> > >> IGNITE-13408 BinaryMetadata view.
> > >>
> > >>> 9 окт. 2020 г., в 04:04, Denis Magda <dm...@apache.org> написал(а):
> > >>>
> > >>> @Alex Plehanov <pl...@gmail.com>,
> > >>>
> > >>> If it still makes sense and not too late, you can cherry-pick the
> > commit
> > >>> with the new docs into the 2.9 branch:
> > >>>
> > >>
> >
> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
> > >>>
> > >>> Anyway, it's not crucial as long as we agreed to make an exception
> for
> > >> this
> > >>> release. The docs were already published to the website and assembled
> > >> from
> > >>> the master. Once the vote passes, I'll make them visible and
> traceable
> > >> from
> > >>> the website menus.
> > >>>
> > >>> -
> > >>> Denis
> > >>>
> > >>>
> > >>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> > >> alexey.goncharuk@gmail.com>
> > >>> wrote:
> > >>>
> > >>>> Saikat,
> > >>>>
> > >>>> The plan sounds good to me! Do you have an idea for the timeline of
> > the
> > >>>> module releases? Let me know if I can help in any way.
> > >>>>
> > >>>> Also, I was looking into the modularization IEP and noticed that
> OSGI
> > >>>> module is discussed to be moved to the extensions, but there is no
> > >>>> corresponding ticket. Should I create one? I will be happy to help
> > with
> > >>>> moving this module to extensions.
> > >>>>
> > >>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <
> saikat.maitra@gmail.com
> > >:
> > >>>>
> > >>>>> Hi Alexey,
> > >>>>>
> > >>>>> All the modules as planned in IEP-36
> > >>>>>
> > >>>>
> > >>
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> > >>>>> are migrated to ignite-extensions repository.
> > >>>>>
> > >>>>> We can definitely plan to release ignite-extensions modules.
> > >>>>>
> > >>>>> I have a pending PR related to the kafka module and the PR is in
> > review
> > >>>>> https://github.com/apache/ignite/pull/8222 but its corresponding
> PR
> > >> has
> > >>>>> been merged in ignite-extensions repository.
> > >>>>>
> > >>>>> My thoughts / action items for the migration efforts and releases
> > were
> > >>>>> as follows:
> > >>>>>
> > >>>>> 1. Merge the changes for
> https://github.com/apache/ignite/pull/8222
> > >>>>> 2. Fix the upload nightly packages task for ignite-core to help fix
> > the
> > >>>>> travis build failure in ignite-extensions repository.
> > >>>>> 3. Review and update the README.md files for the ignite-extensions
> > >>>> modules
> > >>>>> as required.
> > >>>>> 4. Update the docs for ignite-extensions modules in ignite-website.
> > >>>>> 5. Release each module separately and share updates.
> > >>>>>
> > >>>>> Please let me know if you have feedback.
> > >>>>>
> > >>>>> Regards,
> > >>>>> Saikat
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
> > >> wrote:
> > >>>>>
> > >>>>>> It seems that gitbox.apache.org is not available on CI/CD.
> > >>>>>>
> > >>>>>> I will try to update VCS to GitHub.
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <plehanov.alex@gmail.com
> >
> > >>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Guys,
> > >>>>>>>
> > >>>>>>> I've tried to build release candidate using TC [1]. But:
> > >>>>>>> 1. I can't choose a branch in this TC task (there is no such
> > field).
> > >>>>>>> 2. On the "default" branch I got an error: Failed to collect
> > changes,
> > >>>>>>> error: List remote refs failed:
> > >>>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
> > >>>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> > >>>> connect
> > >>>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300,
> parent
> > >>>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
> > >>>>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master
> "}
> > >>>>>>>
> > >>>>>>> Is there some problem with this TC task? What am I doing wrong?
> > >>>>>>>
> > >>>>>>> [1] :
> > >>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> > >>>>>>>
> > >>>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> > >>>>>> alexey.goncharuk@gmail.com>:
> > >>>>>>>
> > >>>>>>>> Saikat, Nikolay,
> > >>>>>>>>
> > >>>>>>>> We have migrated a bunch of modules to ignite-extensions
> recently.
> > >>>>>> Given
> > >>>>>>>> that these modules will not be available in Ignite 2.9 anymore
> > (will
> > >>>>>>>> they?), should we also plan to release the extensions?
> > >>>>>>>>
> > >>>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
> > >> plehanov.alex@gmail.com
> > >>>>> :
> > >>>>>>>>
> > >>>>>>>>> Igniters,
> > >>>>>>>>>
> > >>>>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
> > >>>> review
> > >>>>>>>> it,
> > >>>>>>>>> please?
> > >>>>>>>>>
> > >>>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
> > >>>>>>>>>
> > >>>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> > >>>> plehanov.alex@gmail.com
> > >>>>>>> :
> > >>>>>>>>>
> > >>>>>>>>>> Guys,
> > >>>>>>>>>>
> > >>>>>>>>>> I've filled the ticket with reproducer [1] for the discovery
> > bug.
> > >>>>>> This
> > >>>>>>>>> bug
> > >>>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin
> > privately
> > >>>>>> and
> > >>>>>>>>>> decided to revert this ticket. I will do it today (after TC
> bot
> > >>>> visa)
> > >>>>>>>> if
> > >>>>>>>>>> there are no objections.
> > >>>>>>>>>>
> > >>>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> > >>>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> > >>>>>>>>>>
> > >>>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> > >>>> plehanov.alex@gmail.com
> > >>>>>>> :
> > >>>>>>>>>>
> > >>>>>>>>>>> Guys,
> > >>>>>>>>>>>
> > >>>>>>>>>>> During internal testing, we've found a critical bug with
> > >>>>>>>>>>> discovery (cluster falls apart if two nodes segmented
> > >>>> sequentially).
> > >>>>>>>>> This
> > >>>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
> > >>>>>>>>>>> before release. Under investigation now. I'll let you know
> when
> > >> we
> > >>>>>> get
> > >>>>>>>>>>> something.
> > >>>>>>>>>>>
> > >>>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <agura@apache.org
> >:
> > >>>>>>>>>>>
> > >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> > version
> > >>>> of
> > >>>>>>>>> the
> > >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> > generation
> > >>>> in
> > >>>>>>>>> later
> > >>>>>>>>>>>> release, or keep the code clean and fix the regression in
> the
> > >>>> next
> > >>>>>>>>> releases?
> > >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> > fairly
> > >>>>>>>>>>>> straightforward and does not change the semantics, just
> skips
> > >> the
> > >>>>>>>>> factory
> > >>>>>>>>>>>> closures for certain messages.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual
> > for
> > >>>> all
> > >>>>>>>>>>>> workloads (I didn't get any significant drops during
> > >>>> benchmarking).
> > >>>>>>>> So
> > >>>>>>>>>>>> I prefer the runtime generation in later releases.
> > >>>>>>>>>>>>
> > >>>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> > >>>>>>>>>>>> <al...@gmail.com> wrote:
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Alexey, Andrey, Igniters,
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> > version
> > >>>> of
> > >>>>>>>>> the
> > >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> > generation
> > >>>> in
> > >>>>>>>>> later
> > >>>>>>>>>>>> release, or keep the code clean and fix the regression in
> the
> > >>>> next
> > >>>>>>>>> releases?
> > >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> > fairly
> > >>>>>>>>>>>> straightforward and does not change the semantics, just
> skips
> > >> the
> > >>>>>>>>> factory
> > >>>>>>>>>>>> closures for certain messages.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> Personally, I would prefer fixing the regression given that
> > we
> > >>>>>> also
> > >>>>>>>>>>>> introduced tracing in this release.
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>>
> > >>>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> > >>>>>>>> plehanov.alex@gmail.com
> > >>>>>>>>>> :
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> Alexey,
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> > >>>> ed52559eb95
> > >>>>>>>>> and
> > >>>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
> > >>>> average
> > >>>>>> on
> > >>>>>>>>> our
> > >>>>>>>>>>>> environment (about the same results we got benchmarking
> > 65c30ec6
> > >>>> vs
> > >>>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
> > >>>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
> > >> this
> > >>>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> > >>>> servers, 5
> > >>>>>>>>> clients
> > >>>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
> > >>>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> > >>>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> > >>>>>>>>>>>>>>
> > >>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> > >>>>>>>>>>>> a.budnikov.ignite@gmail.com>:
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> Hi Everyone,
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
> > >>>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite
> 2.9,
> > >> you
> > >>>>>> can
> > >>>>>>>>>>>> update the docs by following the instruction.
> Unfortunately, I
> > >>>>>> won't
> > >>>>>>>> be
> > >>>>>>>>>>>> able to spend any time on this project any longer. You can
> > send
> > >>>>>> your
> > >>>>>>>>> pull
> > >>>>>>>>>>>> requests and questions about the documentation to Denis
> Magda.
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> -Artem
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> [1] :
> > >>>>>>>>>>>>
> > >>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> > >>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> > >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Alexey,
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> I've tried to play with message factories locally, but
> > >>>>>>>>>>>> unfortunately, I
> > >>>>>>>>>>>>>>>> cannot spot the difference between old and new
> > >> implementation
> > >>>>>> in
> > >>>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
> > >>>>>>>>>>>> MessageFactoryImpl
> > >>>>>>>>>>>>>>>> with the old switch statement to the
> > ignite-2.9-revert-12568
> > >>>>>>>>> branch
> > >>>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
> > >>>>>>>> compatible
> > >>>>>>>>>>>> with the
> > >>>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> Can you check if this change makes any difference
> > >>>>>>>> performance-wise
> > >>>>>>>>>>>> in your
> > >>>>>>>>>>>>>>>> environment? If yes, we can go with runtime code
> > generation
> > >>>> in
> > >>>>>>>> the
> > >>>>>>>>>>>> long
> > >>>>>>>>>>>>>>>> term: register classes and generate a dynamic message
> > >> factory
> > >>>>>>>> with
> > >>>>>>>>>>>> a switch
> > >>>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
> > >>>> though,
> > >>>>>>>>>>>> obviously).
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> > >>>>>>>>> plehanov.alex@gmail.com
> > >>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Hello guys,
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket
> > [1]),
> > >>>> it
> > >>>>>>>>>>>> reduced the
> > >>>>>>>>>>>>>>>>> drop, but not completely removed it.
> > >>>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> > >>>> patch?
> > >>>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> > >>>> against
> > >>>>>>>>>>>> 2.8.1
> > >>>>>>>>>>>>>>>>> release on your environment?
> > >>>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3%
> drop
> > >>>> left,
> > >>>>>>>>>>>> it's close
> > >>>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
> > >>>>>>>>>>>> showstopper. Guys,
> > >>>>>>>>>>>>>>>>> WDYT?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC
> thin
> > >>>>>>>> driver
> > >>>>>>>>>>>> between 2.8
> > >>>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker
> and
> > >>>>>>>> should
> > >>>>>>>>>>>> be
> > >>>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> > >>>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4]
> > (NIO
> > >>>>>>>>> message
> > >>>>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick
> > it
> > >>>> if
> > >>>>>>>>>>>> there is no
> > >>>>>>>>>>>>>>>>> objection.
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> > >>>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> > >>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> > >>>>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> > >>>>>>>> mmuzaf@apache.org
> > >>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> Alexey,
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release.
> Since
> > >>>>>>>> this
> > >>>>>>>>>>>> issue is
> > >>>>>>>>>>>>>>>>>> related to the new master key change functionality
> which
> > >>>>>>>>>>>> haven't been
> > >>>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick
> > commit
> > >>>>>>>> to
> > >>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>> release branch.
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> [1]
> https://issues.apache.org/jira/browse/IGNITE-13390
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> > >>>>>>>>>>>> nizhikov@apache.org>
> > >>>>>>>>>>>>>>>>> wrote:
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Hello, Igniters.
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client
> fix
> > >>>>>>>> [1]
> > >>>>>>>>>>>> into the
> > >>>>>>>>>>>>>>>>> 2.9
> > >>>>>>>>>>>>>>>>>> release
> > >>>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
> > >> fields
> > >>>>>>>>> in
> > >>>>>>>>>>>> wrong
> > >>>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>> [1]
> https://issues.apache.org/jira/browse/IGNITE-12809
> > >>>>>>>>>>>>>>>>>>> [2]
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> > >>>>>>>>>>>>>>>>>> написал(а):
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can
> optimize
> > >>>>>>>>>>>> anything out of
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no
> > ideas
> > >>>>>>>>>>>> right now),
> > >>>>>>>>>>>>>>>>> so
> > >>>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to
> > the
> > >>>>>>>>>>>> switch
> > >>>>>>>>>>>>>>>>> approach
> > >>>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably,
> > inlining
> > >>>>>>>>> the
> > >>>>>>>>>>>> Ignite
> > >>>>>>>>>>>>>>>>>> messages
> > >>>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick.
> > Let
> > >>>>>>>>> me
> > >>>>>>>>>>>> explore
> > >>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>> code a bit.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes
> for
> > >>>>>>>> the
> > >>>>>>>>>>>>>>>>> performance.
> > >>>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
> > single
> > >>>>>>>>>>>> virtual call
> > >>>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
> > >>>>>>>> amount
> > >>>>>>>>>>>> of other
> > >>>>>>>>>>>>>>>>>> work
> > >>>>>>>>>>>>>>>>>>>> happening during the message processing.
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> > >>>>>>>>>>>> plehanov.alex@gmail.com
> > >>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> > >>>>>>>>> results.
> > >>>>>>>>>>>>>>>>> Actually,
> > >>>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> > >>>>>>>>> between
> > >>>>>>>>>>>> e6a7f93
> > >>>>>>>>>>>>>>>>>> (first
> > >>>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> > >>>>>>>>>>>> 6592dfa5 (last
> > >>>>>>>>>>>>>>>>>> commit in
> > >>>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> > >>>>>>>>>>>> commits.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible
> > for
> > >>>>>>>> a
> > >>>>>>>>>>>> drop
> > >>>>>>>>>>>>>>>>> between
> > >>>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9
> with
> > >>>>>>>>>>>> reverted
> > >>>>>>>>>>>>>>>>>> IGNITE-13060
> > >>>>>>>>>>>>>>>>>>>>> now and performance looks the same)
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is
> > not
> > >>>>>>>>>>>> related to
> > >>>>>>>>>>>>>>>>>> drop
> > >>>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some
> performance
> > >>>>>>>>>>>> problem, and
> > >>>>>>>>>>>>>>>>> we
> > >>>>>>>>>>>>>>>>>> can
> > >>>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> > >>>>>>>> leave
> > >>>>>>>>>>>> it as is?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> > >>>>>>>>>>>> refactoring)?
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> Alexey,
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has
> > an
> > >>>>>>>>>>>> incorrect fix
> > >>>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
> > branch
> > >>>>>>>>>>>> [1], so it
> > >>>>>>>>>>>>>>>>>> cannot be
> > >>>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> > >>>>>>>> work
> > >>>>>>>>>>>> with fix
> > >>>>>>>>>>>>>>>>>> versions
> > >>>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> > >>>>>>>>>>>> versions.
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> --AG
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> [1]
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>
> > >>
> >
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> > >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> > >>>>>>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> > >>>>>>>>>>>>>>>>> plehanov.alex@gmail.com
> > >>>>>>>>>>>>>>>>>>> :
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Guys,
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> > >>>>>>>>>>>> IGNITE-12568
> > >>>>>>>>>>>>>>>>>> (reverted
> > >>>>>>>>>>>>>>>>>>>>>>>> it
> > >>>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on
> 2.8.1
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to
> > hot
> > >>>>>>>>>>>> paths, to
> > >>>>>>>>>>>>>>>>> trace
> > >>>>>>>>>>>>>>>>>>>>>>>> these
> > >>>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance
> drop
> > >>>>>>>>> here.
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> > >>>>>>>>> switch/case
> > >>>>>>>>>>>> block was
> > >>>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> > >>>>>>>>>>>> message factory
> > >>>>>>>>>>>>>>>>>> is on
> > >>>>>>>>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> > >>>>>>>> impact
> > >>>>>>>>>>>> on total
> > >>>>>>>>>>>>>>>>>>>>>>>> performance.
> > >>>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> > >>>>>>>>>>>> microbenchmarks,
> > >>>>>>>>>>>>>>>>>> and
> > >>>>>>>>>>>>>>>>>>>>>>>> found
> > >>>>>>>>>>>>>>>>>>>>>>>> that old implementation of
> MessageFactory.create()
> > >>>>>>>>>>>> about 30-35%
> > >>>>>>>>>>>>>>>>>> faster
> > >>>>>>>>>>>>>>>>>>>>>>>> than
> > >>>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with
> > switch/case
> > >>>>>>>>> can
> > >>>>>>>>>>>>>>>>> effectively
> > >>>>>>>>>>>>>>>>>>>>>>>> inline
> > >>>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
> > >>>>>>>> suppliers
> > >>>>>>>>>>>> relatively
> > >>>>>>>>>>>>>>>>>> heavy
> > >>>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried
> to
> > >>>>>>>>>>>> rewrite the
> > >>>>>>>>>>>>>>>>> code
> > >>>>>>>>>>>>>>>>>>>>>>>> using
> > >>>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> > >>>>>>>>> interface
> > >>>>>>>>>>>> (to
> > >>>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
> > "invokevirtual"),
> > >>>>>>>>>>>> but it gives
> > >>>>>>>>>>>>>>>>>> back
> > >>>>>>>>>>>>>>>>>>>>>>>> only
> > >>>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> > >>>>>>>> looks
> > >>>>>>>>>>>> ugly
> > >>>>>>>>>>>>>>>>>> (lambdas
> > >>>>>>>>>>>>>>>>>>>>>>>> can't
> > >>>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways
> to
> > >>>>>>>>>>>> optimize the
> > >>>>>>>>>>>>>>>>>> current
> > >>>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case
> block).
> > >>>>>>>>>>>> Andrey Gura,
> > >>>>>>>>>>>>>>>>> as
> > >>>>>>>>>>>>>>>>>> the
> > >>>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some
> ideas
> > >>>>>>>>> about
> > >>>>>>>>>>>>>>>>>> optimization?
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there
> > are
> > >>>>>>>>>>>> some metrics
> > >>>>>>>>>>>>>>>>>>>>>>>> already
> > >>>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old
> > message
> > >>>>>>>>>>>> factory
> > >>>>>>>>>>>>>>>>>>>>>>>> implementation
> > >>>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> > >>>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> Alexey,
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> > >>>>>>>>>>>> already released
> > >>>>>>>>>>>>>>>>>> in
> > >>>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory)
> > is
> > >>>>>>>>>>>> only present
> > >>>>>>>>>>>>>>>>>> in Ignite
> > >>>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever
> > new
> > >>>>>>>>>>>> metrics
> > >>>>>>>>>>>>>>>>>> created for
> > >>>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
> > unblock
> > >>>>>>>>>>>> the release
> > >>>>>>>>>>>>>>>>>> and deal
> > >>>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> > >>>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>>>>>>
> > >>>>>>>>>>>>
> > >>>>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>
> > >>
> > >>
> >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've found 3 more tickets which were targeted to 2.9 but merged only to the
master branch ([1], [2], [3]).
Do we need these tickets in 2.9? Are they critical?
[1]: https://issues.apache.org/jira/browse/IGNITE-12350
[2]: https://issues.apache.org/jira/browse/IGNITE-13376
[3]: https://issues.apache.org/jira/browse/IGNITE-13280
вс, 11 окт. 2020 г. в 16:11, Nikolay Izhikov <ni...@apache.org>:
> Igniters,
>
> IGNITE-13553 fixed and cherry-picked to 2.9.
> We can continue with the release.
>
> > 10 окт. 2020 г., в 00:00, Denis Magda <dm...@apache.org> написал(а):
> >
> > Nikolay,
> >
> > re: the minor improvements. I'm not against of including those if we can
> > prepare the docs before the vote starts. Presently, the docs are "frozen"
> > for the 2.9 release, but I can scratch some time and take part in the
> docs
> > review the next week.
> >
> > -
> > Denis
> >
> >
> > On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov <ni...@apache.org>
> wrote:
> >
> >> Hello, Igniters.
> >>
> >> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
> >> stops in case wrong datatype put in indexed field»
> >> Can someone, please, help me with the review?
> >>
> >> [1] https://github.com/apache/ignite/pull/8330
> >> [2] https://issues.apache.org/jira/browse/IGNITE-13553
> >>
> >> ==================
> >>
> >> I propose to include several minor tickets to the scope of the 2.9 that
> >> increase Ignite User Experience
> >>
> >> CMD tools improvements:
> >>
> >> IGNITE-13488 - Command to print metric value
> >> IGNITE-13426 - Command to print system view content
> >> IGNITE-13422 - Parameter to explicitly enable experimental commands
> >>
> >> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
> >>
> >> New system views:
> >>
> >> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> >> IGNITE-13408 BinaryMetadata view.
> >>
> >>> 9 окт. 2020 г., в 04:04, Denis Magda <dm...@apache.org> написал(а):
> >>>
> >>> @Alex Plehanov <pl...@gmail.com>,
> >>>
> >>> If it still makes sense and not too late, you can cherry-pick the
> commit
> >>> with the new docs into the 2.9 branch:
> >>>
> >>
> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
> >>>
> >>> Anyway, it's not crucial as long as we agreed to make an exception for
> >> this
> >>> release. The docs were already published to the website and assembled
> >> from
> >>> the master. Once the vote passes, I'll make them visible and traceable
> >> from
> >>> the website menus.
> >>>
> >>> -
> >>> Denis
> >>>
> >>>
> >>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com>
> >>> wrote:
> >>>
> >>>> Saikat,
> >>>>
> >>>> The plan sounds good to me! Do you have an idea for the timeline of
> the
> >>>> module releases? Let me know if I can help in any way.
> >>>>
> >>>> Also, I was looking into the modularization IEP and noticed that OSGI
> >>>> module is discussed to be moved to the extensions, but there is no
> >>>> corresponding ticket. Should I create one? I will be happy to help
> with
> >>>> moving this module to extensions.
> >>>>
> >>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <saikat.maitra@gmail.com
> >:
> >>>>
> >>>>> Hi Alexey,
> >>>>>
> >>>>> All the modules as planned in IEP-36
> >>>>>
> >>>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> >>>>> are migrated to ignite-extensions repository.
> >>>>>
> >>>>> We can definitely plan to release ignite-extensions modules.
> >>>>>
> >>>>> I have a pending PR related to the kafka module and the PR is in
> review
> >>>>> https://github.com/apache/ignite/pull/8222 but its corresponding PR
> >> has
> >>>>> been merged in ignite-extensions repository.
> >>>>>
> >>>>> My thoughts / action items for the migration efforts and releases
> were
> >>>>> as follows:
> >>>>>
> >>>>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
> >>>>> 2. Fix the upload nightly packages task for ignite-core to help fix
> the
> >>>>> travis build failure in ignite-extensions repository.
> >>>>> 3. Review and update the README.md files for the ignite-extensions
> >>>> modules
> >>>>> as required.
> >>>>> 4. Update the docs for ignite-extensions modules in ignite-website.
> >>>>> 5. Release each module separately and share updates.
> >>>>>
> >>>>> Please let me know if you have feedback.
> >>>>>
> >>>>> Regards,
> >>>>> Saikat
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
> >> wrote:
> >>>>>
> >>>>>> It seems that gitbox.apache.org is not available on CI/CD.
> >>>>>>
> >>>>>> I will try to update VCS to GitHub.
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
> >>>>>> wrote:
> >>>>>>>
> >>>>>>> Guys,
> >>>>>>>
> >>>>>>> I've tried to build release candidate using TC [1]. But:
> >>>>>>> 1. I can't choose a branch in this TC task (there is no such
> field).
> >>>>>>> 2. On the "default" branch I got an error: Failed to collect
> changes,
> >>>>>>> error: List remote refs failed:
> >>>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
> >>>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> >>>> connect
> >>>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> >>>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
> >>>>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
> >>>>>>>
> >>>>>>> Is there some problem with this TC task? What am I doing wrong?
> >>>>>>>
> >>>>>>> [1] :
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >>>>>>>
> >>>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com>:
> >>>>>>>
> >>>>>>>> Saikat, Nikolay,
> >>>>>>>>
> >>>>>>>> We have migrated a bunch of modules to ignite-extensions recently.
> >>>>>> Given
> >>>>>>>> that these modules will not be available in Ignite 2.9 anymore
> (will
> >>>>>>>> they?), should we also plan to release the extensions?
> >>>>>>>>
> >>>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
> >> plehanov.alex@gmail.com
> >>>>> :
> >>>>>>>>
> >>>>>>>>> Igniters,
> >>>>>>>>>
> >>>>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
> >>>> review
> >>>>>>>> it,
> >>>>>>>>> please?
> >>>>>>>>>
> >>>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
> >>>>>>>>>
> >>>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> >>>> plehanov.alex@gmail.com
> >>>>>>> :
> >>>>>>>>>
> >>>>>>>>>> Guys,
> >>>>>>>>>>
> >>>>>>>>>> I've filled the ticket with reproducer [1] for the discovery
> bug.
> >>>>>> This
> >>>>>>>>> bug
> >>>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin
> privately
> >>>>>> and
> >>>>>>>>>> decided to revert this ticket. I will do it today (after TC bot
> >>>> visa)
> >>>>>>>> if
> >>>>>>>>>> there are no objections.
> >>>>>>>>>>
> >>>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >>>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >>>>>>>>>>
> >>>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> >>>> plehanov.alex@gmail.com
> >>>>>>> :
> >>>>>>>>>>
> >>>>>>>>>>> Guys,
> >>>>>>>>>>>
> >>>>>>>>>>> During internal testing, we've found a critical bug with
> >>>>>>>>>>> discovery (cluster falls apart if two nodes segmented
> >>>> sequentially).
> >>>>>>>>> This
> >>>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >>>>>>>>>>> before release. Under investigation now. I'll let you know when
> >> we
> >>>>>> get
> >>>>>>>>>>> something.
> >>>>>>>>>>>
> >>>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >>>>>>>>>>>
> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> version
> >>>> of
> >>>>>>>>> the
> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> generation
> >>>> in
> >>>>>>>>> later
> >>>>>>>>>>>> release, or keep the code clean and fix the regression in the
> >>>> next
> >>>>>>>>> releases?
> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> fairly
> >>>>>>>>>>>> straightforward and does not change the semantics, just skips
> >> the
> >>>>>>>>> factory
> >>>>>>>>>>>> closures for certain messages.
> >>>>>>>>>>>>
> >>>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual
> for
> >>>> all
> >>>>>>>>>>>> workloads (I didn't get any significant drops during
> >>>> benchmarking).
> >>>>>>>> So
> >>>>>>>>>>>> I prefer the runtime generation in later releases.
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >>>>>>>>>>>> <al...@gmail.com> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Alexey, Andrey, Igniters,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked'
> version
> >>>> of
> >>>>>>>>> the
> >>>>>>>>>>>> message factory in 2.9 and go for the runtime message
> generation
> >>>> in
> >>>>>>>>> later
> >>>>>>>>>>>> release, or keep the code clean and fix the regression in the
> >>>> next
> >>>>>>>>> releases?
> >>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is
> fairly
> >>>>>>>>>>>> straightforward and does not change the semantics, just skips
> >> the
> >>>>>>>>> factory
> >>>>>>>>>>>> closures for certain messages.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Personally, I would prefer fixing the regression given that
> we
> >>>>>> also
> >>>>>>>>>>>> introduced tracing in this release.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>> :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> >>>> ed52559eb95
> >>>>>>>>> and
> >>>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
> >>>> average
> >>>>>> on
> >>>>>>>>> our
> >>>>>>>>>>>> environment (about the same results we got benchmarking
> 65c30ec6
> >>>> vs
> >>>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
> >>>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
> >> this
> >>>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> >>>> servers, 5
> >>>>>>>>> clients
> >>>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
> >>>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >>>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >>>>>>>>>>>> a.budnikov.ignite@gmail.com>:
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hi Everyone,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
> >>>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
> >> you
> >>>>>> can
> >>>>>>>>>>>> update the docs by following the instruction. Unfortunately, I
> >>>>>> won't
> >>>>>>>> be
> >>>>>>>>>>>> able to spend any time on this project any longer. You can
> send
> >>>>>> your
> >>>>>>>>> pull
> >>>>>>>>>>>> requests and questions about the documentation to Denis Magda.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> -Artem
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> [1] :
> >>>>>>>>>>>>
> >>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I've tried to play with message factories locally, but
> >>>>>>>>>>>> unfortunately, I
> >>>>>>>>>>>>>>>> cannot spot the difference between old and new
> >> implementation
> >>>>>> in
> >>>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >>>>>>>>>>>> MessageFactoryImpl
> >>>>>>>>>>>>>>>> with the old switch statement to the
> ignite-2.9-revert-12568
> >>>>>>>>> branch
> >>>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >>>>>>>> compatible
> >>>>>>>>>>>> with the
> >>>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Can you check if this change makes any difference
> >>>>>>>> performance-wise
> >>>>>>>>>>>> in your
> >>>>>>>>>>>>>>>> environment? If yes, we can go with runtime code
> generation
> >>>> in
> >>>>>>>> the
> >>>>>>>>>>>> long
> >>>>>>>>>>>>>>>> term: register classes and generate a dynamic message
> >> factory
> >>>>>>>> with
> >>>>>>>>>>>> a switch
> >>>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
> >>>> though,
> >>>>>>>>>>>> obviously).
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hello guys,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket
> [1]),
> >>>> it
> >>>>>>>>>>>> reduced the
> >>>>>>>>>>>>>>>>> drop, but not completely removed it.
> >>>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> >>>> patch?
> >>>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> >>>> against
> >>>>>>>>>>>> 2.8.1
> >>>>>>>>>>>>>>>>> release on your environment?
> >>>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3% drop
> >>>> left,
> >>>>>>>>>>>> it's close
> >>>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
> >>>>>>>>>>>> showstopper. Guys,
> >>>>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
> >>>>>>>> driver
> >>>>>>>>>>>> between 2.8
> >>>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
> >>>>>>>> should
> >>>>>>>>>>>> be
> >>>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >>>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4]
> (NIO
> >>>>>>>>> message
> >>>>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick
> it
> >>>> if
> >>>>>>>>>>>> there is no
> >>>>>>>>>>>>>>>>> objection.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>>>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >>>>>>>> mmuzaf@apache.org
> >>>>>>>>>> :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
> >>>>>>>> this
> >>>>>>>>>>>> issue is
> >>>>>>>>>>>>>>>>>> related to the new master key change functionality which
> >>>>>>>>>>>> haven't been
> >>>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick
> commit
> >>>>>>>> to
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> release branch.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >>>>>>>>>>>> nizhikov@apache.org>
> >>>>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Hello, Igniters.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
> >>>>>>>> [1]
> >>>>>>>>>>>> into the
> >>>>>>>>>>>>>>>>> 2.9
> >>>>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
> >> fields
> >>>>>>>>> in
> >>>>>>>>>>>> wrong
> >>>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>>>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>>>>>> написал(а):
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
> >>>>>>>>>>>> anything out of
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no
> ideas
> >>>>>>>>>>>> right now),
> >>>>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to
> the
> >>>>>>>>>>>> switch
> >>>>>>>>>>>>>>>>> approach
> >>>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably,
> inlining
> >>>>>>>>> the
> >>>>>>>>>>>> Ignite
> >>>>>>>>>>>>>>>>>> messages
> >>>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick.
> Let
> >>>>>>>>> me
> >>>>>>>>>>>> explore
> >>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>> code a bit.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
> >>>>>>>> the
> >>>>>>>>>>>>>>>>> performance.
> >>>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
> single
> >>>>>>>>>>>> virtual call
> >>>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
> >>>>>>>> amount
> >>>>>>>>>>>> of other
> >>>>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>>>> happening during the message processing.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >>>>>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >>>>>>>>> results.
> >>>>>>>>>>>>>>>>> Actually,
> >>>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >>>>>>>>> between
> >>>>>>>>>>>> e6a7f93
> >>>>>>>>>>>>>>>>>> (first
> >>>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >>>>>>>>>>>> 6592dfa5 (last
> >>>>>>>>>>>>>>>>>> commit in
> >>>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> >>>>>>>>>>>> commits.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible
> for
> >>>>>>>> a
> >>>>>>>>>>>> drop
> >>>>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >>>>>>>>>>>> reverted
> >>>>>>>>>>>>>>>>>> IGNITE-13060
> >>>>>>>>>>>>>>>>>>>>> now and performance looks the same)
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is
> not
> >>>>>>>>>>>> related to
> >>>>>>>>>>>>>>>>>> drop
> >>>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
> >>>>>>>>>>>> problem, and
> >>>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> >>>>>>>> leave
> >>>>>>>>>>>> it as is?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> >>>>>>>>>>>> refactoring)?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has
> an
> >>>>>>>>>>>> incorrect fix
> >>>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
> branch
> >>>>>>>>>>>> [1], so it
> >>>>>>>>>>>>>>>>>> cannot be
> >>>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> >>>>>>>> work
> >>>>>>>>>>>> with fix
> >>>>>>>>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >>>>>>>>>>>> versions.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> --AG
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>>>>>>>>>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >>>>>>>>>>>> IGNITE-12568
> >>>>>>>>>>>>>>>>>> (reverted
> >>>>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to
> hot
> >>>>>>>>>>>> paths, to
> >>>>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
> >>>>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >>>>>>>>> switch/case
> >>>>>>>>>>>> block was
> >>>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> >>>>>>>>>>>> message factory
> >>>>>>>>>>>>>>>>>> is on
> >>>>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >>>>>>>> impact
> >>>>>>>>>>>> on total
> >>>>>>>>>>>>>>>>>>>>>>>> performance.
> >>>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >>>>>>>>>>>> microbenchmarks,
> >>>>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>>>> found
> >>>>>>>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
> >>>>>>>>>>>> about 30-35%
> >>>>>>>>>>>>>>>>>> faster
> >>>>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with
> switch/case
> >>>>>>>>> can
> >>>>>>>>>>>>>>>>> effectively
> >>>>>>>>>>>>>>>>>>>>>>>> inline
> >>>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >>>>>>>> suppliers
> >>>>>>>>>>>> relatively
> >>>>>>>>>>>>>>>>>> heavy
> >>>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
> >>>>>>>>>>>> rewrite the
> >>>>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>>>> using
> >>>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >>>>>>>>> interface
> >>>>>>>>>>>> (to
> >>>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
> "invokevirtual"),
> >>>>>>>>>>>> but it gives
> >>>>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> >>>>>>>> looks
> >>>>>>>>>>>> ugly
> >>>>>>>>>>>>>>>>>> (lambdas
> >>>>>>>>>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
> >>>>>>>>>>>> optimize the
> >>>>>>>>>>>>>>>>>> current
> >>>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
> >>>>>>>>>>>> Andrey Gura,
> >>>>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
> >>>>>>>>> about
> >>>>>>>>>>>>>>>>>> optimization?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there
> are
> >>>>>>>>>>>> some metrics
> >>>>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old
> message
> >>>>>>>>>>>> factory
> >>>>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >>>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> >>>>>>>>>>>> already released
> >>>>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory)
> is
> >>>>>>>>>>>> only present
> >>>>>>>>>>>>>>>>>> in Ignite
> >>>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever
> new
> >>>>>>>>>>>> metrics
> >>>>>>>>>>>>>>>>>> created for
> >>>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
> unblock
> >>>>>>>>>>>> the release
> >>>>>>>>>>>>>>>>>> and deal
> >>>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >>>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>
> >>
>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
Igniters,
IGNITE-13553 fixed and cherry-picked to 2.9.
We can continue with the release.
> 10 окт. 2020 г., в 00:00, Denis Magda <dm...@apache.org> написал(а):
>
> Nikolay,
>
> re: the minor improvements. I'm not against of including those if we can
> prepare the docs before the vote starts. Presently, the docs are "frozen"
> for the 2.9 release, but I can scratch some time and take part in the docs
> review the next week.
>
> -
> Denis
>
>
> On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov <ni...@apache.org> wrote:
>
>> Hello, Igniters.
>>
>> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
>> stops in case wrong datatype put in indexed field»
>> Can someone, please, help me with the review?
>>
>> [1] https://github.com/apache/ignite/pull/8330
>> [2] https://issues.apache.org/jira/browse/IGNITE-13553
>>
>> ==================
>>
>> I propose to include several minor tickets to the scope of the 2.9 that
>> increase Ignite User Experience
>>
>> CMD tools improvements:
>>
>> IGNITE-13488 - Command to print metric value
>> IGNITE-13426 - Command to print system view content
>> IGNITE-13422 - Parameter to explicitly enable experimental commands
>>
>> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
>>
>> New system views:
>>
>> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
>> IGNITE-13408 BinaryMetadata view.
>>
>>> 9 окт. 2020 г., в 04:04, Denis Magda <dm...@apache.org> написал(а):
>>>
>>> @Alex Plehanov <pl...@gmail.com>,
>>>
>>> If it still makes sense and not too late, you can cherry-pick the commit
>>> with the new docs into the 2.9 branch:
>>>
>> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
>>>
>>> Anyway, it's not crucial as long as we agreed to make an exception for
>> this
>>> release. The docs were already published to the website and assembled
>> from
>>> the master. Once the vote passes, I'll make them visible and traceable
>> from
>>> the website menus.
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
>> alexey.goncharuk@gmail.com>
>>> wrote:
>>>
>>>> Saikat,
>>>>
>>>> The plan sounds good to me! Do you have an idea for the timeline of the
>>>> module releases? Let me know if I can help in any way.
>>>>
>>>> Also, I was looking into the modularization IEP and noticed that OSGI
>>>> module is discussed to be moved to the extensions, but there is no
>>>> corresponding ticket. Should I create one? I will be happy to help with
>>>> moving this module to extensions.
>>>>
>>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>>>
>>>>> Hi Alexey,
>>>>>
>>>>> All the modules as planned in IEP-36
>>>>>
>>>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>>>>> are migrated to ignite-extensions repository.
>>>>>
>>>>> We can definitely plan to release ignite-extensions modules.
>>>>>
>>>>> I have a pending PR related to the kafka module and the PR is in review
>>>>> https://github.com/apache/ignite/pull/8222 but its corresponding PR
>> has
>>>>> been merged in ignite-extensions repository.
>>>>>
>>>>> My thoughts / action items for the migration efforts and releases were
>>>>> as follows:
>>>>>
>>>>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>>>>> 2. Fix the upload nightly packages task for ignite-core to help fix the
>>>>> travis build failure in ignite-extensions repository.
>>>>> 3. Review and update the README.md files for the ignite-extensions
>>>> modules
>>>>> as required.
>>>>> 4. Update the docs for ignite-extensions modules in ignite-website.
>>>>> 5. Release each module separately and share updates.
>>>>>
>>>>> Please let me know if you have feedback.
>>>>>
>>>>> Regards,
>>>>> Saikat
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
>> wrote:
>>>>>
>>>>>> It seems that gitbox.apache.org is not available on CI/CD.
>>>>>>
>>>>>> I will try to update VCS to GitHub.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> Guys,
>>>>>>>
>>>>>>> I've tried to build release candidate using TC [1]. But:
>>>>>>> 1. I can't choose a branch in this TC task (there is no such field).
>>>>>>> 2. On the "default" branch I got an error: Failed to collect changes,
>>>>>>> error: List remote refs failed:
>>>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
>>>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>>>> connect
>>>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>>>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
>>>>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>>>>>>>
>>>>>>> Is there some problem with this TC task? What am I doing wrong?
>>>>>>>
>>>>>>> [1] :
>>>>>>>
>>>>>>
>>>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>>>>>>>
>>>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com>:
>>>>>>>
>>>>>>>> Saikat, Nikolay,
>>>>>>>>
>>>>>>>> We have migrated a bunch of modules to ignite-extensions recently.
>>>>>> Given
>>>>>>>> that these modules will not be available in Ignite 2.9 anymore (will
>>>>>>>> they?), should we also plan to release the extensions?
>>>>>>>>
>>>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>> plehanov.alex@gmail.com
>>>>> :
>>>>>>>>
>>>>>>>>> Igniters,
>>>>>>>>>
>>>>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
>>>> review
>>>>>>>> it,
>>>>>>>>> please?
>>>>>>>>>
>>>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
>>>>>>>>>
>>>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>>>>> :
>>>>>>>>>
>>>>>>>>>> Guys,
>>>>>>>>>>
>>>>>>>>>> I've filled the ticket with reproducer [1] for the discovery bug.
>>>>>> This
>>>>>>>>> bug
>>>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>>>>>> and
>>>>>>>>>> decided to revert this ticket. I will do it today (after TC bot
>>>> visa)
>>>>>>>> if
>>>>>>>>>> there are no objections.
>>>>>>>>>>
>>>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>>>>>>>>>
>>>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>>>>> :
>>>>>>>>>>
>>>>>>>>>>> Guys,
>>>>>>>>>>>
>>>>>>>>>>> During internal testing, we've found a critical bug with
>>>>>>>>>>> discovery (cluster falls apart if two nodes segmented
>>>> sequentially).
>>>>>>>>> This
>>>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>>>>>>>>>> before release. Under investigation now. I'll let you know when
>> we
>>>>>> get
>>>>>>>>>>> something.
>>>>>>>>>>>
>>>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>>>>>>>>>>
>>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
>>>> of
>>>>>>>>> the
>>>>>>>>>>>> message factory in 2.9 and go for the runtime message generation
>>>> in
>>>>>>>>> later
>>>>>>>>>>>> release, or keep the code clean and fix the regression in the
>>>> next
>>>>>>>>> releases?
>>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>>>>>>>> straightforward and does not change the semantics, just skips
>> the
>>>>>>>>> factory
>>>>>>>>>>>> closures for certain messages.
>>>>>>>>>>>>
>>>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
>>>> all
>>>>>>>>>>>> workloads (I didn't get any significant drops during
>>>> benchmarking).
>>>>>>>> So
>>>>>>>>>>>> I prefer the runtime generation in later releases.
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>>>>>>>>>>> <al...@gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alexey, Andrey, Igniters,
>>>>>>>>>>>>>
>>>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
>>>> of
>>>>>>>>> the
>>>>>>>>>>>> message factory in 2.9 and go for the runtime message generation
>>>> in
>>>>>>>>> later
>>>>>>>>>>>> release, or keep the code clean and fix the regression in the
>>>> next
>>>>>>>>> releases?
>>>>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>>>>>>>> straightforward and does not change the semantics, just skips
>> the
>>>>>>>>> factory
>>>>>>>>>>>> closures for certain messages.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Personally, I would prefer fixing the regression given that we
>>>>>> also
>>>>>>>>>>>> introduced tracing in this release.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>>>> ed52559eb95
>>>>>>>>> and
>>>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
>>>> average
>>>>>> on
>>>>>>>>> our
>>>>>>>>>>>> environment (about the same results we got benchmarking 65c30ec6
>>>> vs
>>>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
>>>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>> this
>>>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>>>> servers, 5
>>>>>>>>> clients
>>>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
>>>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>>>>>>>>>>> a.budnikov.ignite@gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
>>>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>> you
>>>>>> can
>>>>>>>>>>>> update the docs by following the instruction. Unfortunately, I
>>>>>> won't
>>>>>>>> be
>>>>>>>>>>>> able to spend any time on this project any longer. You can send
>>>>>> your
>>>>>>>>> pull
>>>>>>>>>>>> requests and questions about the documentation to Denis Magda.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> -Artem
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] :
>>>>>>>>>>>>
>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I've tried to play with message factories locally, but
>>>>>>>>>>>> unfortunately, I
>>>>>>>>>>>>>>>> cannot spot the difference between old and new
>> implementation
>>>>>> in
>>>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>>>>>>>>>>> MessageFactoryImpl
>>>>>>>>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>>>>>>>>> branch
>>>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
>>>>>>>> compatible
>>>>>>>>>>>> with the
>>>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you check if this change makes any difference
>>>>>>>> performance-wise
>>>>>>>>>>>> in your
>>>>>>>>>>>>>>>> environment? If yes, we can go with runtime code generation
>>>> in
>>>>>>>> the
>>>>>>>>>>>> long
>>>>>>>>>>>>>>>> term: register classes and generate a dynamic message
>> factory
>>>>>>>> with
>>>>>>>>>>>> a switch
>>>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
>>>> though,
>>>>>>>>>>>> obviously).
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello guys,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
>>>> it
>>>>>>>>>>>> reduced the
>>>>>>>>>>>>>>>>> drop, but not completely removed it.
>>>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>>>> patch?
>>>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>>>> against
>>>>>>>>>>>> 2.8.1
>>>>>>>>>>>>>>>>> release on your environment?
>>>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>>>> left,
>>>>>>>>>>>> it's close
>>>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
>>>>>>>>>>>> showstopper. Guys,
>>>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>>>>>>>> driver
>>>>>>>>>>>> between 2.8
>>>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>>>>>>>> should
>>>>>>>>>>>> be
>>>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>>>>>>>> message
>>>>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
>>>> if
>>>>>>>>>>>> there is no
>>>>>>>>>>>>>>>>> objection.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>>>>>>>> mmuzaf@apache.org
>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>>>>>>>> this
>>>>>>>>>>>> issue is
>>>>>>>>>>>>>>>>>> related to the new master key change functionality which
>>>>>>>>>>>> haven't been
>>>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>>>>>>>> to
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> release branch.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>>>>>>>>>>> nizhikov@apache.org>
>>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hello, Igniters.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>>>>>>>> [1]
>>>>>>>>>>>> into the
>>>>>>>>>>>>>>>>> 2.9
>>>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>> fields
>>>>>>>>> in
>>>>>>>>>>>> wrong
>>>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>>>> написал(а):
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>>>>>>>>>>> anything out of
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>>>>>>>>>>>> right now),
>>>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>>>>>>>>>>> switch
>>>>>>>>>>>>>>>>> approach
>>>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>>>>>>>> the
>>>>>>>>>>>> Ignite
>>>>>>>>>>>>>>>>>> messages
>>>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>>>>>>>>> me
>>>>>>>>>>>> explore
>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>> code a bit.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>>>>>>>> the
>>>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>>>>>>>>>>>> virtual call
>>>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
>>>>>>>> amount
>>>>>>>>>>>> of other
>>>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>>>> happening during the message processing.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>>>>>>>> results.
>>>>>>>>>>>>>>>>> Actually,
>>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>>>>>>>> between
>>>>>>>>>>>> e6a7f93
>>>>>>>>>>>>>>>>>> (first
>>>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>>>>>>>>>>> 6592dfa5 (last
>>>>>>>>>>>>>>>>>> commit in
>>>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>>>>>>>>>>> commits.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>>>>>>>> a
>>>>>>>>>>>> drop
>>>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>>>>>>>>>>> reverted
>>>>>>>>>>>>>>>>>> IGNITE-13060
>>>>>>>>>>>>>>>>>>>>> now and performance looks the same)
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>>>>>>>>>>> related to
>>>>>>>>>>>>>>>>>> drop
>>>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>>>>>>>>>>> problem, and
>>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>>>>>>>> leave
>>>>>>>>>>>> it as is?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>>>>>>>>>>> refactoring)?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>>>>>>>>>>> incorrect fix
>>>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>>>>>>>>>>>> [1], so it
>>>>>>>>>>>>>>>>>> cannot be
>>>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>>>>>>>> work
>>>>>>>>>>>> with fix
>>>>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>>>>>>>>>>> versions.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> --AG
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>>>>>>>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>>>>>>>>>> IGNITE-12568
>>>>>>>>>>>>>>>>>> (reverted
>>>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>>>>>>>>>>> paths, to
>>>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>>>>>>>> switch/case
>>>>>>>>>>>> block was
>>>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>>>>>>>>>>> message factory
>>>>>>>>>>>>>>>>>> is on
>>>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>>>>>>>> impact
>>>>>>>>>>>> on total
>>>>>>>>>>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>>>>>>>>>>> microbenchmarks,
>>>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>>>>>>>>>>> about 30-35%
>>>>>>>>>>>>>>>>>> faster
>>>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>>>>>>>> can
>>>>>>>>>>>>>>>>> effectively
>>>>>>>>>>>>>>>>>>>>>>>> inline
>>>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
>>>>>>>> suppliers
>>>>>>>>>>>> relatively
>>>>>>>>>>>>>>>>>> heavy
>>>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>>>>>>>>>>> rewrite the
>>>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>>>>>>>> interface
>>>>>>>>>>>> (to
>>>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>>>>>>>>>>>> but it gives
>>>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>>>>>>>> looks
>>>>>>>>>>>> ugly
>>>>>>>>>>>>>>>>>> (lambdas
>>>>>>>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>>>>>>>>>>> optimize the
>>>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>>>>>>>>>>> Andrey Gura,
>>>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>>>>>>>> about
>>>>>>>>>>>>>>>>>> optimization?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>>>>>>>>>>>> some metrics
>>>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>>>>>>>>>>> factory
>>>>>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>>>>>>>>>>> already released
>>>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>>>>>>>>>>> only present
>>>>>>>>>>>>>>>>>> in Ignite
>>>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>>>>>>>>>>> metrics
>>>>>>>>>>>>>>>>>> created for
>>>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>>>>>>>>>>>> the release
>>>>>>>>>>>>>>>>>> and deal
>>>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>>>
>>>>
>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Nikolay,
re: the minor improvements. I'm not against of including those if we can
prepare the docs before the vote starts. Presently, the docs are "frozen"
for the 2.9 release, but I can scratch some time and take part in the docs
review the next week.
-
Denis
On Fri, Oct 9, 2020 at 12:40 AM Nikolay Izhikov <ni...@apache.org> wrote:
> Hello, Igniters.
>
> I prepared a patch [1] for blocker ticket [2] - «Server node fail and
> stops in case wrong datatype put in indexed field»
> Can someone, please, help me with the review?
>
> [1] https://github.com/apache/ignite/pull/8330
> [2] https://issues.apache.org/jira/browse/IGNITE-13553
>
> ==================
>
> I propose to include several minor tickets to the scope of the 2.9 that
> increase Ignite User Experience
>
> CMD tools improvements:
>
> IGNITE-13488 - Command to print metric value
> IGNITE-13426 - Command to print system view content
> IGNITE-13422 - Parameter to explicitly enable experimental commands
>
> IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
>
> New system views:
>
> IGNITE-13409 Metastorage and DistributedMetastorage viewы.
> IGNITE-13408 BinaryMetadata view.
>
> > 9 окт. 2020 г., в 04:04, Denis Magda <dm...@apache.org> написал(а):
> >
> > @Alex Plehanov <pl...@gmail.com>,
> >
> > If it still makes sense and not too late, you can cherry-pick the commit
> > with the new docs into the 2.9 branch:
> >
> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
> >
> > Anyway, it's not crucial as long as we agreed to make an exception for
> this
> > release. The docs were already published to the website and assembled
> from
> > the master. Once the vote passes, I'll make them visible and traceable
> from
> > the website menus.
> >
> > -
> > Denis
> >
> >
> > On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> > wrote:
> >
> >> Saikat,
> >>
> >> The plan sounds good to me! Do you have an idea for the timeline of the
> >> module releases? Let me know if I can help in any way.
> >>
> >> Also, I was looking into the modularization IEP and noticed that OSGI
> >> module is discussed to be moved to the extensions, but there is no
> >> corresponding ticket. Should I create one? I will be happy to help with
> >> moving this module to extensions.
> >>
> >> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
> >>
> >>> Hi Alexey,
> >>>
> >>> All the modules as planned in IEP-36
> >>>
> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> >>> are migrated to ignite-extensions repository.
> >>>
> >>> We can definitely plan to release ignite-extensions modules.
> >>>
> >>> I have a pending PR related to the kafka module and the PR is in review
> >>> https://github.com/apache/ignite/pull/8222 but its corresponding PR
> has
> >>> been merged in ignite-extensions repository.
> >>>
> >>> My thoughts / action items for the migration efforts and releases were
> >>> as follows:
> >>>
> >>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
> >>> 2. Fix the upload nightly packages task for ignite-core to help fix the
> >>> travis build failure in ignite-extensions repository.
> >>> 3. Review and update the README.md files for the ignite-extensions
> >> modules
> >>> as required.
> >>> 4. Update the docs for ignite-extensions modules in ignite-website.
> >>> 5. Release each module separately and share updates.
> >>>
> >>> Please let me know if you have feedback.
> >>>
> >>> Regards,
> >>> Saikat
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
> wrote:
> >>>
> >>>> It seems that gitbox.apache.org is not available on CI/CD.
> >>>>
> >>>> I will try to update VCS to GitHub.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
> >>>> wrote:
> >>>>>
> >>>>> Guys,
> >>>>>
> >>>>> I've tried to build release candidate using TC [1]. But:
> >>>>> 1. I can't choose a branch in this TC task (there is no such field).
> >>>>> 2. On the "default" branch I got an error: Failed to collect changes,
> >>>>> error: List remote refs failed:
> >>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
> >>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> >> connect
> >>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> >>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
> >>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
> >>>>>
> >>>>> Is there some problem with this TC task? What am I doing wrong?
> >>>>>
> >>>>> [1] :
> >>>>>
> >>>>
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >>>>>
> >>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> >>>> alexey.goncharuk@gmail.com>:
> >>>>>
> >>>>>> Saikat, Nikolay,
> >>>>>>
> >>>>>> We have migrated a bunch of modules to ignite-extensions recently.
> >>>> Given
> >>>>>> that these modules will not be available in Ignite 2.9 anymore (will
> >>>>>> they?), should we also plan to release the extensions?
> >>>>>>
> >>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
> plehanov.alex@gmail.com
> >>> :
> >>>>>>
> >>>>>>> Igniters,
> >>>>>>>
> >>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
> >> review
> >>>>>> it,
> >>>>>>> please?
> >>>>>>>
> >>>>>>> [1]: https://github.com/apache/ignite/pull/8273
> >>>>>>>
> >>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> >> plehanov.alex@gmail.com
> >>>>> :
> >>>>>>>
> >>>>>>>> Guys,
> >>>>>>>>
> >>>>>>>> I've filled the ticket with reproducer [1] for the discovery bug.
> >>>> This
> >>>>>>> bug
> >>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
> >>>> and
> >>>>>>>> decided to revert this ticket. I will do it today (after TC bot
> >> visa)
> >>>>>> if
> >>>>>>>> there are no objections.
> >>>>>>>>
> >>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >>>>>>>>
> >>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> >> plehanov.alex@gmail.com
> >>>>> :
> >>>>>>>>
> >>>>>>>>> Guys,
> >>>>>>>>>
> >>>>>>>>> During internal testing, we've found a critical bug with
> >>>>>>>>> discovery (cluster falls apart if two nodes segmented
> >> sequentially).
> >>>>>>> This
> >>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >>>>>>>>> before release. Under investigation now. I'll let you know when
> we
> >>>> get
> >>>>>>>>> something.
> >>>>>>>>>
> >>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >>>>>>>>>
> >>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
> >> of
> >>>>>>> the
> >>>>>>>>>> message factory in 2.9 and go for the runtime message generation
> >> in
> >>>>>>> later
> >>>>>>>>>> release, or keep the code clean and fix the regression in the
> >> next
> >>>>>>> releases?
> >>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >>>>>>>>>> straightforward and does not change the semantics, just skips
> the
> >>>>>>> factory
> >>>>>>>>>> closures for certain messages.
> >>>>>>>>>>
> >>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
> >> all
> >>>>>>>>>> workloads (I didn't get any significant drops during
> >> benchmarking).
> >>>>>> So
> >>>>>>>>>> I prefer the runtime generation in later releases.
> >>>>>>>>>>
> >>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >>>>>>>>>> <al...@gmail.com> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> Alexey, Andrey, Igniters,
> >>>>>>>>>>>
> >>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
> >> of
> >>>>>>> the
> >>>>>>>>>> message factory in 2.9 and go for the runtime message generation
> >> in
> >>>>>>> later
> >>>>>>>>>> release, or keep the code clean and fix the regression in the
> >> next
> >>>>>>> releases?
> >>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >>>>>>>>>> straightforward and does not change the semantics, just skips
> the
> >>>>>>> factory
> >>>>>>>>>> closures for certain messages.
> >>>>>>>>>>>
> >>>>>>>>>>> Personally, I would prefer fixing the regression given that we
> >>>> also
> >>>>>>>>>> introduced tracing in this release.
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >>>>>> plehanov.alex@gmail.com
> >>>>>>>> :
> >>>>>>>>>>>>
> >>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>
> >>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> >> ed52559eb95
> >>>>>>> and
> >>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
> >> average
> >>>> on
> >>>>>>> our
> >>>>>>>>>> environment (about the same results we got benchmarking 65c30ec6
> >> vs
> >>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
> >>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
> this
> >>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> >> servers, 5
> >>>>>>> clients
> >>>>>>>>>> 50 threads each. Yardstick results for this configuration:
> >>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>>>>>>>>>>>
> >>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >>>>>>>>>> a.budnikov.ignite@gmail.com>:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hi Everyone,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> I posted an instruction on how to publish the docs on
> >>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
> you
> >>>> can
> >>>>>>>>>> update the docs by following the instruction. Unfortunately, I
> >>>> won't
> >>>>>> be
> >>>>>>>>>> able to spend any time on this project any longer. You can send
> >>>> your
> >>>>>>> pull
> >>>>>>>>>> requests and questions about the documentation to Denis Magda.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> -Artem
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> [1] :
> >>>>>>>>>>
> >> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I've tried to play with message factories locally, but
> >>>>>>>>>> unfortunately, I
> >>>>>>>>>>>>>> cannot spot the difference between old and new
> implementation
> >>>> in
> >>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >>>>>>>>>> MessageFactoryImpl
> >>>>>>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
> >>>>>>> branch
> >>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >>>>>> compatible
> >>>>>>>>>> with the
> >>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Can you check if this change makes any difference
> >>>>>> performance-wise
> >>>>>>>>>> in your
> >>>>>>>>>>>>>> environment? If yes, we can go with runtime code generation
> >> in
> >>>>>> the
> >>>>>>>>>> long
> >>>>>>>>>>>>>> term: register classes and generate a dynamic message
> factory
> >>>>>> with
> >>>>>>>>>> a switch
> >>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
> >> though,
> >>>>>>>>>> obviously).
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>> :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Hello guys,
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
> >> it
> >>>>>>>>>> reduced the
> >>>>>>>>>>>>>>> drop, but not completely removed it.
> >>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> >> patch?
> >>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> >> against
> >>>>>>>>>> 2.8.1
> >>>>>>>>>>>>>>> release on your environment?
> >>>>>>>>>>>>>>> With this patch on our environment, it's about a 3% drop
> >> left,
> >>>>>>>>>> it's close
> >>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
> >>>>>>>>>> showstopper. Guys,
> >>>>>>>>>>>>>>> WDYT?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
> >>>>>> driver
> >>>>>>>>>> between 2.8
> >>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
> >>>>>> should
> >>>>>>>>>> be
> >>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
> >>>>>>> message
> >>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
> >> if
> >>>>>>>>>> there is no
> >>>>>>>>>>>>>>> objection.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >>>>>> mmuzaf@apache.org
> >>>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
> >>>>>> this
> >>>>>>>>>> issue is
> >>>>>>>>>>>>>>>> related to the new master key change functionality which
> >>>>>>>>>> haven't been
> >>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
> >>>>>> to
> >>>>>>>>>> the
> >>>>>>>>>>>>>>>> release branch.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >>>>>>>>>> nizhikov@apache.org>
> >>>>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Hello, Igniters.
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
> >>>>>> [1]
> >>>>>>>>>> into the
> >>>>>>>>>>>>>>> 2.9
> >>>>>>>>>>>>>>>> release
> >>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
> fields
> >>>>>>> in
> >>>>>>>>>> wrong
> >>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>>>>>>>>>>>>>>>> [2]
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>>>>>> написал(а):
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
> >>>>>>>>>> anything out of
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
> >>>>>>>>>> right now),
> >>>>>>>>>>>>>>> so
> >>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to the
> >>>>>>>>>> switch
> >>>>>>>>>>>>>>> approach
> >>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
> >>>>>>> the
> >>>>>>>>>> Ignite
> >>>>>>>>>>>>>>>> messages
> >>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
> >>>>>>> me
> >>>>>>>>>> explore
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> code a bit.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
> >>>>>> the
> >>>>>>>>>>>>>>> performance.
> >>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
> >>>>>>>>>> virtual call
> >>>>>>>>>>>>>>>>>> should not make that much of a difference given the
> >>>>>> amount
> >>>>>>>>>> of other
> >>>>>>>>>>>>>>>> work
> >>>>>>>>>>>>>>>>>> happening during the message processing.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >>>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >>>>>>> results.
> >>>>>>>>>>>>>>> Actually,
> >>>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >>>>>>> between
> >>>>>>>>>> e6a7f93
> >>>>>>>>>>>>>>>> (first
> >>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >>>>>>>>>> 6592dfa5 (last
> >>>>>>>>>>>>>>>> commit in
> >>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> >>>>>>>>>> commits.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
> >>>>>> a
> >>>>>>>>>> drop
> >>>>>>>>>>>>>>> between
> >>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >>>>>>>>>> reverted
> >>>>>>>>>>>>>>>> IGNITE-13060
> >>>>>>>>>>>>>>>>>>> now and performance looks the same)
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >>>>>>>>>> related to
> >>>>>>>>>>>>>>>> drop
> >>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
> >>>>>>>>>> problem, and
> >>>>>>>>>>>>>>> we
> >>>>>>>>>>>>>>>> can
> >>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> >>>>>> leave
> >>>>>>>>>> it as is?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> >>>>>>>>>> refactoring)?
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
> >>>>>>>>>> incorrect fix
> >>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
> >>>>>>>>>> [1], so it
> >>>>>>>>>>>>>>>> cannot be
> >>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> >>>>>> work
> >>>>>>>>>> with fix
> >>>>>>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >>>>>>>>>> versions.
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> --AG
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>>>>>>>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >>>>>>>>>> IGNITE-12568
> >>>>>>>>>>>>>>>> (reverted
> >>>>>>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
> >>>>>>>>>> paths, to
> >>>>>>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
> >>>>>>> here.
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >>>>>>> switch/case
> >>>>>>>>>> block was
> >>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> >>>>>>>>>> message factory
> >>>>>>>>>>>>>>>> is on
> >>>>>>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >>>>>> impact
> >>>>>>>>>> on total
> >>>>>>>>>>>>>>>>>>>>>> performance.
> >>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >>>>>>>>>> microbenchmarks,
> >>>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>>>>>> found
> >>>>>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
> >>>>>>>>>> about 30-35%
> >>>>>>>>>>>>>>>> faster
> >>>>>>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
> >>>>>>> can
> >>>>>>>>>>>>>>> effectively
> >>>>>>>>>>>>>>>>>>>>>> inline
> >>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >>>>>> suppliers
> >>>>>>>>>> relatively
> >>>>>>>>>>>>>>>> heavy
> >>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
> >>>>>>>>>> rewrite the
> >>>>>>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>>>>>> using
> >>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >>>>>>> interface
> >>>>>>>>>> (to
> >>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
> >>>>>>>>>> but it gives
> >>>>>>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> >>>>>> looks
> >>>>>>>>>> ugly
> >>>>>>>>>>>>>>>> (lambdas
> >>>>>>>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
> >>>>>>>>>> optimize the
> >>>>>>>>>>>>>>>> current
> >>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
> >>>>>>>>>> Andrey Gura,
> >>>>>>>>>>>>>>> as
> >>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
> >>>>>>> about
> >>>>>>>>>>>>>>>> optimization?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
> >>>>>>>>>> some metrics
> >>>>>>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
> >>>>>>>>>> factory
> >>>>>>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >>>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> >>>>>>>>>> already released
> >>>>>>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> >>>>>>>>>> only present
> >>>>>>>>>>>>>>>> in Ignite
> >>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> >>>>>>>>>> metrics
> >>>>>>>>>>>>>>>> created for
> >>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
> >>>>>>>>>> the release
> >>>>>>>>>>>>>>>> and deal
> >>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >>>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>
> >>>>>>
> >>>>
> >>>>
> >>
>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
I prepared a patch [1] for blocker ticket [2] - «Server node fail and stops in case wrong datatype put in indexed field»
Can someone, please, help me with the review?
[1] https://github.com/apache/ignite/pull/8330
[2] https://issues.apache.org/jira/browse/IGNITE-13553
==================
I propose to include several minor tickets to the scope of the 2.9 that increase Ignite User Experience
CMD tools improvements:
IGNITE-13488 - Command to print metric value
IGNITE-13426 - Command to print system view content
IGNITE-13422 - Parameter to explicitly enable experimental commands
IGNITE-13380 - Output IgniteSystemProperties via ignite.sh
New system views:
IGNITE-13409 Metastorage and DistributedMetastorage viewы.
IGNITE-13408 BinaryMetadata view.
> 9 окт. 2020 г., в 04:04, Denis Magda <dm...@apache.org> написал(а):
>
> @Alex Plehanov <pl...@gmail.com>,
>
> If it still makes sense and not too late, you can cherry-pick the commit
> with the new docs into the 2.9 branch:
> https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
>
> Anyway, it's not crucial as long as we agreed to make an exception for this
> release. The docs were already published to the website and assembled from
> the master. Once the vote passes, I'll make them visible and traceable from
> the website menus.
>
> -
> Denis
>
>
> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <al...@gmail.com>
> wrote:
>
>> Saikat,
>>
>> The plan sounds good to me! Do you have an idea for the timeline of the
>> module releases? Let me know if I can help in any way.
>>
>> Also, I was looking into the modularization IEP and noticed that OSGI
>> module is discussed to be moved to the extensions, but there is no
>> corresponding ticket. Should I create one? I will be happy to help with
>> moving this module to extensions.
>>
>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>
>>> Hi Alexey,
>>>
>>> All the modules as planned in IEP-36
>>>
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>>> are migrated to ignite-extensions repository.
>>>
>>> We can definitely plan to release ignite-extensions modules.
>>>
>>> I have a pending PR related to the kafka module and the PR is in review
>>> https://github.com/apache/ignite/pull/8222 but its corresponding PR has
>>> been merged in ignite-extensions repository.
>>>
>>> My thoughts / action items for the migration efforts and releases were
>>> as follows:
>>>
>>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>>> 2. Fix the upload nightly packages task for ignite-core to help fix the
>>> travis build failure in ignite-extensions repository.
>>> 3. Review and update the README.md files for the ignite-extensions
>> modules
>>> as required.
>>> 4. Update the docs for ignite-extensions modules in ignite-website.
>>> 5. Release each module separately and share updates.
>>>
>>> Please let me know if you have feedback.
>>>
>>> Regards,
>>> Saikat
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
>>>
>>>> It seems that gitbox.apache.org is not available on CI/CD.
>>>>
>>>> I will try to update VCS to GitHub.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>> On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>>>> wrote:
>>>>>
>>>>> Guys,
>>>>>
>>>>> I've tried to build release candidate using TC [1]. But:
>>>>> 1. I can't choose a branch in this TC task (there is no such field).
>>>>> 2. On the "default" branch I got an error: Failed to collect changes,
>>>>> error: List remote refs failed:
>>>>> org.apache.http.conn.ConnectTimeoutException: Connect to
>>>>> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>> connect
>>>>> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>>>>> internal id=74, parent id=GitBoxAsfIgnite, description: "
>>>>> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>>>>>
>>>>> Is there some problem with this TC task? What am I doing wrong?
>>>>>
>>>>> [1] :
>>>>>
>>>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>>>>>
>>>>> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>>>> alexey.goncharuk@gmail.com>:
>>>>>
>>>>>> Saikat, Nikolay,
>>>>>>
>>>>>> We have migrated a bunch of modules to ignite-extensions recently.
>>>> Given
>>>>>> that these modules will not be available in Ignite 2.9 anymore (will
>>>>>> they?), should we also plan to release the extensions?
>>>>>>
>>>>>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <plehanov.alex@gmail.com
>>> :
>>>>>>
>>>>>>> Igniters,
>>>>>>>
>>>>>>> I've prepared release notes for the 2.9 release [1]. Can anyone
>> review
>>>>>> it,
>>>>>>> please?
>>>>>>>
>>>>>>> [1]: https://github.com/apache/ignite/pull/8273
>>>>>>>
>>>>>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>> plehanov.alex@gmail.com
>>>>> :
>>>>>>>
>>>>>>>> Guys,
>>>>>>>>
>>>>>>>> I've filled the ticket with reproducer [1] for the discovery bug.
>>>> This
>>>>>>> bug
>>>>>>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>>>> and
>>>>>>>> decided to revert this ticket. I will do it today (after TC bot
>> visa)
>>>>>> if
>>>>>>>> there are no objections.
>>>>>>>>
>>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>>>>>>>
>>>>>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>> plehanov.alex@gmail.com
>>>>> :
>>>>>>>>
>>>>>>>>> Guys,
>>>>>>>>>
>>>>>>>>> During internal testing, we've found a critical bug with
>>>>>>>>> discovery (cluster falls apart if two nodes segmented
>> sequentially).
>>>>>>> This
>>>>>>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>>>>>>>> before release. Under investigation now. I'll let you know when we
>>>> get
>>>>>>>>> something.
>>>>>>>>>
>>>>>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>>>>>>>>
>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
>> of
>>>>>>> the
>>>>>>>>>> message factory in 2.9 and go for the runtime message generation
>> in
>>>>>>> later
>>>>>>>>>> release, or keep the code clean and fix the regression in the
>> next
>>>>>>> releases?
>>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>>>>>> straightforward and does not change the semantics, just skips the
>>>>>>> factory
>>>>>>>>>> closures for certain messages.
>>>>>>>>>>
>>>>>>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
>> all
>>>>>>>>>> workloads (I didn't get any significant drops during
>> benchmarking).
>>>>>> So
>>>>>>>>>> I prefer the runtime generation in later releases.
>>>>>>>>>>
>>>>>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>>>>>>>>> <al...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>> Alexey, Andrey, Igniters,
>>>>>>>>>>>
>>>>>>>>>>> So what do you think? Should we proceed with a 'hacked' version
>> of
>>>>>>> the
>>>>>>>>>> message factory in 2.9 and go for the runtime message generation
>> in
>>>>>>> later
>>>>>>>>>> release, or keep the code clean and fix the regression in the
>> next
>>>>>>> releases?
>>>>>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>>>>>> straightforward and does not change the semantics, just skips the
>>>>>>> factory
>>>>>>>>>> closures for certain messages.
>>>>>>>>>>>
>>>>>>>>>>> Personally, I would prefer fixing the regression given that we
>>>> also
>>>>>>>>>> introduced tracing in this release.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>>>>>> plehanov.alex@gmail.com
>>>>>>>> :
>>>>>>>>>>>>
>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>
>>>>>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>> ed52559eb95
>>>>>>> and
>>>>>>>>>> the performance of ed52559eb95 is better for about 2.5% on
>> average
>>>> on
>>>>>>> our
>>>>>>>>>> environment (about the same results we got benchmarking 65c30ec6
>> vs
>>>>>>>>>> 0606f03d). We've made 24 runs for each commit of
>>>>>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>>>>>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>> servers, 5
>>>>>>> clients
>>>>>>>>>> 50 threads each. Yardstick results for this configuration:
>>>>>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>>>>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>>>>>>>>>>>
>>>>>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>>>>>>>>> a.budnikov.ignite@gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Everyone,
>>>>>>>>>>>>>
>>>>>>>>>>>>> I posted an instruction on how to publish the docs on
>>>>>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
>>>> can
>>>>>>>>>> update the docs by following the instruction. Unfortunately, I
>>>> won't
>>>>>> be
>>>>>>>>>> able to spend any time on this project any longer. You can send
>>>> your
>>>>>>> pull
>>>>>>>>>> requests and questions about the documentation to Denis Magda.
>>>>>>>>>>>>>
>>>>>>>>>>>>> -Artem
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] :
>>>>>>>>>>
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>>>>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've tried to play with message factories locally, but
>>>>>>>>>> unfortunately, I
>>>>>>>>>>>>>> cannot spot the difference between old and new implementation
>>>> in
>>>>>>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>>>>>>>>> MessageFactoryImpl
>>>>>>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>>>>>>> branch
>>>>>>>>>>>>>> (discussed this with Andrey Gura, the change should be
>>>>>> compatible
>>>>>>>>>> with the
>>>>>>>>>>>>>> new metrics as we still use the register() mechanics).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can you check if this change makes any difference
>>>>>> performance-wise
>>>>>>>>>> in your
>>>>>>>>>>>>>> environment? If yes, we can go with runtime code generation
>> in
>>>>>> the
>>>>>>>>>> long
>>>>>>>>>>>>>> term: register classes and generate a dynamic message factory
>>>>>> with
>>>>>>>>>> a switch
>>>>>>>>>>>>>> statement once all messages are registered (not in 2.9
>> though,
>>>>>>>>>> obviously).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hello guys,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
>> it
>>>>>>>>>> reduced the
>>>>>>>>>>>>>>> drop, but not completely removed it.
>>>>>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>> patch?
>>>>>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>> against
>>>>>>>>>> 2.8.1
>>>>>>>>>>>>>>> release on your environment?
>>>>>>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>> left,
>>>>>>>>>> it's close
>>>>>>>>>>>>>>> to measurement error and I think such a drop is not a
>>>>>>>>>> showstopper. Guys,
>>>>>>>>>>>>>>> WDYT?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>>>>>> driver
>>>>>>>>>> between 2.8
>>>>>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>>>>>> should
>>>>>>>>>> be
>>>>>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>>>>>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>>>>>> message
>>>>>>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
>> if
>>>>>>>>>> there is no
>>>>>>>>>>>>>>> objection.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>>>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>>>>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>>>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>>>>>> mmuzaf@apache.org
>>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>>>>>> this
>>>>>>>>>> issue is
>>>>>>>>>>>>>>>> related to the new master key change functionality which
>>>>>>>>>> haven't been
>>>>>>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>>>>>> to
>>>>>>>>>> the
>>>>>>>>>>>>>>>> release branch.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>>>>>>>>> nizhikov@apache.org>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hello, Igniters.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>>>>>> [1]
>>>>>>>>>> into the
>>>>>>>>>>>>>>> 2.9
>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
>>>>>>> in
>>>>>>>>>> wrong
>>>>>>>>>>>>>>>> order since the 2 row when fields_count>10"
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>>>>>>>>>>>>>>> [2]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>>>>>> написал(а):
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>>>>>>>>> anything out of
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>>>>>>>>>> right now),
>>>>>>>>>>>>>>> so
>>>>>>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>>>>>>>>> switch
>>>>>>>>>>>>>>> approach
>>>>>>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>>>>>> the
>>>>>>>>>> Ignite
>>>>>>>>>>>>>>>> messages
>>>>>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>>>>>>> me
>>>>>>>>>> explore
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> code a bit.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>>>>>> the
>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>>>>>>>>>> virtual call
>>>>>>>>>>>>>>>>>> should not make that much of a difference given the
>>>>>> amount
>>>>>>>>>> of other
>>>>>>>>>>>>>>>> work
>>>>>>>>>>>>>>>>>> happening during the message processing.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>>>>>> results.
>>>>>>>>>>>>>>> Actually,
>>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>>>>>> between
>>>>>>>>>> e6a7f93
>>>>>>>>>>>>>>>> (first
>>>>>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>>>>>>>>> 6592dfa5 (last
>>>>>>>>>>>>>>>> commit in
>>>>>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>>>>>>>>> commits.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>>>>>> a
>>>>>>>>>> drop
>>>>>>>>>>>>>>> between
>>>>>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>>>>>>>>> reverted
>>>>>>>>>>>>>>>> IGNITE-13060
>>>>>>>>>>>>>>>>>>> now and performance looks the same)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>>>>>>>>> related to
>>>>>>>>>>>>>>>> drop
>>>>>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>>>>>>>>> problem, and
>>>>>>>>>>>>>>> we
>>>>>>>>>>>>>>>> can
>>>>>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>>>>>> leave
>>>>>>>>>> it as is?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>>>>>>>>> refactoring)?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>>>>>>>>> incorrect fix
>>>>>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>>>>>>>>>> [1], so it
>>>>>>>>>>>>>>>> cannot be
>>>>>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>>>>>> work
>>>>>>>>>> with fix
>>>>>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>>>>>>>>> versions.
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> --AG
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>>>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>>>>>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>>>>>>>> IGNITE-12568
>>>>>>>>>>>>>>>> (reverted
>>>>>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>>>>>>>>> paths, to
>>>>>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>>>>>> here.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>>>>>> switch/case
>>>>>>>>>> block was
>>>>>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>>>>>>>>> message factory
>>>>>>>>>>>>>>>> is on
>>>>>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>>>>>> impact
>>>>>>>>>> on total
>>>>>>>>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>>>>>>>>> microbenchmarks,
>>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>>>>>>>>> about 30-35%
>>>>>>>>>>>>>>>> faster
>>>>>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>>>>>> can
>>>>>>>>>>>>>>> effectively
>>>>>>>>>>>>>>>>>>>>>> inline
>>>>>>>>>>>>>>>>>>>>>> message creation code, but with an array of
>>>>>> suppliers
>>>>>>>>>> relatively
>>>>>>>>>>>>>>>> heavy
>>>>>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>>>>>>>>> rewrite the
>>>>>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>>>>>> interface
>>>>>>>>>> (to
>>>>>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>>>>>>>>>> but it gives
>>>>>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>>>>>> looks
>>>>>>>>>> ugly
>>>>>>>>>>>>>>>> (lambdas
>>>>>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>>>>>>>>> optimize the
>>>>>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>>>>>>>>> Andrey Gura,
>>>>>>>>>>>>>>> as
>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>>>>>> about
>>>>>>>>>>>>>>>> optimization?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>>>>>>>>>> some metrics
>>>>>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>>>>>>>>> factory
>>>>>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>>>>>>>>> already released
>>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>>>>>>>>> only present
>>>>>>>>>>>>>>>> in Ignite
>>>>>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>>>>>>>>> metrics
>>>>>>>>>>>>>>>> created for
>>>>>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>>>>>>>>>> the release
>>>>>>>>>>>>>>>> and deal
>>>>>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
@Alex Plehanov <pl...@gmail.com>,
If it still makes sense and not too late, you can cherry-pick the commit
with the new docs into the 2.9 branch:
https://github.com/apache/ignite/commit/073488ac97517bbaad9f6b94b781fc404646f191
Anyway, it's not crucial as long as we agreed to make an exception for this
release. The docs were already published to the website and assembled from
the master. Once the vote passes, I'll make them visible and traceable from
the website menus.
-
Denis
On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <al...@gmail.com>
wrote:
> Saikat,
>
> The plan sounds good to me! Do you have an idea for the timeline of the
> module releases? Let me know if I can help in any way.
>
> Also, I was looking into the modularization IEP and noticed that OSGI
> module is discussed to be moved to the extensions, but there is no
> corresponding ticket. Should I create one? I will be happy to help with
> moving this module to extensions.
>
> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>
> > Hi Alexey,
> >
> > All the modules as planned in IEP-36
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> > are migrated to ignite-extensions repository.
> >
> > We can definitely plan to release ignite-extensions modules.
> >
> > I have a pending PR related to the kafka module and the PR is in review
> > https://github.com/apache/ignite/pull/8222 but its corresponding PR has
> > been merged in ignite-extensions repository.
> >
> > My thoughts / action items for the migration efforts and releases were
> > as follows:
> >
> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
> > 2. Fix the upload nightly packages task for ignite-core to help fix the
> > travis build failure in ignite-extensions repository.
> > 3. Review and update the README.md files for the ignite-extensions
> modules
> > as required.
> > 4. Update the docs for ignite-extensions modules in ignite-website.
> > 5. Release each module separately and share updates.
> >
> > Please let me know if you have feedback.
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
> >
> >> It seems that gitbox.apache.org is not available on CI/CD.
> >>
> >> I will try to update VCS to GitHub.
> >>
> >>
> >>
> >>
> >>
> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
> >> wrote:
> >> >
> >> > Guys,
> >> >
> >> > I've tried to build release candidate using TC [1]. But:
> >> > 1. I can't choose a branch in this TC task (there is no such field).
> >> > 2. On the "default" branch I got an error: Failed to collect changes,
> >> > error: List remote refs failed:
> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> connect
> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
> >> >
> >> > Is there some problem with this TC task? What am I doing wrong?
> >> >
> >> > [1] :
> >> >
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >> >
> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com>:
> >> >
> >> >> Saikat, Nikolay,
> >> >>
> >> >> We have migrated a bunch of modules to ignite-extensions recently.
> >> Given
> >> >> that these modules will not be available in Ignite 2.9 anymore (will
> >> >> they?), should we also plan to release the extensions?
> >> >>
> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >> >>
> >> >>> Igniters,
> >> >>>
> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
> review
> >> >> it,
> >> >>> please?
> >> >>>
> >> >>> [1]: https://github.com/apache/ignite/pull/8273
> >> >>>
> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> plehanov.alex@gmail.com
> >> >:
> >> >>>
> >> >>>> Guys,
> >> >>>>
> >> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
> >> This
> >> >>> bug
> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
> >> and
> >> >>>> decided to revert this ticket. I will do it today (after TC bot
> visa)
> >> >> if
> >> >>>> there are no objections.
> >> >>>>
> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >> >>>>
> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> plehanov.alex@gmail.com
> >> >:
> >> >>>>
> >> >>>>> Guys,
> >> >>>>>
> >> >>>>> During internal testing, we've found a critical bug with
> >> >>>>> discovery (cluster falls apart if two nodes segmented
> sequentially).
> >> >>> This
> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >> >>>>> before release. Under investigation now. I'll let you know when we
> >> get
> >> >>>>> something.
> >> >>>>>
> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >> >>>>>
> >> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
> of
> >> >>> the
> >> >>>>>> message factory in 2.9 and go for the runtime message generation
> in
> >> >>> later
> >> >>>>>> release, or keep the code clean and fix the regression in the
> next
> >> >>> releases?
> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >> >>>>>> straightforward and does not change the semantics, just skips the
> >> >>> factory
> >> >>>>>> closures for certain messages.
> >> >>>>>>
> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
> all
> >> >>>>>> workloads (I didn't get any significant drops during
> benchmarking).
> >> >> So
> >> >>>>>> I prefer the runtime generation in later releases.
> >> >>>>>>
> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >> >>>>>> <al...@gmail.com> wrote:
> >> >>>>>>>
> >> >>>>>>> Alexey, Andrey, Igniters,
> >> >>>>>>>
> >> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
> of
> >> >>> the
> >> >>>>>> message factory in 2.9 and go for the runtime message generation
> in
> >> >>> later
> >> >>>>>> release, or keep the code clean and fix the regression in the
> next
> >> >>> releases?
> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >> >>>>>> straightforward and does not change the semantics, just skips the
> >> >>> factory
> >> >>>>>> closures for certain messages.
> >> >>>>>>>
> >> >>>>>>> Personally, I would prefer fixing the regression given that we
> >> also
> >> >>>>>> introduced tracing in this release.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >> >> plehanov.alex@gmail.com
> >> >>>> :
> >> >>>>>>>>
> >> >>>>>>>> Alexey,
> >> >>>>>>>>
> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> ed52559eb95
> >> >>> and
> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
> average
> >> on
> >> >>> our
> >> >>>>>> environment (about the same results we got benchmarking 65c30ec6
> vs
> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> servers, 5
> >> >>> clients
> >> >>>>>> 50 threads each. Yardstick results for this configuration:
> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >> >>>>>>>>
> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >> >>>>>> a.budnikov.ignite@gmail.com>:
> >> >>>>>>>>>
> >> >>>>>>>>> Hi Everyone,
> >> >>>>>>>>>
> >> >>>>>>>>> I posted an instruction on how to publish the docs on
> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
> >> can
> >> >>>>>> update the docs by following the instruction. Unfortunately, I
> >> won't
> >> >> be
> >> >>>>>> able to spend any time on this project any longer. You can send
> >> your
> >> >>> pull
> >> >>>>>> requests and questions about the documentation to Denis Magda.
> >> >>>>>>>>>
> >> >>>>>>>>> -Artem
> >> >>>>>>>>>
> >> >>>>>>>>> [1] :
> >> >>>>>>
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >> >>>>>>>>>
> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>> Alexey,
> >> >>>>>>>>>>
> >> >>>>>>>>>> I've tried to play with message factories locally, but
> >> >>>>>> unfortunately, I
> >> >>>>>>>>>> cannot spot the difference between old and new implementation
> >> in
> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >> >>>>>> MessageFactoryImpl
> >> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
> >> >>> branch
> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >> >> compatible
> >> >>>>>> with the
> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
> >> >>>>>>>>>>
> >> >>>>>>>>>> Can you check if this change makes any difference
> >> >> performance-wise
> >> >>>>>> in your
> >> >>>>>>>>>> environment? If yes, we can go with runtime code generation
> in
> >> >> the
> >> >>>>>> long
> >> >>>>>>>>>> term: register classes and generate a dynamic message factory
> >> >> with
> >> >>>>>> a switch
> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
> though,
> >> >>>>>> obviously).
> >> >>>>>>>>>>
> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >> >>> plehanov.alex@gmail.com
> >> >>>>>>> :
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Hello guys,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
> it
> >> >>>>>> reduced the
> >> >>>>>>>>>>> drop, but not completely removed it.
> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> patch?
> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> against
> >> >>>>>> 2.8.1
> >> >>>>>>>>>>> release on your environment?
> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
> left,
> >> >>>>>> it's close
> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
> >> >>>>>> showstopper. Guys,
> >> >>>>>>>>>>> WDYT?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
> >> >> driver
> >> >>>>>> between 2.8
> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
> >> >> should
> >> >>>>>> be
> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
> >> >>> message
> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
> if
> >> >>>>>> there is no
> >> >>>>>>>>>>> objection.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >> >> mmuzaf@apache.org
> >> >>>> :
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
> >> >> this
> >> >>>>>> issue is
> >> >>>>>>>>>>>> related to the new master key change functionality which
> >> >>>>>> haven't been
> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
> >> >> to
> >> >>>>>> the
> >> >>>>>>>>>>>> release branch.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >> >>>>>> nizhikov@apache.org>
> >> >>>>>>>>>>> wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Hello, Igniters.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
> >> >> [1]
> >> >>>>>> into the
> >> >>>>>>>>>>> 2.9
> >> >>>>>>>>>>>> release
> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
> >> >>> in
> >> >>>>>> wrong
> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >> >>>>>>>>>>>>> [2]
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
> >> >>>>>>>>>>>> написал(а):
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
> >> >>>>>> anything out of
> >> >>>>>>>>>>>> the
> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
> >> >>>>>> right now),
> >> >>>>>>>>>>> so
> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
> >> >>>>>> switch
> >> >>>>>>>>>>> approach
> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
> >> >>> the
> >> >>>>>> Ignite
> >> >>>>>>>>>>>> messages
> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
> >> >>> me
> >> >>>>>> explore
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>>>>> code a bit.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
> >> >> the
> >> >>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
> >> >>>>>> virtual call
> >> >>>>>>>>>>>>>> should not make that much of a difference given the
> >> >> amount
> >> >>>>>> of other
> >> >>>>>>>>>>>> work
> >> >>>>>>>>>>>>>> happening during the message processing.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >> >>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >> >>> results.
> >> >>>>>>>>>>> Actually,
> >> >>>>>>>>>>>> we
> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >> >>> between
> >> >>>>>> e6a7f93
> >> >>>>>>>>>>>> (first
> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >> >>>>>> 6592dfa5 (last
> >> >>>>>>>>>>>> commit in
> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> >> >>>>>> commits.
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
> >> >> a
> >> >>>>>> drop
> >> >>>>>>>>>>> between
> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >> >>>>>> reverted
> >> >>>>>>>>>>>> IGNITE-13060
> >> >>>>>>>>>>>>>>> now and performance looks the same)
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >> >>>>>> related to
> >> >>>>>>>>>>>> drop
> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
> >> >>>>>> problem, and
> >> >>>>>>>>>>> we
> >> >>>>>>>>>>>> can
> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> >> >> leave
> >> >>>>>> it as is?
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> >> >>>>>> refactoring)?
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
> >> >>>>>> incorrect fix
> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
> >> >>>>>> [1], so it
> >> >>>>>>>>>>>> cannot be
> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> >> >> work
> >> >>>>>> with fix
> >> >>>>>>>>>>>> versions
> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >> >>>>>> versions.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> --AG
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> [1]
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >> >>>>>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> Guys,
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >> >>>>>> IGNITE-12568
> >> >>>>>>>>>>>> (reverted
> >> >>>>>>>>>>>>>>>>>> it
> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
> >> >>>>>> paths, to
> >> >>>>>>>>>>> trace
> >> >>>>>>>>>>>>>>>>>> these
> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
> >> >>> here.
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >> >>> switch/case
> >> >>>>>> block was
> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> >> >>>>>> message factory
> >> >>>>>>>>>>>> is on
> >> >>>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >> >> impact
> >> >>>>>> on total
> >> >>>>>>>>>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >> >>>>>> microbenchmarks,
> >> >>>>>>>>>>>> and
> >> >>>>>>>>>>>>>>>>>> found
> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
> >> >>>>>> about 30-35%
> >> >>>>>>>>>>>> faster
> >> >>>>>>>>>>>>>>>>>> than
> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
> >> >>> can
> >> >>>>>>>>>>> effectively
> >> >>>>>>>>>>>>>>>>>> inline
> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >> >> suppliers
> >> >>>>>> relatively
> >> >>>>>>>>>>>> heavy
> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
> >> >>>>>> rewrite the
> >> >>>>>>>>>>> code
> >> >>>>>>>>>>>>>>>>>> using
> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >> >>> interface
> >> >>>>>> (to
> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
> >> >>>>>> but it gives
> >> >>>>>>>>>>>> back
> >> >>>>>>>>>>>>>>>>>> only
> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> >> >> looks
> >> >>>>>> ugly
> >> >>>>>>>>>>>> (lambdas
> >> >>>>>>>>>>>>>>>>>> can't
> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
> >> >>>>>> optimize the
> >> >>>>>>>>>>>> current
> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
> >> >>>>>> Andrey Gura,
> >> >>>>>>>>>>> as
> >> >>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
> >> >>> about
> >> >>>>>>>>>>>> optimization?
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
> >> >>>>>> some metrics
> >> >>>>>>>>>>>>>>>>>> already
> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
> >> >>>>>> factory
> >> >>>>>>>>>>>>>>>>>> implementation
> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> >> >>>>>> already released
> >> >>>>>>>>>>>> in
> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> >> >>>>>> only present
> >> >>>>>>>>>>>> in Ignite
> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> >> >>>>>> metrics
> >> >>>>>>>>>>>> created for
> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
> >> >>>>>> the release
> >> >>>>>>>>>>>> and deal
> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>
> >> >>
> >>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
@Alex Plehanov <pl...@gmail.com>,
I crossed off the step 6.3.4 (technical docs publication)
<https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-6.3.4.Publishtechnicaldocumentation>
of
the release process by adding the new docs to the website's menu. You can
find them by going to the "Resources"->"Documentation" menu item. I will be
unreachable early next week, but it feels like the 2.9 vote will pass;
thus, decided to proceed with this step.
Just in case, here is the documentation contribution and publication
process. Use it if you need to tweak anything:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document#HowtoDocument-PublishingtotheWebsite
-
Denis
On Mon, Oct 12, 2020 at 12:57 PM Denis Magda <dm...@apache.org> wrote:
> Saikat, Alex,
>
> Based on your inputs here, it sounds like once Ignite 2.9 gets released,
> all the integrations that made their way to the extensions repo [1] will
> get stuck in limbo for some time. What I mean here:
>
> 1. While the users will be bumping up their ignite-core,
> ignite-indexing, etc. versions to 2.9, they won't find a 2.9 artifact for
> Kafka, Camel and other extensions
> 2. Also, the users won't find 1.0 versions of those extensions that
> also need to be released
>
> So, it's unclear how those users are supposed to upgrade to Ignite 2.9 if
> they use any of the extensions. Is it safe to use a 2.8 version of an
> extension? Or, should we release the 1.0 versions of all the migrated
> extensions together with 2.9?
>
>
> [1] https://github.com/apache/ignite-extensions/tree/master/modules
>
>
> -
> Denis
>
>
> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Saikat,
>>
>> The plan sounds good to me! Do you have an idea for the timeline of the
>> module releases? Let me know if I can help in any way.
>>
>> Also, I was looking into the modularization IEP and noticed that OSGI
>> module is discussed to be moved to the extensions, but there is no
>> corresponding ticket. Should I create one? I will be happy to help with
>> moving this module to extensions.
>>
>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>
>> > Hi Alexey,
>> >
>> > All the modules as planned in IEP-36
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>> > are migrated to ignite-extensions repository.
>> >
>> > We can definitely plan to release ignite-extensions modules.
>> >
>> > I have a pending PR related to the kafka module and the PR is in review
>> > https://github.com/apache/ignite/pull/8222 but its corresponding PR has
>> > been merged in ignite-extensions repository.
>> >
>> > My thoughts / action items for the migration efforts and releases were
>> > as follows:
>> >
>> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>> > 2. Fix the upload nightly packages task for ignite-core to help fix the
>> > travis build failure in ignite-extensions repository.
>> > 3. Review and update the README.md files for the ignite-extensions
>> modules
>> > as required.
>> > 4. Update the docs for ignite-extensions modules in ignite-website.
>> > 5. Release each module separately and share updates.
>> >
>> > Please let me know if you have feedback.
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
>> wrote:
>> >
>> >> It seems that gitbox.apache.org is not available on CI/CD.
>> >>
>> >> I will try to update VCS to GitHub.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>> >> wrote:
>> >> >
>> >> > Guys,
>> >> >
>> >> > I've tried to build release candidate using TC [1]. But:
>> >> > 1. I can't choose a branch in this TC task (there is no such field).
>> >> > 2. On the "default" branch I got an error: Failed to collect changes,
>> >> > error: List remote refs failed:
>> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
>> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>> connect
>> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>> >> >
>> >> > Is there some problem with this TC task? What am I doing wrong?
>> >> >
>> >> > [1] :
>> >> >
>> >>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>> >> >
>> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>> >> alexey.goncharuk@gmail.com>:
>> >> >
>> >> >> Saikat, Nikolay,
>> >> >>
>> >> >> We have migrated a bunch of modules to ignite-extensions recently.
>> >> Given
>> >> >> that these modules will not be available in Ignite 2.9 anymore (will
>> >> >> they?), should we also plan to release the extensions?
>> >> >>
>> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>> plehanov.alex@gmail.com>:
>> >> >>
>> >> >>> Igniters,
>> >> >>>
>> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
>> review
>> >> >> it,
>> >> >>> please?
>> >> >>>
>> >> >>> [1]: https://github.com/apache/ignite/pull/8273
>> >> >>>
>> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>> plehanov.alex@gmail.com
>> >> >:
>> >> >>>
>> >> >>>> Guys,
>> >> >>>>
>> >> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
>> >> This
>> >> >>> bug
>> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>> >> and
>> >> >>>> decided to revert this ticket. I will do it today (after TC bot
>> visa)
>> >> >> if
>> >> >>>> there are no objections.
>> >> >>>>
>> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>> >> >>>>
>> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>> plehanov.alex@gmail.com
>> >> >:
>> >> >>>>
>> >> >>>>> Guys,
>> >> >>>>>
>> >> >>>>> During internal testing, we've found a critical bug with
>> >> >>>>> discovery (cluster falls apart if two nodes segmented
>> sequentially).
>> >> >>> This
>> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>> >> >>>>> before release. Under investigation now. I'll let you know when
>> we
>> >> get
>> >> >>>>> something.
>> >> >>>>>
>> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>> >> >>>>>
>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version of
>> >> >>> the
>> >> >>>>>> message factory in 2.9 and go for the runtime message
>> generation in
>> >> >>> later
>> >> >>>>>> release, or keep the code clean and fix the regression in the
>> next
>> >> >>> releases?
>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >> >>>>>> straightforward and does not change the semantics, just skips
>> the
>> >> >>> factory
>> >> >>>>>> closures for certain messages.
>> >> >>>>>>
>> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
>> all
>> >> >>>>>> workloads (I didn't get any significant drops during
>> benchmarking).
>> >> >> So
>> >> >>>>>> I prefer the runtime generation in later releases.
>> >> >>>>>>
>> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>> >> >>>>>> <al...@gmail.com> wrote:
>> >> >>>>>>>
>> >> >>>>>>> Alexey, Andrey, Igniters,
>> >> >>>>>>>
>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version of
>> >> >>> the
>> >> >>>>>> message factory in 2.9 and go for the runtime message
>> generation in
>> >> >>> later
>> >> >>>>>> release, or keep the code clean and fix the regression in the
>> next
>> >> >>> releases?
>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >> >>>>>> straightforward and does not change the semantics, just skips
>> the
>> >> >>> factory
>> >> >>>>>> closures for certain messages.
>> >> >>>>>>>
>> >> >>>>>>> Personally, I would prefer fixing the regression given that we
>> >> also
>> >> >>>>>> introduced tracing in this release.
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> >> >> plehanov.alex@gmail.com
>> >> >>>> :
>> >> >>>>>>>>
>> >> >>>>>>>> Alexey,
>> >> >>>>>>>>
>> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>> ed52559eb95
>> >> >>> and
>> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
>> average
>> >> on
>> >> >>> our
>> >> >>>>>> environment (about the same results we got benchmarking
>> 65c30ec6 vs
>> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
>> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>> this
>> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>> servers, 5
>> >> >>> clients
>> >> >>>>>> 50 threads each. Yardstick results for this configuration:
>> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>> >> >>>>>>>>
>> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>> >> >>>>>> a.budnikov.ignite@gmail.com>:
>> >> >>>>>>>>>
>> >> >>>>>>>>> Hi Everyone,
>> >> >>>>>>>>>
>> >> >>>>>>>>> I posted an instruction on how to publish the docs on
>> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>> you
>> >> can
>> >> >>>>>> update the docs by following the instruction. Unfortunately, I
>> >> won't
>> >> >> be
>> >> >>>>>> able to spend any time on this project any longer. You can send
>> >> your
>> >> >>> pull
>> >> >>>>>> requests and questions about the documentation to Denis Magda.
>> >> >>>>>>>>>
>> >> >>>>>>>>> -Artem
>> >> >>>>>>>>>
>> >> >>>>>>>>> [1] :
>> >> >>>>>>
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Alexey,
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I've tried to play with message factories locally, but
>> >> >>>>>> unfortunately, I
>> >> >>>>>>>>>> cannot spot the difference between old and new
>> implementation
>> >> in
>> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>> >> >>>>>> MessageFactoryImpl
>> >> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>> >> >>> branch
>> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> >> >> compatible
>> >> >>>>>> with the
>> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Can you check if this change makes any difference
>> >> >> performance-wise
>> >> >>>>>> in your
>> >> >>>>>>>>>> environment? If yes, we can go with runtime code generation
>> in
>> >> >> the
>> >> >>>>>> long
>> >> >>>>>>>>>> term: register classes and generate a dynamic message
>> factory
>> >> >> with
>> >> >>>>>> a switch
>> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
>> though,
>> >> >>>>>> obviously).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>> >> >>> plehanov.alex@gmail.com
>> >> >>>>>>> :
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>> Hello guys,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket
>> [1]), it
>> >> >>>>>> reduced the
>> >> >>>>>>>>>>> drop, but not completely removed it.
>> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>> patch?
>> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>> against
>> >> >>>>>> 2.8.1
>> >> >>>>>>>>>>> release on your environment?
>> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>> left,
>> >> >>>>>> it's close
>> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
>> >> >>>>>> showstopper. Guys,
>> >> >>>>>>>>>>> WDYT?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> >> >> driver
>> >> >>>>>> between 2.8
>> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> >> >> should
>> >> >>>>>> be
>> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>> >> >>> message
>> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
>> if
>> >> >>>>>> there is no
>> >> >>>>>>>>>>> objection.
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> >> >> mmuzaf@apache.org
>> >> >>>> :
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> >> >> this
>> >> >>>>>> issue is
>> >> >>>>>>>>>>>> related to the new master key change functionality which
>> >> >>>>>> haven't been
>> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>> >> >> to
>> >> >>>>>> the
>> >> >>>>>>>>>>>> release branch.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>> >> >>>>>> nizhikov@apache.org>
>> >> >>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Hello, Igniters.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> >> >> [1]
>> >> >>>>>> into the
>> >> >>>>>>>>>>> 2.9
>> >> >>>>>>>>>>>> release
>> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>> fields
>> >> >>> in
>> >> >>>>>> wrong
>> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> >> >>>>>>>>>>>>> [2]
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>> >> >>>>>>>>>>>> написал(а):
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>> >> >>>>>> anything out of
>> >> >>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>> >> >>>>>> right now),
>> >> >>>>>>>>>>> so
>> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
>> >> >>>>>> switch
>> >> >>>>>>>>>>> approach
>> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>> >> >>> the
>> >> >>>>>> Ignite
>> >> >>>>>>>>>>>> messages
>> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>> >> >>> me
>> >> >>>>>> explore
>> >> >>>>>>>>>>> the
>> >> >>>>>>>>>>>>>> code a bit.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> >> >> the
>> >> >>>>>>>>>>> performance.
>> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>> >> >>>>>> virtual call
>> >> >>>>>>>>>>>>>> should not make that much of a difference given the
>> >> >> amount
>> >> >>>>>> of other
>> >> >>>>>>>>>>>> work
>> >> >>>>>>>>>>>>>> happening during the message processing.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> >> >>>>>> plehanov.alex@gmail.com
>> >> >>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>> >> >>> results.
>> >> >>>>>>>>>>> Actually,
>> >> >>>>>>>>>>>> we
>> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>> >> >>> between
>> >> >>>>>> e6a7f93
>> >> >>>>>>>>>>>> (first
>> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>> >> >>>>>> 6592dfa5 (last
>> >> >>>>>>>>>>>> commit in
>> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>> >> >>>>>> commits.
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>> >> >> a
>> >> >>>>>> drop
>> >> >>>>>>>>>>> between
>> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>> >> >>>>>> reverted
>> >> >>>>>>>>>>>> IGNITE-13060
>> >> >>>>>>>>>>>>>>> now and performance looks the same)
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>> >> >>>>>> related to
>> >> >>>>>>>>>>>> drop
>> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>> >> >>>>>> problem, and
>> >> >>>>>>>>>>> we
>> >> >>>>>>>>>>>> can
>> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> >> >> leave
>> >> >>>>>> it as is?
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>> >> >>>>>> refactoring)?
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >> >>>>>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>> >> >>>>>> incorrect fix
>> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>> >> >>>>>> [1], so it
>> >> >>>>>>>>>>>> cannot be
>> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> >> >> work
>> >> >>>>>> with fix
>> >> >>>>>>>>>>>> versions
>> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>> >> >>>>>> versions.
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> --AG
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> [1]
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >> >>>>>>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> >> >>>>>>>>>>> plehanov.alex@gmail.com
>> >> >>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> Guys,
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>> >> >>>>>> IGNITE-12568
>> >> >>>>>>>>>>>> (reverted
>> >> >>>>>>>>>>>>>>>>>> it
>> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>> >> >>>>>> paths, to
>> >> >>>>>>>>>>> trace
>> >> >>>>>>>>>>>>>>>>>> these
>> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>> >> >>> here.
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>> >> >>> switch/case
>> >> >>>>>> block was
>> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>> >> >>>>>> message factory
>> >> >>>>>>>>>>>> is on
>> >> >>>>>>>>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> >> >> impact
>> >> >>>>>> on total
>> >> >>>>>>>>>>>>>>>>>> performance.
>> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>> >> >>>>>> microbenchmarks,
>> >> >>>>>>>>>>>> and
>> >> >>>>>>>>>>>>>>>>>> found
>> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>> >> >>>>>> about 30-35%
>> >> >>>>>>>>>>>> faster
>> >> >>>>>>>>>>>>>>>>>> than
>> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>> >> >>> can
>> >> >>>>>>>>>>> effectively
>> >> >>>>>>>>>>>>>>>>>> inline
>> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> >> >> suppliers
>> >> >>>>>> relatively
>> >> >>>>>>>>>>>> heavy
>> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>> >> >>>>>> rewrite the
>> >> >>>>>>>>>>> code
>> >> >>>>>>>>>>>>>>>>>> using
>> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>> >> >>> interface
>> >> >>>>>> (to
>> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>> >> >>>>>> but it gives
>> >> >>>>>>>>>>>> back
>> >> >>>>>>>>>>>>>>>>>> only
>> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> >> >> looks
>> >> >>>>>> ugly
>> >> >>>>>>>>>>>> (lambdas
>> >> >>>>>>>>>>>>>>>>>> can't
>> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>> >> >>>>>> optimize the
>> >> >>>>>>>>>>>> current
>> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>> >> >>>>>> Andrey Gura,
>> >> >>>>>>>>>>> as
>> >> >>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>> >> >>> about
>> >> >>>>>>>>>>>> optimization?
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>> >> >>>>>> some metrics
>> >> >>>>>>>>>>>>>>>>>> already
>> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>> >> >>>>>> factory
>> >> >>>>>>>>>>>>>>>>>> implementation
>> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>> >> >>>>>> already released
>> >> >>>>>>>>>>>> in
>> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>> >> >>>>>> only present
>> >> >>>>>>>>>>>> in Ignite
>> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>> >> >>>>>> metrics
>> >> >>>>>>>>>>>> created for
>> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>> >> >>>>>> the release
>> >> >>>>>>>>>>>> and deal
>> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >>>
>> >> >>
>> >>
>> >>
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've cherry-picked IGNITE-12350, IGNITE-13376, IGNITE-13280, and
documentation to 2.9.
Denis,
I see no benefits of a single vote for Ignite and extensions. I think it
will postpone Ignite 2.9 release. If some user uses some extension, he
still can continue to work with Ignite 2.8 release until ignite-extensions
is released. But those who don't use any extensions can gain benefits of
2.9 earlier.
Also, as far as I understand, there were no changes in extensions since
2.8. Only migration to the new repository. So it's possible to use Ignite
2.8 extensions with Ignite 2.9 core.
вт, 13 окт. 2020 г. в 07:34, Denis Magda <dm...@apache.org>:
> Is it possible/reasonable to run a single vote for Ignite 2.9 and all the
> migrated extensions? This time. @Alex Plehanov <pl...@gmail.com>,
> what do you think?
>
> If we decide to release the versions 1.0.0 of the extensions after 2.9 is
> out then it can last for a week or so...because we need to build, update
> docs, vote, publish.
>
> -
> Denis
>
>
> On Mon, Oct 12, 2020 at 7:19 PM Saikat Maitra <sa...@gmail.com>
> wrote:
>
>> Hi Denis,
>>
>> All the modules except OSGi have been migrated. Alexey is creating a
>> separate ticket for OSGi and we can migrate the same as well.
>>
>> My thoughts are Apache Ignite 2.9.0 and migrated extensions 1.0.0 to be
>> released together. I am thinking depending on the voting process and
>> testing cycles we can release the extensions 1.0.0 together with Apache
>> Ignite 2.9.0 or right after within days of Apache Ignite 2.9.0 release.
>>
>>
>> Regards,
>> Saikat
>>
>> On Mon, Oct 12, 2020 at 2:58 PM Denis Magda <dm...@apache.org> wrote:
>>
>>> Saikat, Alex,
>>>
>>> Based on your inputs here, it sounds like once Ignite 2.9 gets released,
>>> all the integrations that made their way to the extensions repo [1] will
>>> get stuck in limbo for some time. What I mean here:
>>>
>>> 1. While the users will be bumping up their ignite-core,
>>> ignite-indexing, etc. versions to 2.9, they won't find a 2.9 artifact for
>>> Kafka, Camel and other extensions
>>> 2. Also, the users won't find 1.0 versions of those extensions that
>>> also need to be released
>>>
>>> So, it's unclear how those users are supposed to upgrade to Ignite 2.9
>>> if they use any of the extensions. Is it safe to use a 2.8 version of an
>>> extension? Or, should we release the 1.0 versions of all the migrated
>>> extensions together with 2.9?
>>>
>>>
>>> [1] https://github.com/apache/ignite-extensions/tree/master/modules
>>>
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com> wrote:
>>>
>>>> Saikat,
>>>>
>>>> The plan sounds good to me! Do you have an idea for the timeline of the
>>>> module releases? Let me know if I can help in any way.
>>>>
>>>> Also, I was looking into the modularization IEP and noticed that OSGI
>>>> module is discussed to be moved to the extensions, but there is no
>>>> corresponding ticket. Should I create one? I will be happy to help with
>>>> moving this module to extensions.
>>>>
>>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>>>
>>>> > Hi Alexey,
>>>> >
>>>> > All the modules as planned in IEP-36
>>>> >
>>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>>>> > are migrated to ignite-extensions repository.
>>>> >
>>>> > We can definitely plan to release ignite-extensions modules.
>>>> >
>>>> > I have a pending PR related to the kafka module and the PR is in
>>>> review
>>>> > https://github.com/apache/ignite/pull/8222 but its corresponding PR
>>>> has
>>>> > been merged in ignite-extensions repository.
>>>> >
>>>> > My thoughts / action items for the migration efforts and releases were
>>>> > as follows:
>>>> >
>>>> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>>>> > 2. Fix the upload nightly packages task for ignite-core to help fix
>>>> the
>>>> > travis build failure in ignite-extensions repository.
>>>> > 3. Review and update the README.md files for the ignite-extensions
>>>> modules
>>>> > as required.
>>>> > 4. Update the docs for ignite-extensions modules in ignite-website.
>>>> > 5. Release each module separately and share updates.
>>>> >
>>>> > Please let me know if you have feedback.
>>>> >
>>>> > Regards,
>>>> > Saikat
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
>>>> wrote:
>>>> >
>>>> >> It seems that gitbox.apache.org is not available on CI/CD.
>>>> >>
>>>> >> I will try to update VCS to GitHub.
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>>>> >> wrote:
>>>> >> >
>>>> >> > Guys,
>>>> >> >
>>>> >> > I've tried to build release candidate using TC [1]. But:
>>>> >> > 1. I can't choose a branch in this TC task (there is no such
>>>> field).
>>>> >> > 2. On the "default" branch I got an error: Failed to collect
>>>> changes,
>>>> >> > error: List remote refs failed:
>>>> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
>>>> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>>>> connect
>>>> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>>>> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>>>> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>>>> >> >
>>>> >> > Is there some problem with this TC task? What am I doing wrong?
>>>> >> >
>>>> >> > [1] :
>>>> >> >
>>>> >>
>>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>>>> >> >
>>>> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>>>> >> alexey.goncharuk@gmail.com>:
>>>> >> >
>>>> >> >> Saikat, Nikolay,
>>>> >> >>
>>>> >> >> We have migrated a bunch of modules to ignite-extensions recently.
>>>> >> Given
>>>> >> >> that these modules will not be available in Ignite 2.9 anymore
>>>> (will
>>>> >> >> they?), should we also plan to release the extensions?
>>>> >> >>
>>>> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>>>> plehanov.alex@gmail.com>:
>>>> >> >>
>>>> >> >>> Igniters,
>>>> >> >>>
>>>> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
>>>> review
>>>> >> >> it,
>>>> >> >>> please?
>>>> >> >>>
>>>> >> >>> [1]: https://github.com/apache/ignite/pull/8273
>>>> >> >>>
>>>> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>> >> >:
>>>> >> >>>
>>>> >> >>>> Guys,
>>>> >> >>>>
>>>> >> >>>> I've filled the ticket with reproducer [1] for the discovery
>>>> bug.
>>>> >> This
>>>> >> >>> bug
>>>> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin
>>>> privately
>>>> >> and
>>>> >> >>>> decided to revert this ticket. I will do it today (after TC bot
>>>> visa)
>>>> >> >> if
>>>> >> >>>> there are no objections.
>>>> >> >>>>
>>>> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>>> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>>> >> >>>>
>>>> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>> >> >:
>>>> >> >>>>
>>>> >> >>>>> Guys,
>>>> >> >>>>>
>>>> >> >>>>> During internal testing, we've found a critical bug with
>>>> >> >>>>> discovery (cluster falls apart if two nodes segmented
>>>> sequentially).
>>>> >> >>> This
>>>> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>>> >> >>>>> before release. Under investigation now. I'll let you know
>>>> when we
>>>> >> get
>>>> >> >>>>> something.
>>>> >> >>>>>
>>>> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>>> >> >>>>>
>>>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>>>> version of
>>>> >> >>> the
>>>> >> >>>>>> message factory in 2.9 and go for the runtime message
>>>> generation in
>>>> >> >>> later
>>>> >> >>>>>> release, or keep the code clean and fix the regression in the
>>>> next
>>>> >> >>> releases?
>>>> >> >>>>>>> Andrey, can you take a look at my change? I think it is
>>>> fairly
>>>> >> >>>>>> straightforward and does not change the semantics, just skips
>>>> the
>>>> >> >>> factory
>>>> >> >>>>>> closures for certain messages.
>>>> >> >>>>>>
>>>> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual
>>>> for all
>>>> >> >>>>>> workloads (I didn't get any significant drops during
>>>> benchmarking).
>>>> >> >> So
>>>> >> >>>>>> I prefer the runtime generation in later releases.
>>>> >> >>>>>>
>>>> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>>> >> >>>>>> <al...@gmail.com> wrote:
>>>> >> >>>>>>>
>>>> >> >>>>>>> Alexey, Andrey, Igniters,
>>>> >> >>>>>>>
>>>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>>>> version of
>>>> >> >>> the
>>>> >> >>>>>> message factory in 2.9 and go for the runtime message
>>>> generation in
>>>> >> >>> later
>>>> >> >>>>>> release, or keep the code clean and fix the regression in the
>>>> next
>>>> >> >>> releases?
>>>> >> >>>>>>> Andrey, can you take a look at my change? I think it is
>>>> fairly
>>>> >> >>>>>> straightforward and does not change the semantics, just skips
>>>> the
>>>> >> >>> factory
>>>> >> >>>>>> closures for certain messages.
>>>> >> >>>>>>>
>>>> >> >>>>>>> Personally, I would prefer fixing the regression given that
>>>> we
>>>> >> also
>>>> >> >>>>>> introduced tracing in this release.
>>>> >> >>>>>>>
>>>> >> >>>>>>>
>>>> >> >>>>>>>
>>>> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>>>> >> >> plehanov.alex@gmail.com
>>>> >> >>>> :
>>>> >> >>>>>>>>
>>>> >> >>>>>>>> Alexey,
>>>> >> >>>>>>>>
>>>> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>>>> ed52559eb95
>>>> >> >>> and
>>>> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
>>>> average
>>>> >> on
>>>> >> >>> our
>>>> >> >>>>>> environment (about the same results we got benchmarking
>>>> 65c30ec6 vs
>>>> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
>>>> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>>>> this
>>>> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>>>> servers, 5
>>>> >> >>> clients
>>>> >> >>>>>> 50 threads each. Yardstick results for this configuration:
>>>> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>>> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>>> >> >>>>>>>>
>>>> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>>> >> >>>>>> a.budnikov.ignite@gmail.com>:
>>>> >> >>>>>>>>>
>>>> >> >>>>>>>>> Hi Everyone,
>>>> >> >>>>>>>>>
>>>> >> >>>>>>>>> I posted an instruction on how to publish the docs on
>>>> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>>>> you
>>>> >> can
>>>> >> >>>>>> update the docs by following the instruction. Unfortunately, I
>>>> >> won't
>>>> >> >> be
>>>> >> >>>>>> able to spend any time on this project any longer. You can
>>>> send
>>>> >> your
>>>> >> >>> pull
>>>> >> >>>>>> requests and questions about the documentation to Denis Magda.
>>>> >> >>>>>>>>>
>>>> >> >>>>>>>>> -Artem
>>>> >> >>>>>>>>>
>>>> >> >>>>>>>>> [1] :
>>>> >> >>>>>>
>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>> >> >>>>>>>>>
>>>> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>>> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
>>>> >> >>>>>>>>>>
>>>> >> >>>>>>>>>> Alexey,
>>>> >> >>>>>>>>>>
>>>> >> >>>>>>>>>> I've tried to play with message factories locally, but
>>>> >> >>>>>> unfortunately, I
>>>> >> >>>>>>>>>> cannot spot the difference between old and new
>>>> implementation
>>>> >> in
>>>> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>>> >> >>>>>> MessageFactoryImpl
>>>> >> >>>>>>>>>> with the old switch statement to the
>>>> ignite-2.9-revert-12568
>>>> >> >>> branch
>>>> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>>>> >> >> compatible
>>>> >> >>>>>> with the
>>>> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
>>>> >> >>>>>>>>>>
>>>> >> >>>>>>>>>> Can you check if this change makes any difference
>>>> >> >> performance-wise
>>>> >> >>>>>> in your
>>>> >> >>>>>>>>>> environment? If yes, we can go with runtime code
>>>> generation in
>>>> >> >> the
>>>> >> >>>>>> long
>>>> >> >>>>>>>>>> term: register classes and generate a dynamic message
>>>> factory
>>>> >> >> with
>>>> >> >>>>>> a switch
>>>> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
>>>> though,
>>>> >> >>>>>> obviously).
>>>> >> >>>>>>>>>>
>>>> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>>> >> >>> plehanov.alex@gmail.com
>>>> >> >>>>>>> :
>>>> >> >>>>>>>>>>
>>>> >> >>>>>>>>>>> Hello guys,
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket
>>>> [1]), it
>>>> >> >>>>>> reduced the
>>>> >> >>>>>>>>>>> drop, but not completely removed it.
>>>> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>>>> patch?
>>>> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>>>> against
>>>> >> >>>>>> 2.8.1
>>>> >> >>>>>>>>>>> release on your environment?
>>>> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>>>> left,
>>>> >> >>>>>> it's close
>>>> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
>>>> >> >>>>>> showstopper. Guys,
>>>> >> >>>>>>>>>>> WDYT?
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>>>> >> >> driver
>>>> >> >>>>>> between 2.8
>>>> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>>>> >> >> should
>>>> >> >>>>>> be
>>>> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>>> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4]
>>>> (NIO
>>>> >> >>> message
>>>> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick
>>>> it if
>>>> >> >>>>>> there is no
>>>> >> >>>>>>>>>>> objection.
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>>> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>>>> >> >> mmuzaf@apache.org
>>>> >> >>>> :
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>>>>>>> Alexey,
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>>>> >> >> this
>>>> >> >>>>>> issue is
>>>> >> >>>>>>>>>>>> related to the new master key change functionality which
>>>> >> >>>>>> haven't been
>>>> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick
>>>> commit
>>>> >> >> to
>>>> >> >>>>>> the
>>>> >> >>>>>>>>>>>> release branch.
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>>> >> >>>>>> nizhikov@apache.org>
>>>> >> >>>>>>>>>>> wrote:
>>>> >> >>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>> Hello, Igniters.
>>>> >> >>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>>>> >> >> [1]
>>>> >> >>>>>> into the
>>>> >> >>>>>>>>>>> 2.9
>>>> >> >>>>>>>>>>>> release
>>>> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>>>> fields
>>>> >> >>> in
>>>> >> >>>>>> wrong
>>>> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>>>> >> >>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>> >> >>>>>>>>>>>>> [2]
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>
>>>> >> >>>
>>>> >> >>
>>>> >>
>>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>> >> >>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>> >> >>>>>>>>>>>> написал(а):
>>>> >> >>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>>> >> >>>>>> anything out of
>>>> >> >>>>>>>>>>>> the
>>>> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no
>>>> ideas
>>>> >> >>>>>> right now),
>>>> >> >>>>>>>>>>> so
>>>> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to
>>>> the
>>>> >> >>>>>> switch
>>>> >> >>>>>>>>>>> approach
>>>> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably,
>>>> inlining
>>>> >> >>> the
>>>> >> >>>>>> Ignite
>>>> >> >>>>>>>>>>>> messages
>>>> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick.
>>>> Let
>>>> >> >>> me
>>>> >> >>>>>> explore
>>>> >> >>>>>>>>>>> the
>>>> >> >>>>>>>>>>>>>> code a bit.
>>>> >> >>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>>>> >> >> the
>>>> >> >>>>>>>>>>> performance.
>>>> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
>>>> single
>>>> >> >>>>>> virtual call
>>>> >> >>>>>>>>>>>>>> should not make that much of a difference given the
>>>> >> >> amount
>>>> >> >>>>>> of other
>>>> >> >>>>>>>>>>>> work
>>>> >> >>>>>>>>>>>>>> happening during the message processing.
>>>> >> >>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>> >> >>>>>> plehanov.alex@gmail.com
>>>> >> >>>>>>>>>>>> :
>>>> >> >>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>>> >> >>> results.
>>>> >> >>>>>>>>>>> Actually,
>>>> >> >>>>>>>>>>>> we
>>>> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>>> >> >>> between
>>>> >> >>>>>> e6a7f93
>>>> >> >>>>>>>>>>>> (first
>>>> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>>> >> >>>>>> 6592dfa5 (last
>>>> >> >>>>>>>>>>>> commit in
>>>> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>>> >> >>>>>> commits.
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible
>>>> for
>>>> >> >> a
>>>> >> >>>>>> drop
>>>> >> >>>>>>>>>>> between
>>>> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>>> >> >>>>>> reverted
>>>> >> >>>>>>>>>>>> IGNITE-13060
>>>> >> >>>>>>>>>>>>>>> now and performance looks the same)
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is
>>>> not
>>>> >> >>>>>> related to
>>>> >> >>>>>>>>>>>> drop
>>>> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>>> >> >>>>>> problem, and
>>>> >> >>>>>>>>>>> we
>>>> >> >>>>>>>>>>>> can
>>>> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>>>> >> >> leave
>>>> >> >>>>>> it as is?
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>>> >> >>>>>> refactoring)?
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>> >> >>>>>>>>>>>>>>>> :
>>>> >> >>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> Alexey,
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has
>>>> an
>>>> >> >>>>>> incorrect fix
>>>> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
>>>> branch
>>>> >> >>>>>> [1], so it
>>>> >> >>>>>>>>>>>> cannot be
>>>> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>>>> >> >> work
>>>> >> >>>>>> with fix
>>>> >> >>>>>>>>>>>> versions
>>>> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>>> >> >>>>>> versions.
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> --AG
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> [1]
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>
>>>> >> >>>
>>>> >> >>
>>>> >>
>>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>> >> >>>>>>>>>>>>>>>>> :
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>> >> >>>>>>>>>>> plehanov.alex@gmail.com
>>>> >> >>>>>>>>>>>>> :
>>>> >> >>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>> Guys,
>>>> >> >>>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>> >> >>>>>> IGNITE-12568
>>>> >> >>>>>>>>>>>> (reverted
>>>> >> >>>>>>>>>>>>>>>>>> it
>>>> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>>> >> >>>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to
>>>> hot
>>>> >> >>>>>> paths, to
>>>> >> >>>>>>>>>>> trace
>>>> >> >>>>>>>>>>>>>>>>>> these
>>>> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>>> >> >>> here.
>>>> >> >>>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>>> >> >>> switch/case
>>>> >> >>>>>> block was
>>>> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>>> >> >>>>>> message factory
>>>> >> >>>>>>>>>>>> is on
>>>> >> >>>>>>>>>>>>>>>>>> the
>>>> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>>>> >> >> impact
>>>> >> >>>>>> on total
>>>> >> >>>>>>>>>>>>>>>>>> performance.
>>>> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>>> >> >>>>>> microbenchmarks,
>>>> >> >>>>>>>>>>>> and
>>>> >> >>>>>>>>>>>>>>>>>> found
>>>> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>>> >> >>>>>> about 30-35%
>>>> >> >>>>>>>>>>>> faster
>>>> >> >>>>>>>>>>>>>>>>>> than
>>>> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with
>>>> switch/case
>>>> >> >>> can
>>>> >> >>>>>>>>>>> effectively
>>>> >> >>>>>>>>>>>>>>>>>> inline
>>>> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>>>> >> >> suppliers
>>>> >> >>>>>> relatively
>>>> >> >>>>>>>>>>>> heavy
>>>> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>>> >> >>>>>> rewrite the
>>>> >> >>>>>>>>>>> code
>>>> >> >>>>>>>>>>>>>>>>>> using
>>>> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>>> >> >>> interface
>>>> >> >>>>>> (to
>>>> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
>>>> "invokevirtual"),
>>>> >> >>>>>> but it gives
>>>> >> >>>>>>>>>>>> back
>>>> >> >>>>>>>>>>>>>>>>>> only
>>>> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>>>> >> >> looks
>>>> >> >>>>>> ugly
>>>> >> >>>>>>>>>>>> (lambdas
>>>> >> >>>>>>>>>>>>>>>>>> can't
>>>> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>>> >> >>>>>> optimize the
>>>> >> >>>>>>>>>>>> current
>>>> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>>> >> >>>>>> Andrey Gura,
>>>> >> >>>>>>>>>>> as
>>>> >> >>>>>>>>>>>> the
>>>> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>>> >> >>> about
>>>> >> >>>>>>>>>>>> optimization?
>>>> >> >>>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there
>>>> are
>>>> >> >>>>>> some metrics
>>>> >> >>>>>>>>>>>>>>>>>> already
>>>> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old
>>>> message
>>>> >> >>>>>> factory
>>>> >> >>>>>>>>>>>>>>>>>> implementation
>>>> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>>> >> >>>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>> Alexey,
>>>> >> >>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>>> >> >>>>>> already released
>>>> >> >>>>>>>>>>>> in
>>>> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory)
>>>> is
>>>> >> >>>>>> only present
>>>> >> >>>>>>>>>>>> in Ignite
>>>> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever
>>>> new
>>>> >> >>>>>> metrics
>>>> >> >>>>>>>>>>>> created for
>>>> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
>>>> unblock
>>>> >> >>>>>> the release
>>>> >> >>>>>>>>>>>> and deal
>>>> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>>> >> >>>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>>
>>>> >> >>>>>>>>>>>>
>>>> >> >>>>>>>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>
>>>> >> >>>
>>>> >> >>
>>>> >>
>>>> >>
>>>>
>>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Is it possible/reasonable to run a single vote for Ignite 2.9 and all the
migrated extensions? This time. @Alex Plehanov <pl...@gmail.com>,
what do you think?
If we decide to release the versions 1.0.0 of the extensions after 2.9 is
out then it can last for a week or so...because we need to build, update
docs, vote, publish.
-
Denis
On Mon, Oct 12, 2020 at 7:19 PM Saikat Maitra <sa...@gmail.com>
wrote:
> Hi Denis,
>
> All the modules except OSGi have been migrated. Alexey is creating a
> separate ticket for OSGi and we can migrate the same as well.
>
> My thoughts are Apache Ignite 2.9.0 and migrated extensions 1.0.0 to be
> released together. I am thinking depending on the voting process and
> testing cycles we can release the extensions 1.0.0 together with Apache
> Ignite 2.9.0 or right after within days of Apache Ignite 2.9.0 release.
>
>
> Regards,
> Saikat
>
> On Mon, Oct 12, 2020 at 2:58 PM Denis Magda <dm...@apache.org> wrote:
>
>> Saikat, Alex,
>>
>> Based on your inputs here, it sounds like once Ignite 2.9 gets released,
>> all the integrations that made their way to the extensions repo [1] will
>> get stuck in limbo for some time. What I mean here:
>>
>> 1. While the users will be bumping up their ignite-core,
>> ignite-indexing, etc. versions to 2.9, they won't find a 2.9 artifact for
>> Kafka, Camel and other extensions
>> 2. Also, the users won't find 1.0 versions of those extensions that
>> also need to be released
>>
>> So, it's unclear how those users are supposed to upgrade to Ignite 2.9 if
>> they use any of the extensions. Is it safe to use a 2.8 version of an
>> extension? Or, should we release the 1.0 versions of all the migrated
>> extensions together with 2.9?
>>
>>
>> [1] https://github.com/apache/ignite-extensions/tree/master/modules
>>
>>
>> -
>> Denis
>>
>>
>> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
>> alexey.goncharuk@gmail.com> wrote:
>>
>>> Saikat,
>>>
>>> The plan sounds good to me! Do you have an idea for the timeline of the
>>> module releases? Let me know if I can help in any way.
>>>
>>> Also, I was looking into the modularization IEP and noticed that OSGI
>>> module is discussed to be moved to the extensions, but there is no
>>> corresponding ticket. Should I create one? I will be happy to help with
>>> moving this module to extensions.
>>>
>>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>>
>>> > Hi Alexey,
>>> >
>>> > All the modules as planned in IEP-36
>>> >
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>>> > are migrated to ignite-extensions repository.
>>> >
>>> > We can definitely plan to release ignite-extensions modules.
>>> >
>>> > I have a pending PR related to the kafka module and the PR is in review
>>> > https://github.com/apache/ignite/pull/8222 but its corresponding PR
>>> has
>>> > been merged in ignite-extensions repository.
>>> >
>>> > My thoughts / action items for the migration efforts and releases were
>>> > as follows:
>>> >
>>> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>>> > 2. Fix the upload nightly packages task for ignite-core to help fix the
>>> > travis build failure in ignite-extensions repository.
>>> > 3. Review and update the README.md files for the ignite-extensions
>>> modules
>>> > as required.
>>> > 4. Update the docs for ignite-extensions modules in ignite-website.
>>> > 5. Release each module separately and share updates.
>>> >
>>> > Please let me know if you have feedback.
>>> >
>>> > Regards,
>>> > Saikat
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
>>> wrote:
>>> >
>>> >> It seems that gitbox.apache.org is not available on CI/CD.
>>> >>
>>> >> I will try to update VCS to GitHub.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>>> >> wrote:
>>> >> >
>>> >> > Guys,
>>> >> >
>>> >> > I've tried to build release candidate using TC [1]. But:
>>> >> > 1. I can't choose a branch in this TC task (there is no such field).
>>> >> > 2. On the "default" branch I got an error: Failed to collect
>>> changes,
>>> >> > error: List remote refs failed:
>>> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
>>> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>>> connect
>>> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>>> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>>> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>>> >> >
>>> >> > Is there some problem with this TC task? What am I doing wrong?
>>> >> >
>>> >> > [1] :
>>> >> >
>>> >>
>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>>> >> >
>>> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>>> >> alexey.goncharuk@gmail.com>:
>>> >> >
>>> >> >> Saikat, Nikolay,
>>> >> >>
>>> >> >> We have migrated a bunch of modules to ignite-extensions recently.
>>> >> Given
>>> >> >> that these modules will not be available in Ignite 2.9 anymore
>>> (will
>>> >> >> they?), should we also plan to release the extensions?
>>> >> >>
>>> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>>> plehanov.alex@gmail.com>:
>>> >> >>
>>> >> >>> Igniters,
>>> >> >>>
>>> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
>>> review
>>> >> >> it,
>>> >> >>> please?
>>> >> >>>
>>> >> >>> [1]: https://github.com/apache/ignite/pull/8273
>>> >> >>>
>>> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>>> plehanov.alex@gmail.com
>>> >> >:
>>> >> >>>
>>> >> >>>> Guys,
>>> >> >>>>
>>> >> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
>>> >> This
>>> >> >>> bug
>>> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin
>>> privately
>>> >> and
>>> >> >>>> decided to revert this ticket. I will do it today (after TC bot
>>> visa)
>>> >> >> if
>>> >> >>>> there are no objections.
>>> >> >>>>
>>> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>> >> >>>>
>>> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>>> plehanov.alex@gmail.com
>>> >> >:
>>> >> >>>>
>>> >> >>>>> Guys,
>>> >> >>>>>
>>> >> >>>>> During internal testing, we've found a critical bug with
>>> >> >>>>> discovery (cluster falls apart if two nodes segmented
>>> sequentially).
>>> >> >>> This
>>> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>> >> >>>>> before release. Under investigation now. I'll let you know when
>>> we
>>> >> get
>>> >> >>>>> something.
>>> >> >>>>>
>>> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>> >> >>>>>
>>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>>> version of
>>> >> >>> the
>>> >> >>>>>> message factory in 2.9 and go for the runtime message
>>> generation in
>>> >> >>> later
>>> >> >>>>>> release, or keep the code clean and fix the regression in the
>>> next
>>> >> >>> releases?
>>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>> >> >>>>>> straightforward and does not change the semantics, just skips
>>> the
>>> >> >>> factory
>>> >> >>>>>> closures for certain messages.
>>> >> >>>>>>
>>> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual
>>> for all
>>> >> >>>>>> workloads (I didn't get any significant drops during
>>> benchmarking).
>>> >> >> So
>>> >> >>>>>> I prefer the runtime generation in later releases.
>>> >> >>>>>>
>>> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>> >> >>>>>> <al...@gmail.com> wrote:
>>> >> >>>>>>>
>>> >> >>>>>>> Alexey, Andrey, Igniters,
>>> >> >>>>>>>
>>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>>> version of
>>> >> >>> the
>>> >> >>>>>> message factory in 2.9 and go for the runtime message
>>> generation in
>>> >> >>> later
>>> >> >>>>>> release, or keep the code clean and fix the regression in the
>>> next
>>> >> >>> releases?
>>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>> >> >>>>>> straightforward and does not change the semantics, just skips
>>> the
>>> >> >>> factory
>>> >> >>>>>> closures for certain messages.
>>> >> >>>>>>>
>>> >> >>>>>>> Personally, I would prefer fixing the regression given that we
>>> >> also
>>> >> >>>>>> introduced tracing in this release.
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>>> >> >> plehanov.alex@gmail.com
>>> >> >>>> :
>>> >> >>>>>>>>
>>> >> >>>>>>>> Alexey,
>>> >> >>>>>>>>
>>> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>>> ed52559eb95
>>> >> >>> and
>>> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
>>> average
>>> >> on
>>> >> >>> our
>>> >> >>>>>> environment (about the same results we got benchmarking
>>> 65c30ec6 vs
>>> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
>>> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>>> this
>>> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>>> servers, 5
>>> >> >>> clients
>>> >> >>>>>> 50 threads each. Yardstick results for this configuration:
>>> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>> >> >>>>>>>>
>>> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>> >> >>>>>> a.budnikov.ignite@gmail.com>:
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> Hi Everyone,
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> I posted an instruction on how to publish the docs on
>>> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>>> you
>>> >> can
>>> >> >>>>>> update the docs by following the instruction. Unfortunately, I
>>> >> won't
>>> >> >> be
>>> >> >>>>>> able to spend any time on this project any longer. You can send
>>> >> your
>>> >> >>> pull
>>> >> >>>>>> requests and questions about the documentation to Denis Magda.
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> -Artem
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> [1] :
>>> >> >>>>>>
>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>> >> >>>>>>>>>
>>> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Alexey,
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> I've tried to play with message factories locally, but
>>> >> >>>>>> unfortunately, I
>>> >> >>>>>>>>>> cannot spot the difference between old and new
>>> implementation
>>> >> in
>>> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>> >> >>>>>> MessageFactoryImpl
>>> >> >>>>>>>>>> with the old switch statement to the
>>> ignite-2.9-revert-12568
>>> >> >>> branch
>>> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>>> >> >> compatible
>>> >> >>>>>> with the
>>> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> Can you check if this change makes any difference
>>> >> >> performance-wise
>>> >> >>>>>> in your
>>> >> >>>>>>>>>> environment? If yes, we can go with runtime code
>>> generation in
>>> >> >> the
>>> >> >>>>>> long
>>> >> >>>>>>>>>> term: register classes and generate a dynamic message
>>> factory
>>> >> >> with
>>> >> >>>>>> a switch
>>> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
>>> though,
>>> >> >>>>>> obviously).
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>> >> >>> plehanov.alex@gmail.com
>>> >> >>>>>>> :
>>> >> >>>>>>>>>>
>>> >> >>>>>>>>>>> Hello guys,
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket
>>> [1]), it
>>> >> >>>>>> reduced the
>>> >> >>>>>>>>>>> drop, but not completely removed it.
>>> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>>> patch?
>>> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>>> against
>>> >> >>>>>> 2.8.1
>>> >> >>>>>>>>>>> release on your environment?
>>> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>>> left,
>>> >> >>>>>> it's close
>>> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
>>> >> >>>>>> showstopper. Guys,
>>> >> >>>>>>>>>>> WDYT?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>>> >> >> driver
>>> >> >>>>>> between 2.8
>>> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>>> >> >> should
>>> >> >>>>>> be
>>> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>> >> >>> message
>>> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick
>>> it if
>>> >> >>>>>> there is no
>>> >> >>>>>>>>>>> objection.
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>>> >> >> mmuzaf@apache.org
>>> >> >>>> :
>>> >> >>>>>>>>>>>
>>> >> >>>>>>>>>>>> Alexey,
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>>> >> >> this
>>> >> >>>>>> issue is
>>> >> >>>>>>>>>>>> related to the new master key change functionality which
>>> >> >>>>>> haven't been
>>> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick
>>> commit
>>> >> >> to
>>> >> >>>>>> the
>>> >> >>>>>>>>>>>> release branch.
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>> >> >>>>>> nizhikov@apache.org>
>>> >> >>>>>>>>>>> wrote:
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Hello, Igniters.
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>>> >> >> [1]
>>> >> >>>>>> into the
>>> >> >>>>>>>>>>> 2.9
>>> >> >>>>>>>>>>>> release
>>> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>>> fields
>>> >> >>> in
>>> >> >>>>>> wrong
>>> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>> >> >>>>>>>>>>>>> [2]
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>
>>> >> >>>>>>
>>> >> >>>
>>> >> >>
>>> >>
>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>>> >> >>>>>>>>>>>> написал(а):
>>> >> >>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>> >> >>>>>> anything out of
>>> >> >>>>>>>>>>>> the
>>> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no
>>> ideas
>>> >> >>>>>> right now),
>>> >> >>>>>>>>>>> so
>>> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>> >> >>>>>> switch
>>> >> >>>>>>>>>>> approach
>>> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>> >> >>> the
>>> >> >>>>>> Ignite
>>> >> >>>>>>>>>>>> messages
>>> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick.
>>> Let
>>> >> >>> me
>>> >> >>>>>> explore
>>> >> >>>>>>>>>>> the
>>> >> >>>>>>>>>>>>>> code a bit.
>>> >> >>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>>> >> >> the
>>> >> >>>>>>>>>>> performance.
>>> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a
>>> single
>>> >> >>>>>> virtual call
>>> >> >>>>>>>>>>>>>> should not make that much of a difference given the
>>> >> >> amount
>>> >> >>>>>> of other
>>> >> >>>>>>>>>>>> work
>>> >> >>>>>>>>>>>>>> happening during the message processing.
>>> >> >>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>> >> >>>>>> plehanov.alex@gmail.com
>>> >> >>>>>>>>>>>> :
>>> >> >>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>> >> >>> results.
>>> >> >>>>>>>>>>> Actually,
>>> >> >>>>>>>>>>>> we
>>> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>> >> >>> between
>>> >> >>>>>> e6a7f93
>>> >> >>>>>>>>>>>> (first
>>> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>> >> >>>>>> 6592dfa5 (last
>>> >> >>>>>>>>>>>> commit in
>>> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>> >> >>>>>> commits.
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>>> >> >> a
>>> >> >>>>>> drop
>>> >> >>>>>>>>>>> between
>>> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>> >> >>>>>> reverted
>>> >> >>>>>>>>>>>> IGNITE-13060
>>> >> >>>>>>>>>>>>>>> now and performance looks the same)
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is
>>> not
>>> >> >>>>>> related to
>>> >> >>>>>>>>>>>> drop
>>> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>> >> >>>>>> problem, and
>>> >> >>>>>>>>>>> we
>>> >> >>>>>>>>>>>> can
>>> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>>> >> >> leave
>>> >> >>>>>> it as is?
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>> >> >>>>>> refactoring)?
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>> >> >>>>>>>>>>>>>>>> :
>>> >> >>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> Alexey,
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>> >> >>>>>> incorrect fix
>>> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1
>>> branch
>>> >> >>>>>> [1], so it
>>> >> >>>>>>>>>>>> cannot be
>>> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>>> >> >> work
>>> >> >>>>>> with fix
>>> >> >>>>>>>>>>>> versions
>>> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>> >> >>>>>> versions.
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> --AG
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> [1]
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>
>>> >> >>>>>>
>>> >> >>>
>>> >> >>
>>> >>
>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>> >> >>>>>>>>>>>>>>>>> :
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>> >> >>>>>>>>>>> plehanov.alex@gmail.com
>>> >> >>>>>>>>>>>>> :
>>> >> >>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>> Guys,
>>> >> >>>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>> >> >>>>>> IGNITE-12568
>>> >> >>>>>>>>>>>> (reverted
>>> >> >>>>>>>>>>>>>>>>>> it
>>> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>> >> >>>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>> >> >>>>>> paths, to
>>> >> >>>>>>>>>>> trace
>>> >> >>>>>>>>>>>>>>>>>> these
>>> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>> >> >>> here.
>>> >> >>>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>> >> >>> switch/case
>>> >> >>>>>> block was
>>> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>> >> >>>>>> message factory
>>> >> >>>>>>>>>>>> is on
>>> >> >>>>>>>>>>>>>>>>>> the
>>> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>>> >> >> impact
>>> >> >>>>>> on total
>>> >> >>>>>>>>>>>>>>>>>> performance.
>>> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>> >> >>>>>> microbenchmarks,
>>> >> >>>>>>>>>>>> and
>>> >> >>>>>>>>>>>>>>>>>> found
>>> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>> >> >>>>>> about 30-35%
>>> >> >>>>>>>>>>>> faster
>>> >> >>>>>>>>>>>>>>>>>> than
>>> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>> >> >>> can
>>> >> >>>>>>>>>>> effectively
>>> >> >>>>>>>>>>>>>>>>>> inline
>>> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>>> >> >> suppliers
>>> >> >>>>>> relatively
>>> >> >>>>>>>>>>>> heavy
>>> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>> >> >>>>>> rewrite the
>>> >> >>>>>>>>>>> code
>>> >> >>>>>>>>>>>>>>>>>> using
>>> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>> >> >>> interface
>>> >> >>>>>> (to
>>> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the
>>> "invokevirtual"),
>>> >> >>>>>> but it gives
>>> >> >>>>>>>>>>>> back
>>> >> >>>>>>>>>>>>>>>>>> only
>>> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>>> >> >> looks
>>> >> >>>>>> ugly
>>> >> >>>>>>>>>>>> (lambdas
>>> >> >>>>>>>>>>>>>>>>>> can't
>>> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>> >> >>>>>> optimize the
>>> >> >>>>>>>>>>>> current
>>> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>> >> >>>>>> Andrey Gura,
>>> >> >>>>>>>>>>> as
>>> >> >>>>>>>>>>>> the
>>> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>> >> >>> about
>>> >> >>>>>>>>>>>> optimization?
>>> >> >>>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there
>>> are
>>> >> >>>>>> some metrics
>>> >> >>>>>>>>>>>>>>>>>> already
>>> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>> >> >>>>>> factory
>>> >> >>>>>>>>>>>>>>>>>> implementation
>>> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>> >> >>>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>> Alexey,
>>> >> >>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>> >> >>>>>> already released
>>> >> >>>>>>>>>>>> in
>>> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>> >> >>>>>> only present
>>> >> >>>>>>>>>>>> in Ignite
>>> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever
>>> new
>>> >> >>>>>> metrics
>>> >> >>>>>>>>>>>> created for
>>> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to
>>> unblock
>>> >> >>>>>> the release
>>> >> >>>>>>>>>>>> and deal
>>> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>> >> >>>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>>
>>> >> >>>>>>>>>>>>
>>> >> >>>>>>>>>>>
>>> >> >>>>>>
>>> >> >>>>>
>>> >> >>>
>>> >> >>
>>> >>
>>> >>
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Denis,
All the modules except OSGi have been migrated. Alexey is creating a
separate ticket for OSGi and we can migrate the same as well.
My thoughts are Apache Ignite 2.9.0 and migrated extensions 1.0.0 to be
released together. I am thinking depending on the voting process and
testing cycles we can release the extensions 1.0.0 together with Apache
Ignite 2.9.0 or right after within days of Apache Ignite 2.9.0 release.
Regards,
Saikat
On Mon, Oct 12, 2020 at 2:58 PM Denis Magda <dm...@apache.org> wrote:
> Saikat, Alex,
>
> Based on your inputs here, it sounds like once Ignite 2.9 gets released,
> all the integrations that made their way to the extensions repo [1] will
> get stuck in limbo for some time. What I mean here:
>
> 1. While the users will be bumping up their ignite-core,
> ignite-indexing, etc. versions to 2.9, they won't find a 2.9 artifact for
> Kafka, Camel and other extensions
> 2. Also, the users won't find 1.0 versions of those extensions that
> also need to be released
>
> So, it's unclear how those users are supposed to upgrade to Ignite 2.9 if
> they use any of the extensions. Is it safe to use a 2.8 version of an
> extension? Or, should we release the 1.0 versions of all the migrated
> extensions together with 2.9?
>
>
> [1] https://github.com/apache/ignite-extensions/tree/master/modules
>
>
> -
> Denis
>
>
> On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Saikat,
>>
>> The plan sounds good to me! Do you have an idea for the timeline of the
>> module releases? Let me know if I can help in any way.
>>
>> Also, I was looking into the modularization IEP and noticed that OSGI
>> module is discussed to be moved to the extensions, but there is no
>> corresponding ticket. Should I create one? I will be happy to help with
>> moving this module to extensions.
>>
>> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>>
>> > Hi Alexey,
>> >
>> > All the modules as planned in IEP-36
>> >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>> > are migrated to ignite-extensions repository.
>> >
>> > We can definitely plan to release ignite-extensions modules.
>> >
>> > I have a pending PR related to the kafka module and the PR is in review
>> > https://github.com/apache/ignite/pull/8222 but its corresponding PR has
>> > been merged in ignite-extensions repository.
>> >
>> > My thoughts / action items for the migration efforts and releases were
>> > as follows:
>> >
>> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>> > 2. Fix the upload nightly packages task for ignite-core to help fix the
>> > travis build failure in ignite-extensions repository.
>> > 3. Review and update the README.md files for the ignite-extensions
>> modules
>> > as required.
>> > 4. Update the docs for ignite-extensions modules in ignite-website.
>> > 5. Release each module separately and share updates.
>> >
>> > Please let me know if you have feedback.
>> >
>> > Regards,
>> > Saikat
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com>
>> wrote:
>> >
>> >> It seems that gitbox.apache.org is not available on CI/CD.
>> >>
>> >> I will try to update VCS to GitHub.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>> >> wrote:
>> >> >
>> >> > Guys,
>> >> >
>> >> > I've tried to build release candidate using TC [1]. But:
>> >> > 1. I can't choose a branch in this TC task (there is no such field).
>> >> > 2. On the "default" branch I got an error: Failed to collect changes,
>> >> > error: List remote refs failed:
>> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
>> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
>> connect
>> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>> >> >
>> >> > Is there some problem with this TC task? What am I doing wrong?
>> >> >
>> >> > [1] :
>> >> >
>> >>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>> >> >
>> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>> >> alexey.goncharuk@gmail.com>:
>> >> >
>> >> >> Saikat, Nikolay,
>> >> >>
>> >> >> We have migrated a bunch of modules to ignite-extensions recently.
>> >> Given
>> >> >> that these modules will not be available in Ignite 2.9 anymore (will
>> >> >> they?), should we also plan to release the extensions?
>> >> >>
>> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <
>> plehanov.alex@gmail.com>:
>> >> >>
>> >> >>> Igniters,
>> >> >>>
>> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
>> review
>> >> >> it,
>> >> >>> please?
>> >> >>>
>> >> >>> [1]: https://github.com/apache/ignite/pull/8273
>> >> >>>
>> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
>> plehanov.alex@gmail.com
>> >> >:
>> >> >>>
>> >> >>>> Guys,
>> >> >>>>
>> >> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
>> >> This
>> >> >>> bug
>> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>> >> and
>> >> >>>> decided to revert this ticket. I will do it today (after TC bot
>> visa)
>> >> >> if
>> >> >>>> there are no objections.
>> >> >>>>
>> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>> >> >>>>
>> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>> plehanov.alex@gmail.com
>> >> >:
>> >> >>>>
>> >> >>>>> Guys,
>> >> >>>>>
>> >> >>>>> During internal testing, we've found a critical bug with
>> >> >>>>> discovery (cluster falls apart if two nodes segmented
>> sequentially).
>> >> >>> This
>> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>> >> >>>>> before release. Under investigation now. I'll let you know when
>> we
>> >> get
>> >> >>>>> something.
>> >> >>>>>
>> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>> >> >>>>>
>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version of
>> >> >>> the
>> >> >>>>>> message factory in 2.9 and go for the runtime message
>> generation in
>> >> >>> later
>> >> >>>>>> release, or keep the code clean and fix the regression in the
>> next
>> >> >>> releases?
>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >> >>>>>> straightforward and does not change the semantics, just skips
>> the
>> >> >>> factory
>> >> >>>>>> closures for certain messages.
>> >> >>>>>>
>> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
>> all
>> >> >>>>>> workloads (I didn't get any significant drops during
>> benchmarking).
>> >> >> So
>> >> >>>>>> I prefer the runtime generation in later releases.
>> >> >>>>>>
>> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>> >> >>>>>> <al...@gmail.com> wrote:
>> >> >>>>>>>
>> >> >>>>>>> Alexey, Andrey, Igniters,
>> >> >>>>>>>
>> >> >>>>>>> So what do you think? Should we proceed with a 'hacked'
>> version of
>> >> >>> the
>> >> >>>>>> message factory in 2.9 and go for the runtime message
>> generation in
>> >> >>> later
>> >> >>>>>> release, or keep the code clean and fix the regression in the
>> next
>> >> >>> releases?
>> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >> >>>>>> straightforward and does not change the semantics, just skips
>> the
>> >> >>> factory
>> >> >>>>>> closures for certain messages.
>> >> >>>>>>>
>> >> >>>>>>> Personally, I would prefer fixing the regression given that we
>> >> also
>> >> >>>>>> introduced tracing in this release.
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> >> >> plehanov.alex@gmail.com
>> >> >>>> :
>> >> >>>>>>>>
>> >> >>>>>>>> Alexey,
>> >> >>>>>>>>
>> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>> ed52559eb95
>> >> >>> and
>> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
>> average
>> >> on
>> >> >>> our
>> >> >>>>>> environment (about the same results we got benchmarking
>> 65c30ec6 vs
>> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
>> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on
>> this
>> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
>> servers, 5
>> >> >>> clients
>> >> >>>>>> 50 threads each. Yardstick results for this configuration:
>> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>> >> >>>>>>>>
>> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>> >> >>>>>> a.budnikov.ignite@gmail.com>:
>> >> >>>>>>>>>
>> >> >>>>>>>>> Hi Everyone,
>> >> >>>>>>>>>
>> >> >>>>>>>>> I posted an instruction on how to publish the docs on
>> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9,
>> you
>> >> can
>> >> >>>>>> update the docs by following the instruction. Unfortunately, I
>> >> won't
>> >> >> be
>> >> >>>>>> able to spend any time on this project any longer. You can send
>> >> your
>> >> >>> pull
>> >> >>>>>> requests and questions about the documentation to Denis Magda.
>> >> >>>>>>>>>
>> >> >>>>>>>>> -Artem
>> >> >>>>>>>>>
>> >> >>>>>>>>> [1] :
>> >> >>>>>>
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Alexey,
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I've tried to play with message factories locally, but
>> >> >>>>>> unfortunately, I
>> >> >>>>>>>>>> cannot spot the difference between old and new
>> implementation
>> >> in
>> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>> >> >>>>>> MessageFactoryImpl
>> >> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>> >> >>> branch
>> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> >> >> compatible
>> >> >>>>>> with the
>> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> Can you check if this change makes any difference
>> >> >> performance-wise
>> >> >>>>>> in your
>> >> >>>>>>>>>> environment? If yes, we can go with runtime code generation
>> in
>> >> >> the
>> >> >>>>>> long
>> >> >>>>>>>>>> term: register classes and generate a dynamic message
>> factory
>> >> >> with
>> >> >>>>>> a switch
>> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
>> though,
>> >> >>>>>> obviously).
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>> >> >>> plehanov.alex@gmail.com
>> >> >>>>>>> :
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>> Hello guys,
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket
>> [1]), it
>> >> >>>>>> reduced the
>> >> >>>>>>>>>>> drop, but not completely removed it.
>> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
>> patch?
>> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>> against
>> >> >>>>>> 2.8.1
>> >> >>>>>>>>>>> release on your environment?
>> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>> left,
>> >> >>>>>> it's close
>> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
>> >> >>>>>> showstopper. Guys,
>> >> >>>>>>>>>>> WDYT?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> >> >> driver
>> >> >>>>>> between 2.8
>> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> >> >> should
>> >> >>>>>> be
>> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>> >> >>> message
>> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
>> if
>> >> >>>>>> there is no
>> >> >>>>>>>>>>> objection.
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> >> >> mmuzaf@apache.org
>> >> >>>> :
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> >> >> this
>> >> >>>>>> issue is
>> >> >>>>>>>>>>>> related to the new master key change functionality which
>> >> >>>>>> haven't been
>> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>> >> >> to
>> >> >>>>>> the
>> >> >>>>>>>>>>>> release branch.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>> >> >>>>>> nizhikov@apache.org>
>> >> >>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Hello, Igniters.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> >> >> [1]
>> >> >>>>>> into the
>> >> >>>>>>>>>>> 2.9
>> >> >>>>>>>>>>>> release
>> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns
>> fields
>> >> >>> in
>> >> >>>>>> wrong
>> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> >> >>>>>>>>>>>>> [2]
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>> >> >>>>>>>>>>>> написал(а):
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>> >> >>>>>> anything out of
>> >> >>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>> >> >>>>>> right now),
>> >> >>>>>>>>>>> so
>> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
>> >> >>>>>> switch
>> >> >>>>>>>>>>> approach
>> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>> >> >>> the
>> >> >>>>>> Ignite
>> >> >>>>>>>>>>>> messages
>> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>> >> >>> me
>> >> >>>>>> explore
>> >> >>>>>>>>>>> the
>> >> >>>>>>>>>>>>>> code a bit.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> >> >> the
>> >> >>>>>>>>>>> performance.
>> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>> >> >>>>>> virtual call
>> >> >>>>>>>>>>>>>> should not make that much of a difference given the
>> >> >> amount
>> >> >>>>>> of other
>> >> >>>>>>>>>>>> work
>> >> >>>>>>>>>>>>>> happening during the message processing.
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> >> >>>>>> plehanov.alex@gmail.com
>> >> >>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>> >> >>> results.
>> >> >>>>>>>>>>> Actually,
>> >> >>>>>>>>>>>> we
>> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>> >> >>> between
>> >> >>>>>> e6a7f93
>> >> >>>>>>>>>>>> (first
>> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>> >> >>>>>> 6592dfa5 (last
>> >> >>>>>>>>>>>> commit in
>> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>> >> >>>>>> commits.
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>> >> >> a
>> >> >>>>>> drop
>> >> >>>>>>>>>>> between
>> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>> >> >>>>>> reverted
>> >> >>>>>>>>>>>> IGNITE-13060
>> >> >>>>>>>>>>>>>>> now and performance looks the same)
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>> >> >>>>>> related to
>> >> >>>>>>>>>>>> drop
>> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>> >> >>>>>> problem, and
>> >> >>>>>>>>>>> we
>> >> >>>>>>>>>>>> can
>> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> >> >> leave
>> >> >>>>>> it as is?
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>> >> >>>>>> refactoring)?
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >> >>>>>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>> >> >>>>>> incorrect fix
>> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>> >> >>>>>> [1], so it
>> >> >>>>>>>>>>>> cannot be
>> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> >> >> work
>> >> >>>>>> with fix
>> >> >>>>>>>>>>>> versions
>> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>> >> >>>>>> versions.
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> --AG
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> [1]
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>
>> >> >>
>> >>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >> >>>>>>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> >> >>>>>>>>>>> plehanov.alex@gmail.com
>> >> >>>>>>>>>>>>> :
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> Guys,
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>> >> >>>>>> IGNITE-12568
>> >> >>>>>>>>>>>> (reverted
>> >> >>>>>>>>>>>>>>>>>> it
>> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>> >> >>>>>> paths, to
>> >> >>>>>>>>>>> trace
>> >> >>>>>>>>>>>>>>>>>> these
>> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>> >> >>> here.
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>> >> >>> switch/case
>> >> >>>>>> block was
>> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>> >> >>>>>> message factory
>> >> >>>>>>>>>>>> is on
>> >> >>>>>>>>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> >> >> impact
>> >> >>>>>> on total
>> >> >>>>>>>>>>>>>>>>>> performance.
>> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>> >> >>>>>> microbenchmarks,
>> >> >>>>>>>>>>>> and
>> >> >>>>>>>>>>>>>>>>>> found
>> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>> >> >>>>>> about 30-35%
>> >> >>>>>>>>>>>> faster
>> >> >>>>>>>>>>>>>>>>>> than
>> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>> >> >>> can
>> >> >>>>>>>>>>> effectively
>> >> >>>>>>>>>>>>>>>>>> inline
>> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> >> >> suppliers
>> >> >>>>>> relatively
>> >> >>>>>>>>>>>> heavy
>> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>> >> >>>>>> rewrite the
>> >> >>>>>>>>>>> code
>> >> >>>>>>>>>>>>>>>>>> using
>> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>> >> >>> interface
>> >> >>>>>> (to
>> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>> >> >>>>>> but it gives
>> >> >>>>>>>>>>>> back
>> >> >>>>>>>>>>>>>>>>>> only
>> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> >> >> looks
>> >> >>>>>> ugly
>> >> >>>>>>>>>>>> (lambdas
>> >> >>>>>>>>>>>>>>>>>> can't
>> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>> >> >>>>>> optimize the
>> >> >>>>>>>>>>>> current
>> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>> >> >>>>>> Andrey Gura,
>> >> >>>>>>>>>>> as
>> >> >>>>>>>>>>>> the
>> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>> >> >>> about
>> >> >>>>>>>>>>>> optimization?
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>> >> >>>>>> some metrics
>> >> >>>>>>>>>>>>>>>>>> already
>> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>> >> >>>>>> factory
>> >> >>>>>>>>>>>>>>>>>> implementation
>> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>> >> >>>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> Alexey,
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>> >> >>>>>> already released
>> >> >>>>>>>>>>>> in
>> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>> >> >>>>>> only present
>> >> >>>>>>>>>>>> in Ignite
>> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>> >> >>>>>> metrics
>> >> >>>>>>>>>>>> created for
>> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>> >> >>>>>> the release
>> >> >>>>>>>>>>>> and deal
>> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>> >> >>>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>>>>
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>
>> >> >>>>>>
>> >> >>>>>
>> >> >>>
>> >> >>
>> >>
>> >>
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Saikat, Alex,
Based on your inputs here, it sounds like once Ignite 2.9 gets released,
all the integrations that made their way to the extensions repo [1] will
get stuck in limbo for some time. What I mean here:
1. While the users will be bumping up their ignite-core,
ignite-indexing, etc. versions to 2.9, they won't find a 2.9 artifact for
Kafka, Camel and other extensions
2. Also, the users won't find 1.0 versions of those extensions that also
need to be released
So, it's unclear how those users are supposed to upgrade to Ignite 2.9 if
they use any of the extensions. Is it safe to use a 2.8 version of an
extension? Or, should we release the 1.0 versions of all the migrated
extensions together with 2.9?
[1] https://github.com/apache/ignite-extensions/tree/master/modules
-
Denis
On Tue, Oct 6, 2020 at 7:20 AM Alexey Goncharuk <al...@gmail.com>
wrote:
> Saikat,
>
> The plan sounds good to me! Do you have an idea for the timeline of the
> module releases? Let me know if I can help in any way.
>
> Also, I was looking into the modularization IEP and noticed that OSGI
> module is discussed to be moved to the extensions, but there is no
> corresponding ticket. Should I create one? I will be happy to help with
> moving this module to extensions.
>
> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>
> > Hi Alexey,
> >
> > All the modules as planned in IEP-36
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> > are migrated to ignite-extensions repository.
> >
> > We can definitely plan to release ignite-extensions modules.
> >
> > I have a pending PR related to the kafka module and the PR is in review
> > https://github.com/apache/ignite/pull/8222 but its corresponding PR has
> > been merged in ignite-extensions repository.
> >
> > My thoughts / action items for the migration efforts and releases were
> > as follows:
> >
> > 1. Merge the changes for https://github.com/apache/ignite/pull/8222
> > 2. Fix the upload nightly packages task for ignite-core to help fix the
> > travis build failure in ignite-extensions repository.
> > 3. Review and update the README.md files for the ignite-extensions
> modules
> > as required.
> > 4. Update the docs for ignite-extensions modules in ignite-website.
> > 5. Release each module separately and share updates.
> >
> > Please let me know if you have feedback.
> >
> > Regards,
> > Saikat
> >
> >
> >
> >
> >
> >
> >
> >
> > On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
> >
> >> It seems that gitbox.apache.org is not available on CI/CD.
> >>
> >> I will try to update VCS to GitHub.
> >>
> >>
> >>
> >>
> >>
> >> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
> >> wrote:
> >> >
> >> > Guys,
> >> >
> >> > I've tried to build release candidate using TC [1]. But:
> >> > 1. I can't choose a branch in this TC task (there is no such field).
> >> > 2. On the "default" branch I got an error: Failed to collect changes,
> >> > error: List remote refs failed:
> >> > org.apache.http.conn.ConnectTimeoutException: Connect to
> >> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed:
> connect
> >> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> >> > internal id=74, parent id=GitBoxAsfIgnite, description: "
> >> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
> >> >
> >> > Is there some problem with this TC task? What am I doing wrong?
> >> >
> >> > [1] :
> >> >
> >>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >> >
> >> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> >> alexey.goncharuk@gmail.com>:
> >> >
> >> >> Saikat, Nikolay,
> >> >>
> >> >> We have migrated a bunch of modules to ignite-extensions recently.
> >> Given
> >> >> that these modules will not be available in Ignite 2.9 anymore (will
> >> >> they?), should we also plan to release the extensions?
> >> >>
> >> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >> >>
> >> >>> Igniters,
> >> >>>
> >> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
> review
> >> >> it,
> >> >>> please?
> >> >>>
> >> >>> [1]: https://github.com/apache/ignite/pull/8273
> >> >>>
> >> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <
> plehanov.alex@gmail.com
> >> >:
> >> >>>
> >> >>>> Guys,
> >> >>>>
> >> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
> >> This
> >> >>> bug
> >> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
> >> and
> >> >>>> decided to revert this ticket. I will do it today (after TC bot
> visa)
> >> >> if
> >> >>>> there are no objections.
> >> >>>>
> >> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >> >>>>
> >> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
> plehanov.alex@gmail.com
> >> >:
> >> >>>>
> >> >>>>> Guys,
> >> >>>>>
> >> >>>>> During internal testing, we've found a critical bug with
> >> >>>>> discovery (cluster falls apart if two nodes segmented
> sequentially).
> >> >>> This
> >> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >> >>>>> before release. Under investigation now. I'll let you know when we
> >> get
> >> >>>>> something.
> >> >>>>>
> >> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >> >>>>>
> >> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
> of
> >> >>> the
> >> >>>>>> message factory in 2.9 and go for the runtime message generation
> in
> >> >>> later
> >> >>>>>> release, or keep the code clean and fix the regression in the
> next
> >> >>> releases?
> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >> >>>>>> straightforward and does not change the semantics, just skips the
> >> >>> factory
> >> >>>>>> closures for certain messages.
> >> >>>>>>
> >> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
> all
> >> >>>>>> workloads (I didn't get any significant drops during
> benchmarking).
> >> >> So
> >> >>>>>> I prefer the runtime generation in later releases.
> >> >>>>>>
> >> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >> >>>>>> <al...@gmail.com> wrote:
> >> >>>>>>>
> >> >>>>>>> Alexey, Andrey, Igniters,
> >> >>>>>>>
> >> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
> of
> >> >>> the
> >> >>>>>> message factory in 2.9 and go for the runtime message generation
> in
> >> >>> later
> >> >>>>>> release, or keep the code clean and fix the regression in the
> next
> >> >>> releases?
> >> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >> >>>>>> straightforward and does not change the semantics, just skips the
> >> >>> factory
> >> >>>>>> closures for certain messages.
> >> >>>>>>>
> >> >>>>>>> Personally, I would prefer fixing the regression given that we
> >> also
> >> >>>>>> introduced tracing in this release.
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >> >> plehanov.alex@gmail.com
> >> >>>> :
> >> >>>>>>>>
> >> >>>>>>>> Alexey,
> >> >>>>>>>>
> >> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
> ed52559eb95
> >> >>> and
> >> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
> average
> >> on
> >> >>> our
> >> >>>>>> environment (about the same results we got benchmarking 65c30ec6
> vs
> >> >>>>>> 0606f03d). We've made 24 runs for each commit of
> >> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> >> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6
> servers, 5
> >> >>> clients
> >> >>>>>> 50 threads each. Yardstick results for this configuration:
> >> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >> >>>>>>>>
> >> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >> >>>>>> a.budnikov.ignite@gmail.com>:
> >> >>>>>>>>>
> >> >>>>>>>>> Hi Everyone,
> >> >>>>>>>>>
> >> >>>>>>>>> I posted an instruction on how to publish the docs on
> >> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
> >> can
> >> >>>>>> update the docs by following the instruction. Unfortunately, I
> >> won't
> >> >> be
> >> >>>>>> able to spend any time on this project any longer. You can send
> >> your
> >> >>> pull
> >> >>>>>> requests and questions about the documentation to Denis Magda.
> >> >>>>>>>>>
> >> >>>>>>>>> -Artem
> >> >>>>>>>>>
> >> >>>>>>>>> [1] :
> >> >>>>>>
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >> >>>>>>>>>
> >> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >> >>>>>> alexey.goncharuk@gmail.com> wrote:
> >> >>>>>>>>>>
> >> >>>>>>>>>> Alexey,
> >> >>>>>>>>>>
> >> >>>>>>>>>> I've tried to play with message factories locally, but
> >> >>>>>> unfortunately, I
> >> >>>>>>>>>> cannot spot the difference between old and new implementation
> >> in
> >> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >> >>>>>> MessageFactoryImpl
> >> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
> >> >>> branch
> >> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >> >> compatible
> >> >>>>>> with the
> >> >>>>>>>>>> new metrics as we still use the register() mechanics).
> >> >>>>>>>>>>
> >> >>>>>>>>>> Can you check if this change makes any difference
> >> >> performance-wise
> >> >>>>>> in your
> >> >>>>>>>>>> environment? If yes, we can go with runtime code generation
> in
> >> >> the
> >> >>>>>> long
> >> >>>>>>>>>> term: register classes and generate a dynamic message factory
> >> >> with
> >> >>>>>> a switch
> >> >>>>>>>>>> statement once all messages are registered (not in 2.9
> though,
> >> >>>>>> obviously).
> >> >>>>>>>>>>
> >> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >> >>> plehanov.alex@gmail.com
> >> >>>>>>> :
> >> >>>>>>>>>>
> >> >>>>>>>>>>> Hello guys,
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
> it
> >> >>>>>> reduced the
> >> >>>>>>>>>>> drop, but not completely removed it.
> >> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the
> patch?
> >> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
> against
> >> >>>>>> 2.8.1
> >> >>>>>>>>>>> release on your environment?
> >> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
> left,
> >> >>>>>> it's close
> >> >>>>>>>>>>> to measurement error and I think such a drop is not a
> >> >>>>>> showstopper. Guys,
> >> >>>>>>>>>>> WDYT?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
> >> >> driver
> >> >>>>>> between 2.8
> >> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
> >> >> should
> >> >>>>>> be
> >> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
> >> >>> message
> >> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it
> if
> >> >>>>>> there is no
> >> >>>>>>>>>>> objection.
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >> >>>>>>>>>>>
> >> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >> >> mmuzaf@apache.org
> >> >>>> :
> >> >>>>>>>>>>>
> >> >>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
> >> >> this
> >> >>>>>> issue is
> >> >>>>>>>>>>>> related to the new master key change functionality which
> >> >>>>>> haven't been
> >> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
> >> >> to
> >> >>>>>> the
> >> >>>>>>>>>>>> release branch.
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >> >>>>>> nizhikov@apache.org>
> >> >>>>>>>>>>> wrote:
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Hello, Igniters.
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
> >> >> [1]
> >> >>>>>> into the
> >> >>>>>>>>>>> 2.9
> >> >>>>>>>>>>>> release
> >> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
> >> >>> in
> >> >>>>>> wrong
> >> >>>>>>>>>>>> order since the 2 row when fields_count>10"
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >> >>>>>>>>>>>>> [2]
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >> >>>>>>>>>>> alexey.goncharuk@gmail.com>
> >> >>>>>>>>>>>> написал(а):
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
> >> >>>>>> anything out of
> >> >>>>>>>>>>>> the
> >> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
> >> >>>>>> right now),
> >> >>>>>>>>>>> so
> >> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
> >> >>>>>> switch
> >> >>>>>>>>>>> approach
> >> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
> >> >>> the
> >> >>>>>> Ignite
> >> >>>>>>>>>>>> messages
> >> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
> >> >>> me
> >> >>>>>> explore
> >> >>>>>>>>>>> the
> >> >>>>>>>>>>>>>> code a bit.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
> >> >> the
> >> >>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
> >> >>>>>> virtual call
> >> >>>>>>>>>>>>>> should not make that much of a difference given the
> >> >> amount
> >> >>>>>> of other
> >> >>>>>>>>>>>> work
> >> >>>>>>>>>>>>>> happening during the message processing.
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >> >>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >> >>> results.
> >> >>>>>>>>>>> Actually,
> >> >>>>>>>>>>>> we
> >> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >> >>> between
> >> >>>>>> e6a7f93
> >> >>>>>>>>>>>> (first
> >> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >> >>>>>> 6592dfa5 (last
> >> >>>>>>>>>>>> commit in
> >> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> >> >>>>>> commits.
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
> >> >> a
> >> >>>>>> drop
> >> >>>>>>>>>>> between
> >> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >> >>>>>> reverted
> >> >>>>>>>>>>>> IGNITE-13060
> >> >>>>>>>>>>>>>>> now and performance looks the same)
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >> >>>>>> related to
> >> >>>>>>>>>>>> drop
> >> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
> >> >>>>>> problem, and
> >> >>>>>>>>>>> we
> >> >>>>>>>>>>>> can
> >> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> >> >> leave
> >> >>>>>> it as is?
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> >> >>>>>> refactoring)?
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
> >> >>>>>> incorrect fix
> >> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
> >> >>>>>> [1], so it
> >> >>>>>>>>>>>> cannot be
> >> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> >> >> work
> >> >>>>>> with fix
> >> >>>>>>>>>>>> versions
> >> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >> >>>>>> versions.
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> --AG
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> [1]
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>
> >> >>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >> >>>>>>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >> >>>>>>>>>>> plehanov.alex@gmail.com
> >> >>>>>>>>>>>>> :
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> Guys,
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >> >>>>>> IGNITE-12568
> >> >>>>>>>>>>>> (reverted
> >> >>>>>>>>>>>>>>>>>> it
> >> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
> >> >>>>>> paths, to
> >> >>>>>>>>>>> trace
> >> >>>>>>>>>>>>>>>>>> these
> >> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
> >> >>> here.
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >> >>> switch/case
> >> >>>>>> block was
> >> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> >> >>>>>> message factory
> >> >>>>>>>>>>>> is on
> >> >>>>>>>>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >> >> impact
> >> >>>>>> on total
> >> >>>>>>>>>>>>>>>>>> performance.
> >> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >> >>>>>> microbenchmarks,
> >> >>>>>>>>>>>> and
> >> >>>>>>>>>>>>>>>>>> found
> >> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
> >> >>>>>> about 30-35%
> >> >>>>>>>>>>>> faster
> >> >>>>>>>>>>>>>>>>>> than
> >> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
> >> >>> can
> >> >>>>>>>>>>> effectively
> >> >>>>>>>>>>>>>>>>>> inline
> >> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >> >> suppliers
> >> >>>>>> relatively
> >> >>>>>>>>>>>> heavy
> >> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
> >> >>>>>> rewrite the
> >> >>>>>>>>>>> code
> >> >>>>>>>>>>>>>>>>>> using
> >> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >> >>> interface
> >> >>>>>> (to
> >> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
> >> >>>>>> but it gives
> >> >>>>>>>>>>>> back
> >> >>>>>>>>>>>>>>>>>> only
> >> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> >> >> looks
> >> >>>>>> ugly
> >> >>>>>>>>>>>> (lambdas
> >> >>>>>>>>>>>>>>>>>> can't
> >> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
> >> >>>>>> optimize the
> >> >>>>>>>>>>>> current
> >> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
> >> >>>>>> Andrey Gura,
> >> >>>>>>>>>>> as
> >> >>>>>>>>>>>> the
> >> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
> >> >>> about
> >> >>>>>>>>>>>> optimization?
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
> >> >>>>>> some metrics
> >> >>>>>>>>>>>>>>>>>> already
> >> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
> >> >>>>>> factory
> >> >>>>>>>>>>>>>>>>>> implementation
> >> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >> >>>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> Alexey,
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> >> >>>>>> already released
> >> >>>>>>>>>>>> in
> >> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> >> >>>>>> only present
> >> >>>>>>>>>>>> in Ignite
> >> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> >> >>>>>> metrics
> >> >>>>>>>>>>>> created for
> >> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
> >> >>>>>> the release
> >> >>>>>>>>>>>> and deal
> >> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >> >>>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>>>>
> >> >>>>>>>>>>>>>
> >> >>>>>>>>>>>>
> >> >>>>>>>>>>>
> >> >>>>>>
> >> >>>>>
> >> >>>
> >> >>
> >>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Alexey,
Thank you for your email. Yes, I can certainly use help in the release
process. I am thinking around the month of Nov for the releases after
the In-Memory Computing Summit Virtual 2020 but any of the module can be
released before that timeline. I have a tech talk coming up on Oct 28 and I
may get a little occupied for the same.
https://www.imcsummit.org/2020/virtual/agenda/schedule/day-1
Also, I have not created any ticket for the OSGI module, please feel free
to create a ticket and I can help with the migration.
Regards,
Saikat
On Tue, Oct 6, 2020 at 9:19 AM Alexey Goncharuk <al...@gmail.com>
wrote:
> Saikat,
>
> The plan sounds good to me! Do you have an idea for the timeline of the
> module releases? Let me know if I can help in any way.
>
> Also, I was looking into the modularization IEP and noticed that OSGI
> module is discussed to be moved to the extensions, but there is no
> corresponding ticket. Should I create one? I will be happy to help with
> moving this module to extensions.
>
> вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
>
>> Hi Alexey,
>>
>> All the modules as planned in IEP-36
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
>> are migrated to ignite-extensions repository.
>>
>> We can definitely plan to release ignite-extensions modules.
>>
>> I have a pending PR related to the kafka module and the PR is in review
>> https://github.com/apache/ignite/pull/8222 but its corresponding PR has
>> been merged in ignite-extensions repository.
>>
>> My thoughts / action items for the migration efforts and releases were
>> as follows:
>>
>> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
>> 2. Fix the upload nightly packages task for ignite-core to help fix the
>> travis build failure in ignite-extensions repository.
>> 3. Review and update the README.md files for the ignite-extensions
>> modules as required.
>> 4. Update the docs for ignite-extensions modules in ignite-website.
>> 5. Release each module separately and share updates.
>>
>> Please let me know if you have feedback.
>>
>> Regards,
>> Saikat
>>
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
>>
>>> It seems that gitbox.apache.org is not available on CI/CD.
>>>
>>> I will try to update VCS to GitHub.
>>>
>>>
>>>
>>>
>>>
>>> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>>> wrote:
>>> >
>>> > Guys,
>>> >
>>> > I've tried to build release candidate using TC [1]. But:
>>> > 1. I can't choose a branch in this TC task (there is no such field).
>>> > 2. On the "default" branch I got an error: Failed to collect changes,
>>> > error: List remote refs failed:
>>> > org.apache.http.conn.ConnectTimeoutException: Connect to
>>> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
>>> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>>> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>>> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>>> >
>>> > Is there some problem with this TC task? What am I doing wrong?
>>> >
>>> > [1] :
>>> >
>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>>> >
>>> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com>:
>>> >
>>> >> Saikat, Nikolay,
>>> >>
>>> >> We have migrated a bunch of modules to ignite-extensions recently.
>>> Given
>>> >> that these modules will not be available in Ignite 2.9 anymore (will
>>> >> they?), should we also plan to release the extensions?
>>> >>
>>> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <plehanov.alex@gmail.com
>>> >:
>>> >>
>>> >>> Igniters,
>>> >>>
>>> >>> I've prepared release notes for the 2.9 release [1]. Can anyone
>>> review
>>> >> it,
>>> >>> please?
>>> >>>
>>> >>> [1]: https://github.com/apache/ignite/pull/8273
>>> >>>
>>> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <plehanov.alex@gmail.com
>>> >:
>>> >>>
>>> >>>> Guys,
>>> >>>>
>>> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
>>> This
>>> >>> bug
>>> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>>> and
>>> >>>> decided to revert this ticket. I will do it today (after TC bot
>>> visa)
>>> >> if
>>> >>>> there are no objections.
>>> >>>>
>>> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>> >>>>
>>> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <
>>> plehanov.alex@gmail.com>:
>>> >>>>
>>> >>>>> Guys,
>>> >>>>>
>>> >>>>> During internal testing, we've found a critical bug with
>>> >>>>> discovery (cluster falls apart if two nodes segmented
>>> sequentially).
>>> >>> This
>>> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>> >>>>> before release. Under investigation now. I'll let you know when we
>>> get
>>> >>>>> something.
>>> >>>>>
>>> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>> >>>>>
>>> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
>>> of
>>> >>> the
>>> >>>>>> message factory in 2.9 and go for the runtime message generation
>>> in
>>> >>> later
>>> >>>>>> release, or keep the code clean and fix the regression in the next
>>> >>> releases?
>>> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>> >>>>>> straightforward and does not change the semantics, just skips the
>>> >>> factory
>>> >>>>>> closures for certain messages.
>>> >>>>>>
>>> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for
>>> all
>>> >>>>>> workloads (I didn't get any significant drops during
>>> benchmarking).
>>> >> So
>>> >>>>>> I prefer the runtime generation in later releases.
>>> >>>>>>
>>> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>> >>>>>> <al...@gmail.com> wrote:
>>> >>>>>>>
>>> >>>>>>> Alexey, Andrey, Igniters,
>>> >>>>>>>
>>> >>>>>>> So what do you think? Should we proceed with a 'hacked' version
>>> of
>>> >>> the
>>> >>>>>> message factory in 2.9 and go for the runtime message generation
>>> in
>>> >>> later
>>> >>>>>> release, or keep the code clean and fix the regression in the next
>>> >>> releases?
>>> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>> >>>>>> straightforward and does not change the semantics, just skips the
>>> >>> factory
>>> >>>>>> closures for certain messages.
>>> >>>>>>>
>>> >>>>>>> Personally, I would prefer fixing the regression given that we
>>> also
>>> >>>>>> introduced tracing in this release.
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>>
>>> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>>> >> plehanov.alex@gmail.com
>>> >>>> :
>>> >>>>>>>>
>>> >>>>>>>> Alexey,
>>> >>>>>>>>
>>> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs
>>> ed52559eb95
>>> >>> and
>>> >>>>>> the performance of ed52559eb95 is better for about 2.5% on
>>> average on
>>> >>> our
>>> >>>>>> environment (about the same results we got benchmarking 65c30ec6
>>> vs
>>> >>>>>> 0606f03d). We've made 24 runs for each commit of
>>> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>>> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers,
>>> 5
>>> >>> clients
>>> >>>>>> 50 threads each. Yardstick results for this configuration:
>>> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>> >>>>>>>>
>>> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>> >>>>>> a.budnikov.ignite@gmail.com>:
>>> >>>>>>>>>
>>> >>>>>>>>> Hi Everyone,
>>> >>>>>>>>>
>>> >>>>>>>>> I posted an instruction on how to publish the docs on
>>> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
>>> can
>>> >>>>>> update the docs by following the instruction. Unfortunately, I
>>> won't
>>> >> be
>>> >>>>>> able to spend any time on this project any longer. You can send
>>> your
>>> >>> pull
>>> >>>>>> requests and questions about the documentation to Denis Magda.
>>> >>>>>>>>>
>>> >>>>>>>>> -Artem
>>> >>>>>>>>>
>>> >>>>>>>>> [1] :
>>> >>>>>>
>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>> >>>>>>>>>
>>> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>> >>>>>> alexey.goncharuk@gmail.com> wrote:
>>> >>>>>>>>>>
>>> >>>>>>>>>> Alexey,
>>> >>>>>>>>>>
>>> >>>>>>>>>> I've tried to play with message factories locally, but
>>> >>>>>> unfortunately, I
>>> >>>>>>>>>> cannot spot the difference between old and new implementation
>>> in
>>> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>> >>>>>> MessageFactoryImpl
>>> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>>> >>> branch
>>> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>>> >> compatible
>>> >>>>>> with the
>>> >>>>>>>>>> new metrics as we still use the register() mechanics).
>>> >>>>>>>>>>
>>> >>>>>>>>>> Can you check if this change makes any difference
>>> >> performance-wise
>>> >>>>>> in your
>>> >>>>>>>>>> environment? If yes, we can go with runtime code generation in
>>> >> the
>>> >>>>>> long
>>> >>>>>>>>>> term: register classes and generate a dynamic message factory
>>> >> with
>>> >>>>>> a switch
>>> >>>>>>>>>> statement once all messages are registered (not in 2.9 though,
>>> >>>>>> obviously).
>>> >>>>>>>>>>
>>> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>> >>> plehanov.alex@gmail.com
>>> >>>>>>> :
>>> >>>>>>>>>>
>>> >>>>>>>>>>> Hello guys,
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]),
>>> it
>>> >>>>>> reduced the
>>> >>>>>>>>>>> drop, but not completely removed it.
>>> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the patch?
>>> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2]
>>> against
>>> >>>>>> 2.8.1
>>> >>>>>>>>>>> release on your environment?
>>> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop
>>> left,
>>> >>>>>> it's close
>>> >>>>>>>>>>> to measurement error and I think such a drop is not a
>>> >>>>>> showstopper. Guys,
>>> >>>>>>>>>>> WDYT?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>>> >> driver
>>> >>>>>> between 2.8
>>> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>>> >> should
>>> >>>>>> be
>>> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>> >>> message
>>> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it if
>>> >>>>>> there is no
>>> >>>>>>>>>>> objection.
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>> >>>>>>>>>>>
>>> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>>> >> mmuzaf@apache.org
>>> >>>> :
>>> >>>>>>>>>>>
>>> >>>>>>>>>>>> Alexey,
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>>> >> this
>>> >>>>>> issue is
>>> >>>>>>>>>>>> related to the new master key change functionality which
>>> >>>>>> haven't been
>>> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>>> >> to
>>> >>>>>> the
>>> >>>>>>>>>>>> release branch.
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>> >>>>>> nizhikov@apache.org>
>>> >>>>>>>>>>> wrote:
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Hello, Igniters.
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>>> >> [1]
>>> >>>>>> into the
>>> >>>>>>>>>>> 2.9
>>> >>>>>>>>>>>> release
>>> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
>>> >>> in
>>> >>>>>> wrong
>>> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>> >>>>>>>>>>>>> [2]
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>
>>> >>>
>>> >>
>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>>> >>>>>>>>>>>> написал(а):
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>> >>>>>> anything out of
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>>> >>>>>> right now),
>>> >>>>>>>>>>> so
>>> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>> >>>>>> switch
>>> >>>>>>>>>>> approach
>>> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>> >>> the
>>> >>>>>> Ignite
>>> >>>>>>>>>>>> messages
>>> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>>> >>> me
>>> >>>>>> explore
>>> >>>>>>>>>>> the
>>> >>>>>>>>>>>>>> code a bit.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>>> >> the
>>> >>>>>>>>>>> performance.
>>> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>>> >>>>>> virtual call
>>> >>>>>>>>>>>>>> should not make that much of a difference given the
>>> >> amount
>>> >>>>>> of other
>>> >>>>>>>>>>>> work
>>> >>>>>>>>>>>>>> happening during the message processing.
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>> >>>>>> plehanov.alex@gmail.com
>>> >>>>>>>>>>>> :
>>> >>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>> >>> results.
>>> >>>>>>>>>>> Actually,
>>> >>>>>>>>>>>> we
>>> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>> >>> between
>>> >>>>>> e6a7f93
>>> >>>>>>>>>>>> (first
>>> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>> >>>>>> 6592dfa5 (last
>>> >>>>>>>>>>>> commit in
>>> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>> >>>>>> commits.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>>> >> a
>>> >>>>>> drop
>>> >>>>>>>>>>> between
>>> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>> >>>>>> reverted
>>> >>>>>>>>>>>> IGNITE-13060
>>> >>>>>>>>>>>>>>> now and performance looks the same)
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>> >>>>>> related to
>>> >>>>>>>>>>>> drop
>>> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>> >>>>>> problem, and
>>> >>>>>>>>>>> we
>>> >>>>>>>>>>>> can
>>> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>>> >> leave
>>> >>>>>> it as is?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>> >>>>>> refactoring)?
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>> >>>>>>>>>>>>>>>> :
>>> >>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> Alexey,
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>> >>>>>> incorrect fix
>>> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>>> >>>>>> [1], so it
>>> >>>>>>>>>>>> cannot be
>>> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>>> >> work
>>> >>>>>> with fix
>>> >>>>>>>>>>>> versions
>>> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>> >>>>>> versions.
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> --AG
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> [1]
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>
>>> >>>
>>> >>
>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>>> >>>>>>>>>>>>>>>>> :
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>> >>>>>>>>>>> plehanov.alex@gmail.com
>>> >>>>>>>>>>>>> :
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Guys,
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>> >>>>>> IGNITE-12568
>>> >>>>>>>>>>>> (reverted
>>> >>>>>>>>>>>>>>>>>> it
>>> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>> >>>>>> paths, to
>>> >>>>>>>>>>> trace
>>> >>>>>>>>>>>>>>>>>> these
>>> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>> >>> here.
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>> >>> switch/case
>>> >>>>>> block was
>>> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>> >>>>>> message factory
>>> >>>>>>>>>>>> is on
>>> >>>>>>>>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>>> >> impact
>>> >>>>>> on total
>>> >>>>>>>>>>>>>>>>>> performance.
>>> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>> >>>>>> microbenchmarks,
>>> >>>>>>>>>>>> and
>>> >>>>>>>>>>>>>>>>>> found
>>> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>> >>>>>> about 30-35%
>>> >>>>>>>>>>>> faster
>>> >>>>>>>>>>>>>>>>>> than
>>> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>> >>> can
>>> >>>>>>>>>>> effectively
>>> >>>>>>>>>>>>>>>>>> inline
>>> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>>> >> suppliers
>>> >>>>>> relatively
>>> >>>>>>>>>>>> heavy
>>> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>> >>>>>> rewrite the
>>> >>>>>>>>>>> code
>>> >>>>>>>>>>>>>>>>>> using
>>> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>> >>> interface
>>> >>>>>> (to
>>> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>>> >>>>>> but it gives
>>> >>>>>>>>>>>> back
>>> >>>>>>>>>>>>>>>>>> only
>>> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>>> >> looks
>>> >>>>>> ugly
>>> >>>>>>>>>>>> (lambdas
>>> >>>>>>>>>>>>>>>>>> can't
>>> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>> >>>>>> optimize the
>>> >>>>>>>>>>>> current
>>> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>> >>>>>> Andrey Gura,
>>> >>>>>>>>>>> as
>>> >>>>>>>>>>>> the
>>> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>> >>> about
>>> >>>>>>>>>>>> optimization?
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>>> >>>>>> some metrics
>>> >>>>>>>>>>>>>>>>>> already
>>> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>> >>>>>> factory
>>> >>>>>>>>>>>>>>>>>> implementation
>>> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>> >>>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> Alexey,
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>> >>>>>> already released
>>> >>>>>>>>>>>> in
>>> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>> >>>>>> only present
>>> >>>>>>>>>>>> in Ignite
>>> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>> >>>>>> metrics
>>> >>>>>>>>>>>> created for
>>> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>>> >>>>>> the release
>>> >>>>>>>>>>>> and deal
>>> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>> >>>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>>>>
>>> >>>>>>>>>>>>>
>>> >>>>>>>>>>>>
>>> >>>>>>>>>>>
>>> >>>>>>
>>> >>>>>
>>> >>>
>>> >>
>>>
>>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Saikat,
The plan sounds good to me! Do you have an idea for the timeline of the
module releases? Let me know if I can help in any way.
Also, I was looking into the modularization IEP and noticed that OSGI
module is discussed to be moved to the extensions, but there is no
corresponding ticket. Should I create one? I will be happy to help with
moving this module to extensions.
вт, 29 сент. 2020 г. в 03:03, Saikat Maitra <sa...@gmail.com>:
> Hi Alexey,
>
> All the modules as planned in IEP-36
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
> are migrated to ignite-extensions repository.
>
> We can definitely plan to release ignite-extensions modules.
>
> I have a pending PR related to the kafka module and the PR is in review
> https://github.com/apache/ignite/pull/8222 but its corresponding PR has
> been merged in ignite-extensions repository.
>
> My thoughts / action items for the migration efforts and releases were
> as follows:
>
> 1. Merge the changes for https://github.com/apache/ignite/pull/8222
> 2. Fix the upload nightly packages task for ignite-core to help fix the
> travis build failure in ignite-extensions repository.
> 3. Review and update the README.md files for the ignite-extensions modules
> as required.
> 4. Update the docs for ignite-extensions modules in ignite-website.
> 5. Release each module separately and share updates.
>
> Please let me know if you have feedback.
>
> Regards,
> Saikat
>
>
>
>
>
>
>
>
> On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
>
>> It seems that gitbox.apache.org is not available on CI/CD.
>>
>> I will try to update VCS to GitHub.
>>
>>
>>
>>
>>
>> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com>
>> wrote:
>> >
>> > Guys,
>> >
>> > I've tried to build release candidate using TC [1]. But:
>> > 1. I can't choose a branch in this TC task (there is no such field).
>> > 2. On the "default" branch I got an error: Failed to collect changes,
>> > error: List remote refs failed:
>> > org.apache.http.conn.ConnectTimeoutException: Connect to
>> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
>> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
>> > internal id=74, parent id=GitBoxAsfIgnite, description: "
>> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>> >
>> > Is there some problem with this TC task? What am I doing wrong?
>> >
>> > [1] :
>> >
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>> >
>> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
>> alexey.goncharuk@gmail.com>:
>> >
>> >> Saikat, Nikolay,
>> >>
>> >> We have migrated a bunch of modules to ignite-extensions recently.
>> Given
>> >> that these modules will not be available in Ignite 2.9 anymore (will
>> >> they?), should we also plan to release the extensions?
>> >>
>> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <pl...@gmail.com>:
>> >>
>> >>> Igniters,
>> >>>
>> >>> I've prepared release notes for the 2.9 release [1]. Can anyone review
>> >> it,
>> >>> please?
>> >>>
>> >>> [1]: https://github.com/apache/ignite/pull/8273
>> >>>
>> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <plehanov.alex@gmail.com
>> >:
>> >>>
>> >>>> Guys,
>> >>>>
>> >>>> I've filled the ticket with reproducer [1] for the discovery bug.
>> This
>> >>> bug
>> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately
>> and
>> >>>> decided to revert this ticket. I will do it today (after TC bot visa)
>> >> if
>> >>>> there are no objections.
>> >>>>
>> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>> >>>>
>> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <plehanov.alex@gmail.com
>> >:
>> >>>>
>> >>>>> Guys,
>> >>>>>
>> >>>>> During internal testing, we've found a critical bug with
>> >>>>> discovery (cluster falls apart if two nodes segmented sequentially).
>> >>> This
>> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
>> >>>>> before release. Under investigation now. I'll let you know when we
>> get
>> >>>>> something.
>> >>>>>
>> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>> >>>>>
>> >>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>> >>> the
>> >>>>>> message factory in 2.9 and go for the runtime message generation in
>> >>> later
>> >>>>>> release, or keep the code clean and fix the regression in the next
>> >>> releases?
>> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >>>>>> straightforward and does not change the semantics, just skips the
>> >>> factory
>> >>>>>> closures for certain messages.
>> >>>>>>
>> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for all
>> >>>>>> workloads (I didn't get any significant drops during benchmarking).
>> >> So
>> >>>>>> I prefer the runtime generation in later releases.
>> >>>>>>
>> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>> >>>>>> <al...@gmail.com> wrote:
>> >>>>>>>
>> >>>>>>> Alexey, Andrey, Igniters,
>> >>>>>>>
>> >>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>> >>> the
>> >>>>>> message factory in 2.9 and go for the runtime message generation in
>> >>> later
>> >>>>>> release, or keep the code clean and fix the regression in the next
>> >>> releases?
>> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
>> >>>>>> straightforward and does not change the semantics, just skips the
>> >>> factory
>> >>>>>> closures for certain messages.
>> >>>>>>>
>> >>>>>>> Personally, I would prefer fixing the regression given that we
>> also
>> >>>>>> introduced tracing in this release.
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> >> plehanov.alex@gmail.com
>> >>>> :
>> >>>>>>>>
>> >>>>>>>> Alexey,
>> >>>>>>>>
>> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
>> >>> and
>> >>>>>> the performance of ed52559eb95 is better for about 2.5% on average
>> on
>> >>> our
>> >>>>>> environment (about the same results we got benchmarking 65c30ec6 vs
>> >>>>>> 0606f03d). We've made 24 runs for each commit of
>> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
>> >>> clients
>> >>>>>> 50 threads each. Yardstick results for this configuration:
>> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>> >>>>>>>>
>> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>> >>>>>> a.budnikov.ignite@gmail.com>:
>> >>>>>>>>>
>> >>>>>>>>> Hi Everyone,
>> >>>>>>>>>
>> >>>>>>>>> I posted an instruction on how to publish the docs on
>> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
>> can
>> >>>>>> update the docs by following the instruction. Unfortunately, I
>> won't
>> >> be
>> >>>>>> able to spend any time on this project any longer. You can send
>> your
>> >>> pull
>> >>>>>> requests and questions about the documentation to Denis Magda.
>> >>>>>>>>>
>> >>>>>>>>> -Artem
>> >>>>>>>>>
>> >>>>>>>>> [1] :
>> >>>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>> >>>>>>>>>
>> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> >>>>>> alexey.goncharuk@gmail.com> wrote:
>> >>>>>>>>>>
>> >>>>>>>>>> Alexey,
>> >>>>>>>>>>
>> >>>>>>>>>> I've tried to play with message factories locally, but
>> >>>>>> unfortunately, I
>> >>>>>>>>>> cannot spot the difference between old and new implementation
>> in
>> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
>> >>>>>> MessageFactoryImpl
>> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>> >>> branch
>> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> >> compatible
>> >>>>>> with the
>> >>>>>>>>>> new metrics as we still use the register() mechanics).
>> >>>>>>>>>>
>> >>>>>>>>>> Can you check if this change makes any difference
>> >> performance-wise
>> >>>>>> in your
>> >>>>>>>>>> environment? If yes, we can go with runtime code generation in
>> >> the
>> >>>>>> long
>> >>>>>>>>>> term: register classes and generate a dynamic message factory
>> >> with
>> >>>>>> a switch
>> >>>>>>>>>> statement once all messages are registered (not in 2.9 though,
>> >>>>>> obviously).
>> >>>>>>>>>>
>> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>> >>> plehanov.alex@gmail.com
>> >>>>>>> :
>> >>>>>>>>>>
>> >>>>>>>>>>> Hello guys,
>> >>>>>>>>>>>
>> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]), it
>> >>>>>> reduced the
>> >>>>>>>>>>> drop, but not completely removed it.
>> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the patch?
>> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2] against
>> >>>>>> 2.8.1
>> >>>>>>>>>>> release on your environment?
>> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop left,
>> >>>>>> it's close
>> >>>>>>>>>>> to measurement error and I think such a drop is not a
>> >>>>>> showstopper. Guys,
>> >>>>>>>>>>> WDYT?
>> >>>>>>>>>>>
>> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> >> driver
>> >>>>>> between 2.8
>> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> >> should
>> >>>>>> be
>> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
>> >>>>>>>>>>>
>> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>> >>> message
>> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it if
>> >>>>>> there is no
>> >>>>>>>>>>> objection.
>> >>>>>>>>>>>
>> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >>>>>>>>>>>
>> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> >> mmuzaf@apache.org
>> >>>> :
>> >>>>>>>>>>>
>> >>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> >> this
>> >>>>>> issue is
>> >>>>>>>>>>>> related to the new master key change functionality which
>> >>>>>> haven't been
>> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>> >> to
>> >>>>>> the
>> >>>>>>>>>>>> release branch.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>> >>>>>> nizhikov@apache.org>
>> >>>>>>>>>>> wrote:
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Hello, Igniters.
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> >> [1]
>> >>>>>> into the
>> >>>>>>>>>>> 2.9
>> >>>>>>>>>>>> release
>> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
>> >>> in
>> >>>>>> wrong
>> >>>>>>>>>>>> order since the 2 row when fields_count>10"
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> >>>>>>>>>>>>> [2]
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>
>> >>>
>> >>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> >>>>>>>>>>> alexey.goncharuk@gmail.com>
>> >>>>>>>>>>>> написал(а):
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>> >>>>>> anything out of
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>> >>>>>> right now),
>> >>>>>>>>>>> so
>> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
>> >>>>>> switch
>> >>>>>>>>>>> approach
>> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>> >>> the
>> >>>>>> Ignite
>> >>>>>>>>>>>> messages
>> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>> >>> me
>> >>>>>> explore
>> >>>>>>>>>>> the
>> >>>>>>>>>>>>>> code a bit.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> >> the
>> >>>>>>>>>>> performance.
>> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>> >>>>>> virtual call
>> >>>>>>>>>>>>>> should not make that much of a difference given the
>> >> amount
>> >>>>>> of other
>> >>>>>>>>>>>> work
>> >>>>>>>>>>>>>> happening during the message processing.
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> >>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>>>> :
>> >>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>> >>> results.
>> >>>>>>>>>>> Actually,
>> >>>>>>>>>>>> we
>> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>> >>> between
>> >>>>>> e6a7f93
>> >>>>>>>>>>>> (first
>> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>> >>>>>> 6592dfa5 (last
>> >>>>>>>>>>>> commit in
>> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>> >>>>>> commits.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>> >> a
>> >>>>>> drop
>> >>>>>>>>>>> between
>> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>> >>>>>> reverted
>> >>>>>>>>>>>> IGNITE-13060
>> >>>>>>>>>>>>>>> now and performance looks the same)
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>> >>>>>> related to
>> >>>>>>>>>>>> drop
>> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>> >>>>>> problem, and
>> >>>>>>>>>>> we
>> >>>>>>>>>>>> can
>> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> >> leave
>> >>>>>> it as is?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>> >>>>>> refactoring)?
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>> >>>>>> incorrect fix
>> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>> >>>>>> [1], so it
>> >>>>>>>>>>>> cannot be
>> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> >> work
>> >>>>>> with fix
>> >>>>>>>>>>>> versions
>> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>> >>>>>> versions.
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> --AG
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> [1]
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>
>> >>>
>> >>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> >>>>>>>>>>>> alexey.goncharuk@gmail.com
>> >>>>>>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> >>>>>>>>>>> plehanov.alex@gmail.com
>> >>>>>>>>>>>>> :
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Guys,
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>> >>>>>> IGNITE-12568
>> >>>>>>>>>>>> (reverted
>> >>>>>>>>>>>>>>>>>> it
>> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>> >>>>>> paths, to
>> >>>>>>>>>>> trace
>> >>>>>>>>>>>>>>>>>> these
>> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>> >>> here.
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>> >>> switch/case
>> >>>>>> block was
>> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>> >>>>>> message factory
>> >>>>>>>>>>>> is on
>> >>>>>>>>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> >> impact
>> >>>>>> on total
>> >>>>>>>>>>>>>>>>>> performance.
>> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>> >>>>>> microbenchmarks,
>> >>>>>>>>>>>> and
>> >>>>>>>>>>>>>>>>>> found
>> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>> >>>>>> about 30-35%
>> >>>>>>>>>>>> faster
>> >>>>>>>>>>>>>>>>>> than
>> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>> >>> can
>> >>>>>>>>>>> effectively
>> >>>>>>>>>>>>>>>>>> inline
>> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> >> suppliers
>> >>>>>> relatively
>> >>>>>>>>>>>> heavy
>> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>> >>>>>> rewrite the
>> >>>>>>>>>>> code
>> >>>>>>>>>>>>>>>>>> using
>> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>> >>> interface
>> >>>>>> (to
>> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>> >>>>>> but it gives
>> >>>>>>>>>>>> back
>> >>>>>>>>>>>>>>>>>> only
>> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> >> looks
>> >>>>>> ugly
>> >>>>>>>>>>>> (lambdas
>> >>>>>>>>>>>>>>>>>> can't
>> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>> >>>>>> optimize the
>> >>>>>>>>>>>> current
>> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>> >>>>>> Andrey Gura,
>> >>>>>>>>>>> as
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>> >>> about
>> >>>>>>>>>>>> optimization?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>> >>>>>> some metrics
>> >>>>>>>>>>>>>>>>>> already
>> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>> >>>>>> factory
>> >>>>>>>>>>>>>>>>>> implementation
>> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>> >>>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> Alexey,
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>> >>>>>> already released
>> >>>>>>>>>>>> in
>> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>> >>>>>> only present
>> >>>>>>>>>>>> in Ignite
>> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>> >>>>>> metrics
>> >>>>>>>>>>>> created for
>> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>> >>>>>> the release
>> >>>>>>>>>>>> and deal
>> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>> >>>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>>>>
>> >>>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>
>> >>>>>>
>> >>>>>
>> >>>
>> >>
>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Saikat Maitra <sa...@gmail.com>.
Hi Alexey,
All the modules as planned in IEP-36
https://cwiki.apache.org/confluence/display/IGNITE/IEP-36:+Modularization
are migrated to ignite-extensions repository.
We can definitely plan to release ignite-extensions modules.
I have a pending PR related to the kafka module and the PR is in review
https://github.com/apache/ignite/pull/8222 but its corresponding PR has
been merged in ignite-extensions repository.
My thoughts / action items for the migration efforts and releases were
as follows:
1. Merge the changes for https://github.com/apache/ignite/pull/8222
2. Fix the upload nightly packages task for ignite-core to help fix the
travis build failure in ignite-extensions repository.
3. Review and update the README.md files for the ignite-extensions modules
as required.
4. Update the docs for ignite-extensions modules in ignite-website.
5. Release each module separately and share updates.
Please let me know if you have feedback.
Regards,
Saikat
On Mon, Sep 28, 2020 at 7:36 AM Petr Ivanov <mr...@gmail.com> wrote:
> It seems that gitbox.apache.org is not available on CI/CD.
>
> I will try to update VCS to GitHub.
>
>
>
>
>
> > On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com> wrote:
> >
> > Guys,
> >
> > I've tried to build release candidate using TC [1]. But:
> > 1. I can't choose a branch in this TC task (there is no such field).
> > 2. On the "default" branch I got an error: Failed to collect changes,
> > error: List remote refs failed:
> > org.apache.http.conn.ConnectTimeoutException: Connect to
> > gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
> > timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> > internal id=74, parent id=GitBoxAsfIgnite, description: "
> > https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
> >
> > Is there some problem with this TC task? What am I doing wrong?
> >
> > [1] :
> >
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
> >
> > пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <
> alexey.goncharuk@gmail.com>:
> >
> >> Saikat, Nikolay,
> >>
> >> We have migrated a bunch of modules to ignite-extensions recently. Given
> >> that these modules will not be available in Ignite 2.9 anymore (will
> >> they?), should we also plan to release the extensions?
> >>
> >> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <pl...@gmail.com>:
> >>
> >>> Igniters,
> >>>
> >>> I've prepared release notes for the 2.9 release [1]. Can anyone review
> >> it,
> >>> please?
> >>>
> >>> [1]: https://github.com/apache/ignite/pull/8273
> >>>
> >>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <pl...@gmail.com>:
> >>>
> >>>> Guys,
> >>>>
> >>>> I've filled the ticket with reproducer [1] for the discovery bug. This
> >>> bug
> >>>> caused by [2] ticket. We discussed with Vladimir Steshin privately and
> >>>> decided to revert this ticket. I will do it today (after TC bot visa)
> >> if
> >>>> there are no objections.
> >>>>
> >>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> >>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >>>>
> >>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >>>>
> >>>>> Guys,
> >>>>>
> >>>>> During internal testing, we've found a critical bug with
> >>>>> discovery (cluster falls apart if two nodes segmented sequentially).
> >>> This
> >>>>> problem is not reproduced in 2.8.1. I think we should fix it
> >>>>> before release. Under investigation now. I'll let you know when we
> get
> >>>>> something.
> >>>>>
> >>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >>>>>
> >>>>>>> So what do you think? Should we proceed with a 'hacked' version of
> >>> the
> >>>>>> message factory in 2.9 and go for the runtime message generation in
> >>> later
> >>>>>> release, or keep the code clean and fix the regression in the next
> >>> releases?
> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >>>>>> straightforward and does not change the semantics, just skips the
> >>> factory
> >>>>>> closures for certain messages.
> >>>>>>
> >>>>>> IMHO 2.5% isn't too much especially because it isn't actual for all
> >>>>>> workloads (I didn't get any significant drops during benchmarking).
> >> So
> >>>>>> I prefer the runtime generation in later releases.
> >>>>>>
> >>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >>>>>> <al...@gmail.com> wrote:
> >>>>>>>
> >>>>>>> Alexey, Andrey, Igniters,
> >>>>>>>
> >>>>>>> So what do you think? Should we proceed with a 'hacked' version of
> >>> the
> >>>>>> message factory in 2.9 and go for the runtime message generation in
> >>> later
> >>>>>> release, or keep the code clean and fix the regression in the next
> >>> releases?
> >>>>>>> Andrey, can you take a look at my change? I think it is fairly
> >>>>>> straightforward and does not change the semantics, just skips the
> >>> factory
> >>>>>> closures for certain messages.
> >>>>>>>
> >>>>>>> Personally, I would prefer fixing the regression given that we also
> >>>>>> introduced tracing in this release.
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> >> plehanov.alex@gmail.com
> >>>> :
> >>>>>>>>
> >>>>>>>> Alexey,
> >>>>>>>>
> >>>>>>>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
> >>> and
> >>>>>> the performance of ed52559eb95 is better for about 2.5% on average
> on
> >>> our
> >>>>>> environment (about the same results we got benchmarking 65c30ec6 vs
> >>>>>> 0606f03d). We've made 24 runs for each commit of
> >>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> >>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
> >>> clients
> >>>>>> 50 threads each. Yardstick results for this configuration:
> >>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>>>>>>>
> >>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >>>>>> a.budnikov.ignite@gmail.com>:
> >>>>>>>>>
> >>>>>>>>> Hi Everyone,
> >>>>>>>>>
> >>>>>>>>> I posted an instruction on how to publish the docs on
> >>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you
> can
> >>>>>> update the docs by following the instruction. Unfortunately, I won't
> >> be
> >>>>>> able to spend any time on this project any longer. You can send your
> >>> pull
> >>>>>> requests and questions about the documentation to Denis Magda.
> >>>>>>>>>
> >>>>>>>>> -Artem
> >>>>>>>>>
> >>>>>>>>> [1] :
> >>>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>>>>>>>>
> >>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >>>>>> alexey.goncharuk@gmail.com> wrote:
> >>>>>>>>>>
> >>>>>>>>>> Alexey,
> >>>>>>>>>>
> >>>>>>>>>> I've tried to play with message factories locally, but
> >>>>>> unfortunately, I
> >>>>>>>>>> cannot spot the difference between old and new implementation in
> >>>>>>>>>> distributed benchmarks. I pushed an implementation of
> >>>>>> MessageFactoryImpl
> >>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
> >>> branch
> >>>>>>>>>> (discussed this with Andrey Gura, the change should be
> >> compatible
> >>>>>> with the
> >>>>>>>>>> new metrics as we still use the register() mechanics).
> >>>>>>>>>>
> >>>>>>>>>> Can you check if this change makes any difference
> >> performance-wise
> >>>>>> in your
> >>>>>>>>>> environment? If yes, we can go with runtime code generation in
> >> the
> >>>>>> long
> >>>>>>>>>> term: register classes and generate a dynamic message factory
> >> with
> >>>>>> a switch
> >>>>>>>>>> statement once all messages are registered (not in 2.9 though,
> >>>>>> obviously).
> >>>>>>>>>>
> >>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> >>> plehanov.alex@gmail.com
> >>>>>>> :
> >>>>>>>>>>
> >>>>>>>>>>> Hello guys,
> >>>>>>>>>>>
> >>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]), it
> >>>>>> reduced the
> >>>>>>>>>>> drop, but not completely removed it.
> >>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the patch?
> >>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2] against
> >>>>>> 2.8.1
> >>>>>>>>>>> release on your environment?
> >>>>>>>>>>> With this patch on our environment, it's about a 3% drop left,
> >>>>>> it's close
> >>>>>>>>>>> to measurement error and I think such a drop is not a
> >>>>>> showstopper. Guys,
> >>>>>>>>>>> WDYT?
> >>>>>>>>>>>
> >>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
> >> driver
> >>>>>> between 2.8
> >>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
> >> should
> >>>>>> be
> >>>>>>>>>>> fixed in 2.9. I've prepared the patch.
> >>>>>>>>>>> Taras Ledkov, can you please review this patch?
> >>>>>>>>>>>
> >>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
> >>> message
> >>>>>>>>>>> send problem in some circumstances). I will cherry-pick it if
> >>>>>> there is no
> >>>>>>>>>>> objection.
> >>>>>>>>>>>
> >>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
> >>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>>>>>>>>>>
> >>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> >> mmuzaf@apache.org
> >>>> :
> >>>>>>>>>>>
> >>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>
> >>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
> >> this
> >>>>>> issue is
> >>>>>>>>>>>> related to the new master key change functionality which
> >>>>>> haven't been
> >>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
> >> to
> >>>>>> the
> >>>>>>>>>>>> release branch.
> >>>>>>>>>>>>
> >>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >>>>>> nizhikov@apache.org>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Hello, Igniters.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
> >> [1]
> >>>>>> into the
> >>>>>>>>>>> 2.9
> >>>>>>>>>>>> release
> >>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
> >>> in
> >>>>>> wrong
> >>>>>>>>>>>> order since the 2 row when fields_count>10"
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>>>>>>>>>>>> [2]
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> >>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>>>>>>>>>> alexey.goncharuk@gmail.com>
> >>>>>>>>>>>> написал(а):
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
> >>>>>> anything out of
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
> >>>>>> right now),
> >>>>>>>>>>> so
> >>>>>>>>>>>>>> most likely the only move here is to switch back to the
> >>>>>> switch
> >>>>>>>>>>> approach
> >>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
> >>> the
> >>>>>> Ignite
> >>>>>>>>>>>> messages
> >>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
> >>> me
> >>>>>> explore
> >>>>>>>>>>> the
> >>>>>>>>>>>>>> code a bit.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
> >> the
> >>>>>>>>>>> performance.
> >>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
> >>>>>> virtual call
> >>>>>>>>>>>>>> should not make that much of a difference given the
> >> amount
> >>>>>> of other
> >>>>>>>>>>>> work
> >>>>>>>>>>>>>> happening during the message processing.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>> :
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
> >>> results.
> >>>>>>>>>>> Actually,
> >>>>>>>>>>>> we
> >>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
> >>> between
> >>>>>> e6a7f93
> >>>>>>>>>>>> (first
> >>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
> >>>>>> 6592dfa5 (last
> >>>>>>>>>>>> commit in
> >>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
> >>>>>> commits.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
> >> a
> >>>>>> drop
> >>>>>>>>>>> between
> >>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >>>>>> reverted
> >>>>>>>>>>>> IGNITE-13060
> >>>>>>>>>>>>>>> now and performance looks the same)
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >>>>>> related to
> >>>>>>>>>>>> drop
> >>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
> >>>>>> problem, and
> >>>>>>>>>>> we
> >>>>>>>>>>>> can
> >>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
> >> leave
> >>>>>> it as is?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
> >>>>>> refactoring)?
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
> >>>>>> incorrect fix
> >>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
> >>>>>> [1], so it
> >>>>>>>>>>>> cannot be
> >>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
> >> work
> >>>>>> with fix
> >>>>>>>>>>>> versions
> >>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
> >>>>>> versions.
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> --AG
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> [1]
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>
> >>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>>>>>>>>>>> alexey.goncharuk@gmail.com
> >>>>>>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>>>>>>>>>> plehanov.alex@gmail.com
> >>>>>>>>>>>>> :
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Guys,
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >>>>>> IGNITE-12568
> >>>>>>>>>>>> (reverted
> >>>>>>>>>>>>>>>>>> it
> >>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
> >>>>>> paths, to
> >>>>>>>>>>> trace
> >>>>>>>>>>>>>>>>>> these
> >>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
> >>> here.
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
> >>> switch/case
> >>>>>> block was
> >>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
> >>>>>> message factory
> >>>>>>>>>>>> is on
> >>>>>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
> >> impact
> >>>>>> on total
> >>>>>>>>>>>>>>>>>> performance.
> >>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
> >>>>>> microbenchmarks,
> >>>>>>>>>>>> and
> >>>>>>>>>>>>>>>>>> found
> >>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
> >>>>>> about 30-35%
> >>>>>>>>>>>> faster
> >>>>>>>>>>>>>>>>>> than
> >>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
> >>> can
> >>>>>>>>>>> effectively
> >>>>>>>>>>>>>>>>>> inline
> >>>>>>>>>>>>>>>>>> message creation code, but with an array of
> >> suppliers
> >>>>>> relatively
> >>>>>>>>>>>> heavy
> >>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
> >>>>>> rewrite the
> >>>>>>>>>>> code
> >>>>>>>>>>>>>>>>>> using
> >>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
> >>> interface
> >>>>>> (to
> >>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
> >>>>>> but it gives
> >>>>>>>>>>>> back
> >>>>>>>>>>>>>>>>>> only
> >>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
> >> looks
> >>>>>> ugly
> >>>>>>>>>>>> (lambdas
> >>>>>>>>>>>>>>>>>> can't
> >>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
> >>>>>> optimize the
> >>>>>>>>>>>> current
> >>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
> >>>>>> Andrey Gura,
> >>>>>>>>>>> as
> >>>>>>>>>>>> the
> >>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
> >>> about
> >>>>>>>>>>>> optimization?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
> >>>>>> some metrics
> >>>>>>>>>>>>>>>>>> already
> >>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
> >>>>>> factory
> >>>>>>>>>>>>>>>>>> implementation
> >>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
> >>>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> Alexey,
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
> >>>>>> already released
> >>>>>>>>>>>> in
> >>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> >>>>>> only present
> >>>>>>>>>>>> in Ignite
> >>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> >>>>>> metrics
> >>>>>>>>>>>> created for
> >>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
> >>>>>> the release
> >>>>>>>>>>>> and deal
> >>>>>>>>>>>>>>>>> with the optimizations in 2.10?
> >>>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>
> >>>>>
> >>>
> >>
>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Petr Ivanov <mr...@gmail.com>.
It seems that gitbox.apache.org is not available on CI/CD.
I will try to update VCS to GitHub.
> On 28 Sep 2020, at 14:07, Alex Plehanov <pl...@gmail.com> wrote:
>
> Guys,
>
> I've tried to build release candidate using TC [1]. But:
> 1. I can't choose a branch in this TC task (there is no such field).
> 2. On the "default" branch I got an error: Failed to collect changes,
> error: List remote refs failed:
> org.apache.http.conn.ConnectTimeoutException: Connect to
> gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
> timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
> internal id=74, parent id=GitBoxAsfIgnite, description: "
> https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
>
> Is there some problem with this TC task? What am I doing wrong?
>
> [1] :
> https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
>
> пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <al...@gmail.com>:
>
>> Saikat, Nikolay,
>>
>> We have migrated a bunch of modules to ignite-extensions recently. Given
>> that these modules will not be available in Ignite 2.9 anymore (will
>> they?), should we also plan to release the extensions?
>>
>> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <pl...@gmail.com>:
>>
>>> Igniters,
>>>
>>> I've prepared release notes for the 2.9 release [1]. Can anyone review
>> it,
>>> please?
>>>
>>> [1]: https://github.com/apache/ignite/pull/8273
>>>
>>> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <pl...@gmail.com>:
>>>
>>>> Guys,
>>>>
>>>> I've filled the ticket with reproducer [1] for the discovery bug. This
>>> bug
>>>> caused by [2] ticket. We discussed with Vladimir Steshin privately and
>>>> decided to revert this ticket. I will do it today (after TC bot visa)
>> if
>>>> there are no objections.
>>>>
>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>>>>
>>>> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <pl...@gmail.com>:
>>>>
>>>>> Guys,
>>>>>
>>>>> During internal testing, we've found a critical bug with
>>>>> discovery (cluster falls apart if two nodes segmented sequentially).
>>> This
>>>>> problem is not reproduced in 2.8.1. I think we should fix it
>>>>> before release. Under investigation now. I'll let you know when we get
>>>>> something.
>>>>>
>>>>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>>>>
>>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>>> the
>>>>>> message factory in 2.9 and go for the runtime message generation in
>>> later
>>>>>> release, or keep the code clean and fix the regression in the next
>>> releases?
>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>> straightforward and does not change the semantics, just skips the
>>> factory
>>>>>> closures for certain messages.
>>>>>>
>>>>>> IMHO 2.5% isn't too much especially because it isn't actual for all
>>>>>> workloads (I didn't get any significant drops during benchmarking).
>> So
>>>>>> I prefer the runtime generation in later releases.
>>>>>>
>>>>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>>>>> <al...@gmail.com> wrote:
>>>>>>>
>>>>>>> Alexey, Andrey, Igniters,
>>>>>>>
>>>>>>> So what do you think? Should we proceed with a 'hacked' version of
>>> the
>>>>>> message factory in 2.9 and go for the runtime message generation in
>>> later
>>>>>> release, or keep the code clean and fix the regression in the next
>>> releases?
>>>>>>> Andrey, can you take a look at my change? I think it is fairly
>>>>>> straightforward and does not change the semantics, just skips the
>>> factory
>>>>>> closures for certain messages.
>>>>>>>
>>>>>>> Personally, I would prefer fixing the regression given that we also
>>>>>> introduced tracing in this release.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
>> plehanov.alex@gmail.com
>>>> :
>>>>>>>>
>>>>>>>> Alexey,
>>>>>>>>
>>>>>>>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
>>> and
>>>>>> the performance of ed52559eb95 is better for about 2.5% on average on
>>> our
>>>>>> environment (about the same results we got benchmarking 65c30ec6 vs
>>>>>> 0606f03d). We've made 24 runs for each commit of
>>>>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>>>>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
>>> clients
>>>>>> 50 threads each. Yardstick results for this configuration:
>>>>>>>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>>>>>>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>>>>>>>
>>>>>>>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>>>>> a.budnikov.ignite@gmail.com>:
>>>>>>>>>
>>>>>>>>> Hi Everyone,
>>>>>>>>>
>>>>>>>>> I posted an instruction on how to publish the docs on
>>>>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>>>>>> update the docs by following the instruction. Unfortunately, I won't
>> be
>>>>>> able to spend any time on this project any longer. You can send your
>>> pull
>>>>>> requests and questions about the documentation to Denis Magda.
>>>>>>>>>
>>>>>>>>> -Artem
>>>>>>>>>
>>>>>>>>> [1] :
>>>>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>>>>>>>
>>>>>>>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>>>>> alexey.goncharuk@gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>> Alexey,
>>>>>>>>>>
>>>>>>>>>> I've tried to play with message factories locally, but
>>>>>> unfortunately, I
>>>>>>>>>> cannot spot the difference between old and new implementation in
>>>>>>>>>> distributed benchmarks. I pushed an implementation of
>>>>>> MessageFactoryImpl
>>>>>>>>>> with the old switch statement to the ignite-2.9-revert-12568
>>> branch
>>>>>>>>>> (discussed this with Andrey Gura, the change should be
>> compatible
>>>>>> with the
>>>>>>>>>> new metrics as we still use the register() mechanics).
>>>>>>>>>>
>>>>>>>>>> Can you check if this change makes any difference
>> performance-wise
>>>>>> in your
>>>>>>>>>> environment? If yes, we can go with runtime code generation in
>> the
>>>>>> long
>>>>>>>>>> term: register classes and generate a dynamic message factory
>> with
>>>>>> a switch
>>>>>>>>>> statement once all messages are registered (not in 2.9 though,
>>>>>> obviously).
>>>>>>>>>>
>>>>>>>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
>>> plehanov.alex@gmail.com
>>>>>>> :
>>>>>>>>>>
>>>>>>>>>>> Hello guys,
>>>>>>>>>>>
>>>>>>>>>>> I've tried to optimize tracing implementation (ticket [1]), it
>>>>>> reduced the
>>>>>>>>>>> drop, but not completely removed it.
>>>>>>>>>>> Ivan Rakov, Alexander Lapin, can you please review the patch?
>>>>>>>>>>> Ivan Artiukhov, can you please benchmark the patch [2] against
>>>>>> 2.8.1
>>>>>>>>>>> release on your environment?
>>>>>>>>>>> With this patch on our environment, it's about a 3% drop left,
>>>>>> it's close
>>>>>>>>>>> to measurement error and I think such a drop is not a
>>>>>> showstopper. Guys,
>>>>>>>>>>> WDYT?
>>>>>>>>>>>
>>>>>>>>>>> Also, I found that compatibility is broken for JDBC thin
>> driver
>>>>>> between 2.8
>>>>>>>>>>> and 2.9 versions (ticket [3]). I think it's a blocker and
>> should
>>>>>> be
>>>>>>>>>>> fixed in 2.9. I've prepared the patch.
>>>>>>>>>>> Taras Ledkov, can you please review this patch?
>>>>>>>>>>>
>>>>>>>>>>> And one more ticket I propose to include into 2.9 [4] (NIO
>>> message
>>>>>>>>>>> send problem in some circumstances). I will cherry-pick it if
>>>>>> there is no
>>>>>>>>>>> objection.
>>>>>>>>>>>
>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>>>>>>>>> [2] https://github.com/apache/ignite/pull/8223
>>>>>>>>>>> [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>>>>>>>>> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>>>>>>>>>
>>>>>>>>>>> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
>> mmuzaf@apache.org
>>>> :
>>>>>>>>>>>
>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>
>>>>>>>>>>>> I propose to include [1] issue to the 2.9 release. Since
>> this
>>>>>> issue is
>>>>>>>>>>>> related to the new master key change functionality which
>>>>>> haven't been
>>>>>>>>>>>> released yet I think it will be safe to cherry-pick commit
>> to
>>>>>> the
>>>>>>>>>>>> release branch.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>>>>> nizhikov@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hello, Igniters.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alexey, please, include one more Python thin client fix
>> [1]
>>>>>> into the
>>>>>>>>>>> 2.9
>>>>>>>>>>>> release
>>>>>>>>>>>>> It fixes kinda major issue - "Python client returns fields
>>> in
>>>>>> wrong
>>>>>>>>>>>> order since the 2 row when fields_count>10"
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>>>>>>>>>>> [2]
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>>>>>>>>>>>
>>>>>>>>>>>>>> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>>>>>>>>> alexey.goncharuk@gmail.com>
>>>>>>>>>>>> написал(а):
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Alexey, thanks, got it. I am not sure we can optimize
>>>>>> anything out of
>>>>>>>>>>>> the
>>>>>>>>>>>>>> message factory with suppliers (at least I have no ideas
>>>>>> right now),
>>>>>>>>>>> so
>>>>>>>>>>>>>> most likely the only move here is to switch back to the
>>>>>> switch
>>>>>>>>>>> approach
>>>>>>>>>>>>>> somehow preserving the metrics part. Probably, inlining
>>> the
>>>>>> Ignite
>>>>>>>>>>>> messages
>>>>>>>>>>>>>> to the IgniteMessageFactoryImpl should do the trick. Let
>>> me
>>>>>> explore
>>>>>>>>>>> the
>>>>>>>>>>>>>> code a bit.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> P.S. I am surprised by the impact this part makes for
>> the
>>>>>>>>>>> performance.
>>>>>>>>>>>>>> Message creation is indeed on the hot path, but a single
>>>>>> virtual call
>>>>>>>>>>>>>> should not make that much of a difference given the
>> amount
>>>>>> of other
>>>>>>>>>>>> work
>>>>>>>>>>>>>> happening during the message processing.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>> :
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Alexey, sorry, I wrongly interpreted our benchmark
>>> results.
>>>>>>>>>>> Actually,
>>>>>>>>>>>> we
>>>>>>>>>>>>>>> were looking for a drop using bi-sect in the range
>>> between
>>>>>> e6a7f93
>>>>>>>>>>>> (first
>>>>>>>>>>>>>>> commit in the 2.9 branch after 2.8 branch cut) and
>>>>>> 6592dfa5 (last
>>>>>>>>>>>> commit in
>>>>>>>>>>>>>>> the 2.9 branch). And we found these two problematic
>>>>>> commits.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Perhaps only IGNITE-13060 (Tracing) is responsible for
>> a
>>>>>> drop
>>>>>>>>>>> between
>>>>>>>>>>>>>>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>>>>> reverted
>>>>>>>>>>>> IGNITE-13060
>>>>>>>>>>>>>>> now and performance looks the same)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>>>>> related to
>>>>>>>>>>>> drop
>>>>>>>>>>>>>>> between 2.8.1 and 2.9, but still has some performance
>>>>>> problem, and
>>>>>>>>>>> we
>>>>>>>>>>>> can
>>>>>>>>>>>>>>> win back IGNITE-13060 drop by this ticket.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Do we need more investigation on IGNITE-13060 or we
>> leave
>>>>>> it as is?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> What should we do with IGNITE-12568 (MessageFactory
>>>>>> refactoring)?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> While investigating, I found that IGNITE-12568 has an
>>>>>> incorrect fix
>>>>>>>>>>>>>>>> version and is actually present in ignite-2.8.1 branch
>>>>>> [1], so it
>>>>>>>>>>>> cannot be
>>>>>>>>>>>>>>>> the source of the drop against 2.8.1.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> P.S. Looks like we need to enforce a more accurate
>> work
>>>>>> with fix
>>>>>>>>>>>> versions
>>>>>>>>>>>>>>>> or develop some sort of tooling to verify the fix
>>>>>> versions.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --AG
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>>>>>>>>>> alexey.goncharuk@gmail.com
>>>>>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>>>>>>>>> plehanov.alex@gmail.com
>>>>>>>>>>>>> :
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Guys,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>>>> IGNITE-12568
>>>>>>>>>>>> (reverted
>>>>>>>>>>>>>>>>>> it
>>>>>>>>>>>>>>>>>> locally) and got the same performance as on 2.8.1
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>>>>> paths, to
>>>>>>>>>>> trace
>>>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>>>> hot paths, it's clear why we have performance drop
>>> here.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> IGNITE-12568 (MessageFactory refactoring) -
>>> switch/case
>>>>>> block was
>>>>>>>>>>>>>>>>>> refactored to an array of message suppliers. The
>>>>>> message factory
>>>>>>>>>>>> is on
>>>>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> hot path, which explains why this commit has an
>> impact
>>>>>> on total
>>>>>>>>>>>>>>>>>> performance.
>>>>>>>>>>>>>>>>>> I've checked JIT assembly output, done some JMH
>>>>>> microbenchmarks,
>>>>>>>>>>>> and
>>>>>>>>>>>>>>>>>> found
>>>>>>>>>>>>>>>>>> that old implementation of MessageFactory.create()
>>>>>> about 30-35%
>>>>>>>>>>>> faster
>>>>>>>>>>>>>>>>>> than
>>>>>>>>>>>>>>>>>> the new one. The reason - approach with switch/case
>>> can
>>>>>>>>>>> effectively
>>>>>>>>>>>>>>>>>> inline
>>>>>>>>>>>>>>>>>> message creation code, but with an array of
>> suppliers
>>>>>> relatively
>>>>>>>>>>>> heavy
>>>>>>>>>>>>>>>>>> "invokeinterface" cannot be skipped. I've tried to
>>>>>> rewrite the
>>>>>>>>>>> code
>>>>>>>>>>>>>>>>>> using
>>>>>>>>>>>>>>>>>> an abstract class for suppliers instead of an
>>> interface
>>>>>> (to
>>>>>>>>>>>>>>>>>> replace "invokeinterface" with the "invokevirtual"),
>>>>>> but it gives
>>>>>>>>>>>> back
>>>>>>>>>>>>>>>>>> only
>>>>>>>>>>>>>>>>>> 10% of method performance and in this case, code
>> looks
>>>>>> ugly
>>>>>>>>>>>> (lambdas
>>>>>>>>>>>>>>>>>> can't
>>>>>>>>>>>>>>>>>> be used). Currently, I can't find any more ways to
>>>>>> optimize the
>>>>>>>>>>>> current
>>>>>>>>>>>>>>>>>> approach (except return to the switch/case block).
>>>>>> Andrey Gura,
>>>>>>>>>>> as
>>>>>>>>>>>> the
>>>>>>>>>>>>>>>>>> author of IGNITE-12568, maybe you have some ideas
>>> about
>>>>>>>>>>>> optimization?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Perhaps we should revert IGNITE-12568, but there are
>>>>>> some metrics
>>>>>>>>>>>>>>>>>> already
>>>>>>>>>>>>>>>>>> created, which can't be rewritten using old message
>>>>>> factory
>>>>>>>>>>>>>>>>>> implementation
>>>>>>>>>>>>>>>>>> (IGNITE-12756). Guys, WDYT?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Alexey,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I see that IGNITE-12756 (metrics improvements) is
>>>>>> already released
>>>>>>>>>>>> in
>>>>>>>>>>>>>>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>>>>> only present
>>>>>>>>>>>> in Ignite
>>>>>>>>>>>>>>>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>>>>> metrics
>>>>>>>>>>>> created for
>>>>>>>>>>>>>>>>> 2.9 that depend on the new message factory to unblock
>>>>>> the release
>>>>>>>>>>>> and deal
>>>>>>>>>>>>>>>>> with the optimizations in 2.10?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>
>>>>>
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've tried to build release candidate using TC [1]. But:
1. I can't choose a branch in this TC task (there is no such field).
2. On the "default" branch I got an error: Failed to collect changes,
error: List remote refs failed:
org.apache.http.conn.ConnectTimeoutException: Connect to
gitbox.apache.org:443 [gitbox.apache.org/52.202.80.70] failed: connect
timed out, VCS root: "GitBox [asf/ignite]" {instance id=300, parent
internal id=74, parent id=GitBoxAsfIgnite, description: "
https://gitbox.apache.org/repos/asf/ignite.git#refs/heads/master"}
Is there some problem with this TC task? What am I doing wrong?
[1] :
https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild
пн, 28 сент. 2020 г. в 13:15, Alexey Goncharuk <al...@gmail.com>:
> Saikat, Nikolay,
>
> We have migrated a bunch of modules to ignite-extensions recently. Given
> that these modules will not be available in Ignite 2.9 anymore (will
> they?), should we also plan to release the extensions?
>
> ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <pl...@gmail.com>:
>
> > Igniters,
> >
> > I've prepared release notes for the 2.9 release [1]. Can anyone review
> it,
> > please?
> >
> > [1]: https://github.com/apache/ignite/pull/8273
> >
> > вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <pl...@gmail.com>:
> >
> > > Guys,
> > >
> > > I've filled the ticket with reproducer [1] for the discovery bug. This
> > bug
> > > caused by [2] ticket. We discussed with Vladimir Steshin privately and
> > > decided to revert this ticket. I will do it today (after TC bot visa)
> if
> > > there are no objections.
> > >
> > > [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> > > [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> > >
> > > пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <pl...@gmail.com>:
> > >
> > >> Guys,
> > >>
> > >> During internal testing, we've found a critical bug with
> > >> discovery (cluster falls apart if two nodes segmented sequentially).
> > This
> > >> problem is not reproduced in 2.8.1. I think we should fix it
> > >> before release. Under investigation now. I'll let you know when we get
> > >> something.
> > >>
> > >> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> > >>
> > >>> > So what do you think? Should we proceed with a 'hacked' version of
> > the
> > >>> message factory in 2.9 and go for the runtime message generation in
> > later
> > >>> release, or keep the code clean and fix the regression in the next
> > releases?
> > >>> > Andrey, can you take a look at my change? I think it is fairly
> > >>> straightforward and does not change the semantics, just skips the
> > factory
> > >>> closures for certain messages.
> > >>>
> > >>> IMHO 2.5% isn't too much especially because it isn't actual for all
> > >>> workloads (I didn't get any significant drops during benchmarking).
> So
> > >>> I prefer the runtime generation in later releases.
> > >>>
> > >>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> > >>> <al...@gmail.com> wrote:
> > >>> >
> > >>> > Alexey, Andrey, Igniters,
> > >>> >
> > >>> > So what do you think? Should we proceed with a 'hacked' version of
> > the
> > >>> message factory in 2.9 and go for the runtime message generation in
> > later
> > >>> release, or keep the code clean and fix the regression in the next
> > releases?
> > >>> > Andrey, can you take a look at my change? I think it is fairly
> > >>> straightforward and does not change the semantics, just skips the
> > factory
> > >>> closures for certain messages.
> > >>> >
> > >>> > Personally, I would prefer fixing the regression given that we also
> > >>> introduced tracing in this release.
> > >>> >
> > >>> >
> > >>> >
> > >>> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <
> plehanov.alex@gmail.com
> > >:
> > >>> >>
> > >>> >> Alexey,
> > >>> >>
> > >>> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
> > and
> > >>> the performance of ed52559eb95 is better for about 2.5% on average on
> > our
> > >>> environment (about the same results we got benchmarking 65c30ec6 vs
> > >>> 0606f03d). We've made 24 runs for each commit of
> > >>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> > >>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
> > clients
> > >>> 50 threads each. Yardstick results for this configuration:
> > >>> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> > >>> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> > >>> >>
> > >>> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> > >>> a.budnikov.ignite@gmail.com>:
> > >>> >>>
> > >>> >>> Hi Everyone,
> > >>> >>>
> > >>> >>> I posted an instruction on how to publish the docs on
> > >>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
> > >>> update the docs by following the instruction. Unfortunately, I won't
> be
> > >>> able to spend any time on this project any longer. You can send your
> > pull
> > >>> requests and questions about the documentation to Denis Magda.
> > >>> >>>
> > >>> >>> -Artem
> > >>> >>>
> > >>> >>> [1] :
> > >>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> > >>> >>>
> > >>> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> > >>> alexey.goncharuk@gmail.com> wrote:
> > >>> >>>>
> > >>> >>>> Alexey,
> > >>> >>>>
> > >>> >>>> I've tried to play with message factories locally, but
> > >>> unfortunately, I
> > >>> >>>> cannot spot the difference between old and new implementation in
> > >>> >>>> distributed benchmarks. I pushed an implementation of
> > >>> MessageFactoryImpl
> > >>> >>>> with the old switch statement to the ignite-2.9-revert-12568
> > branch
> > >>> >>>> (discussed this with Andrey Gura, the change should be
> compatible
> > >>> with the
> > >>> >>>> new metrics as we still use the register() mechanics).
> > >>> >>>>
> > >>> >>>> Can you check if this change makes any difference
> performance-wise
> > >>> in your
> > >>> >>>> environment? If yes, we can go with runtime code generation in
> the
> > >>> long
> > >>> >>>> term: register classes and generate a dynamic message factory
> with
> > >>> a switch
> > >>> >>>> statement once all messages are registered (not in 2.9 though,
> > >>> obviously).
> > >>> >>>>
> > >>> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> > plehanov.alex@gmail.com
> > >>> >:
> > >>> >>>>
> > >>> >>>> > Hello guys,
> > >>> >>>> >
> > >>> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
> > >>> reduced the
> > >>> >>>> > drop, but not completely removed it.
> > >>> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
> > >>> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against
> > >>> 2.8.1
> > >>> >>>> > release on your environment?
> > >>> >>>> > With this patch on our environment, it's about a 3% drop left,
> > >>> it's close
> > >>> >>>> > to measurement error and I think such a drop is not a
> > >>> showstopper. Guys,
> > >>> >>>> > WDYT?
> > >>> >>>> >
> > >>> >>>> > Also, I found that compatibility is broken for JDBC thin
> driver
> > >>> between 2.8
> > >>> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and
> should
> > >>> be
> > >>> >>>> > fixed in 2.9. I've prepared the patch.
> > >>> >>>> > Taras Ledkov, can you please review this patch?
> > >>> >>>> >
> > >>> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO
> > message
> > >>> >>>> > send problem in some circumstances). I will cherry-pick it if
> > >>> there is no
> > >>> >>>> > objection.
> > >>> >>>> >
> > >>> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
> > >>> >>>> > [2] https://github.com/apache/ignite/pull/8223
> > >>> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
> > >>> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
> > >>> >>>> >
> > >>> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <
> mmuzaf@apache.org
> > >:
> > >>> >>>> >
> > >>> >>>> > > Alexey,
> > >>> >>>> > >
> > >>> >>>> > > I propose to include [1] issue to the 2.9 release. Since
> this
> > >>> issue is
> > >>> >>>> > > related to the new master key change functionality which
> > >>> haven't been
> > >>> >>>> > > released yet I think it will be safe to cherry-pick commit
> to
> > >>> the
> > >>> >>>> > > release branch.
> > >>> >>>> > >
> > >>> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> > >>> >>>> > >
> > >>> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> > >>> nizhikov@apache.org>
> > >>> >>>> > wrote:
> > >>> >>>> > > >
> > >>> >>>> > > > Hello, Igniters.
> > >>> >>>> > > >
> > >>> >>>> > > > Alexey, please, include one more Python thin client fix
> [1]
> > >>> into the
> > >>> >>>> > 2.9
> > >>> >>>> > > release
> > >>> >>>> > > > It fixes kinda major issue - "Python client returns fields
> > in
> > >>> wrong
> > >>> >>>> > > order since the 2 row when fields_count>10"
> > >>> >>>> > > >
> > >>> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> > >>> >>>> > > > [2]
> > >>> >>>> > >
> > >>> >>>> >
> > >>>
> >
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> > >>> >>>> > > >
> > >>> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> > >>> >>>> > alexey.goncharuk@gmail.com>
> > >>> >>>> > > написал(а):
> > >>> >>>> > > > >
> > >>> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize
> > >>> anything out of
> > >>> >>>> > > the
> > >>> >>>> > > > > message factory with suppliers (at least I have no ideas
> > >>> right now),
> > >>> >>>> > so
> > >>> >>>> > > > > most likely the only move here is to switch back to the
> > >>> switch
> > >>> >>>> > approach
> > >>> >>>> > > > > somehow preserving the metrics part. Probably, inlining
> > the
> > >>> Ignite
> > >>> >>>> > > messages
> > >>> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let
> > me
> > >>> explore
> > >>> >>>> > the
> > >>> >>>> > > > > code a bit.
> > >>> >>>> > > > >
> > >>> >>>> > > > > P.S. I am surprised by the impact this part makes for
> the
> > >>> >>>> > performance.
> > >>> >>>> > > > > Message creation is indeed on the hot path, but a single
> > >>> virtual call
> > >>> >>>> > > > > should not make that much of a difference given the
> amount
> > >>> of other
> > >>> >>>> > > work
> > >>> >>>> > > > > happening during the message processing.
> > >>> >>>> > > > >
> > >>> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> > >>> plehanov.alex@gmail.com
> > >>> >>>> > >:
> > >>> >>>> > > > >
> > >>> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark
> > results.
> > >>> >>>> > Actually,
> > >>> >>>> > > we
> > >>> >>>> > > > >> were looking for a drop using bi-sect in the range
> > between
> > >>> e6a7f93
> > >>> >>>> > > (first
> > >>> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and
> > >>> 6592dfa5 (last
> > >>> >>>> > > commit in
> > >>> >>>> > > > >> the 2.9 branch). And we found these two problematic
> > >>> commits.
> > >>> >>>> > > > >>
> > >>> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for
> a
> > >>> drop
> > >>> >>>> > between
> > >>> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> > >>> reverted
> > >>> >>>> > > IGNITE-13060
> > >>> >>>> > > > >> now and performance looks the same)
> > >>> >>>> > > > >>
> > >>> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> > >>> related to
> > >>> >>>> > > drop
> > >>> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
> > >>> problem, and
> > >>> >>>> > we
> > >>> >>>> > > can
> > >>> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
> > >>> >>>> > > > >>
> > >>> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we
> leave
> > >>> it as is?
> > >>> >>>> > > > >>
> > >>> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
> > >>> refactoring)?
> > >>> >>>> > > > >>
> > >>> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> > >>> >>>> > > alexey.goncharuk@gmail.com
> > >>> >>>> > > > >>> :
> > >>> >>>> > > > >>
> > >>> >>>> > > > >>> Alexey,
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
> > >>> incorrect fix
> > >>> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch
> > >>> [1], so it
> > >>> >>>> > > cannot be
> > >>> >>>> > > > >>> the source of the drop against 2.8.1.
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate
> work
> > >>> with fix
> > >>> >>>> > > versions
> > >>> >>>> > > > >>> or develop some sort of tooling to verify the fix
> > >>> versions.
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>> --AG
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>> [1]
> > >>> >>>> > > > >>>
> > >>> >>>> > >
> > >>> >>>> >
> > >>>
> >
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> > >>> >>>> > > alexey.goncharuk@gmail.com
> > >>> >>>> > > > >>>> :
> > >>> >>>> > > > >>>
> > >>> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> > >>> >>>> > plehanov.alex@gmail.com
> > >>> >>>> > > >:
> > >>> >>>> > > > >>>>
> > >>> >>>> > > > >>>>> Guys,
> > >>> >>>> > > > >>>>>
> > >>> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
> > >>> IGNITE-12568
> > >>> >>>> > > (reverted
> > >>> >>>> > > > >>>>> it
> > >>> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
> > >>> >>>> > > > >>>>>
> > >>> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot
> > >>> paths, to
> > >>> >>>> > trace
> > >>> >>>> > > > >>>>> these
> > >>> >>>> > > > >>>>> hot paths, it's clear why we have performance drop
> > here.
> > >>> >>>> > > > >>>>>
> > >>> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) -
> > switch/case
> > >>> block was
> > >>> >>>> > > > >>>>> refactored to an array of message suppliers. The
> > >>> message factory
> > >>> >>>> > > is on
> > >>> >>>> > > > >>>>> the
> > >>> >>>> > > > >>>>> hot path, which explains why this commit has an
> impact
> > >>> on total
> > >>> >>>> > > > >>>>> performance.
> > >>> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
> > >>> microbenchmarks,
> > >>> >>>> > > and
> > >>> >>>> > > > >>>>> found
> > >>> >>>> > > > >>>>> that old implementation of MessageFactory.create()
> > >>> about 30-35%
> > >>> >>>> > > faster
> > >>> >>>> > > > >>>>> than
> > >>> >>>> > > > >>>>> the new one. The reason - approach with switch/case
> > can
> > >>> >>>> > effectively
> > >>> >>>> > > > >>>>> inline
> > >>> >>>> > > > >>>>> message creation code, but with an array of
> suppliers
> > >>> relatively
> > >>> >>>> > > heavy
> > >>> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to
> > >>> rewrite the
> > >>> >>>> > code
> > >>> >>>> > > > >>>>> using
> > >>> >>>> > > > >>>>> an abstract class for suppliers instead of an
> > interface
> > >>> (to
> > >>> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"),
> > >>> but it gives
> > >>> >>>> > > back
> > >>> >>>> > > > >>>>> only
> > >>> >>>> > > > >>>>> 10% of method performance and in this case, code
> looks
> > >>> ugly
> > >>> >>>> > > (lambdas
> > >>> >>>> > > > >>>>> can't
> > >>> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
> > >>> optimize the
> > >>> >>>> > > current
> > >>> >>>> > > > >>>>> approach (except return to the switch/case block).
> > >>> Andrey Gura,
> > >>> >>>> > as
> > >>> >>>> > > the
> > >>> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas
> > about
> > >>> >>>> > > optimization?
> > >>> >>>> > > > >>>>>
> > >>> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are
> > >>> some metrics
> > >>> >>>> > > > >>>>> already
> > >>> >>>> > > > >>>>> created, which can't be rewritten using old message
> > >>> factory
> > >>> >>>> > > > >>>>> implementation
> > >>> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
> > >>> >>>> > > > >>>>>
> > >>> >>>> > > > >>>>
> > >>> >>>> > > > >>>> Alexey,
> > >>> >>>> > > > >>>>
> > >>> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is
> > >>> already released
> > >>> >>>> > > in
> > >>> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> > >>> only present
> > >>> >>>> > > in Ignite
> > >>> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> > >>> metrics
> > >>> >>>> > > created for
> > >>> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock
> > >>> the release
> > >>> >>>> > > and deal
> > >>> >>>> > > > >>>> with the optimizations in 2.10?
> > >>> >>>> > > > >>>>
> > >>> >>>> > > > >>>
> > >>> >>>> > > >
> > >>> >>>> > >
> > >>> >>>> >
> > >>>
> > >>
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Saikat, Nikolay,
We have migrated a bunch of modules to ignite-extensions recently. Given
that these modules will not be available in Ignite 2.9 anymore (will
they?), should we also plan to release the extensions?
ср, 23 сент. 2020 г. в 21:49, Alex Plehanov <pl...@gmail.com>:
> Igniters,
>
> I've prepared release notes for the 2.9 release [1]. Can anyone review it,
> please?
>
> [1]: https://github.com/apache/ignite/pull/8273
>
> вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <pl...@gmail.com>:
>
> > Guys,
> >
> > I've filled the ticket with reproducer [1] for the discovery bug. This
> bug
> > caused by [2] ticket. We discussed with Vladimir Steshin privately and
> > decided to revert this ticket. I will do it today (after TC bot visa) if
> > there are no objections.
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> > [2]: https://issues.apache.org/jira/browse/IGNITE-13134
> >
> > пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <pl...@gmail.com>:
> >
> >> Guys,
> >>
> >> During internal testing, we've found a critical bug with
> >> discovery (cluster falls apart if two nodes segmented sequentially).
> This
> >> problem is not reproduced in 2.8.1. I think we should fix it
> >> before release. Under investigation now. I'll let you know when we get
> >> something.
> >>
> >> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> >>
> >>> > So what do you think? Should we proceed with a 'hacked' version of
> the
> >>> message factory in 2.9 and go for the runtime message generation in
> later
> >>> release, or keep the code clean and fix the regression in the next
> releases?
> >>> > Andrey, can you take a look at my change? I think it is fairly
> >>> straightforward and does not change the semantics, just skips the
> factory
> >>> closures for certain messages.
> >>>
> >>> IMHO 2.5% isn't too much especially because it isn't actual for all
> >>> workloads (I didn't get any significant drops during benchmarking). So
> >>> I prefer the runtime generation in later releases.
> >>>
> >>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> >>> <al...@gmail.com> wrote:
> >>> >
> >>> > Alexey, Andrey, Igniters,
> >>> >
> >>> > So what do you think? Should we proceed with a 'hacked' version of
> the
> >>> message factory in 2.9 and go for the runtime message generation in
> later
> >>> release, or keep the code clean and fix the regression in the next
> releases?
> >>> > Andrey, can you take a look at my change? I think it is fairly
> >>> straightforward and does not change the semantics, just skips the
> factory
> >>> closures for certain messages.
> >>> >
> >>> > Personally, I would prefer fixing the regression given that we also
> >>> introduced tracing in this release.
> >>> >
> >>> >
> >>> >
> >>> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >>> >>
> >>> >> Alexey,
> >>> >>
> >>> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95
> and
> >>> the performance of ed52559eb95 is better for about 2.5% on average on
> our
> >>> environment (about the same results we got benchmarking 65c30ec6 vs
> >>> 0606f03d). We've made 24 runs for each commit of
> >>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> >>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
> clients
> >>> 50 threads each. Yardstick results for this configuration:
> >>> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >>> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>> >>
> >>> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> >>> a.budnikov.ignite@gmail.com>:
> >>> >>>
> >>> >>> Hi Everyone,
> >>> >>>
> >>> >>> I posted an instruction on how to publish the docs on
> >>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
> >>> update the docs by following the instruction. Unfortunately, I won't be
> >>> able to spend any time on this project any longer. You can send your
> pull
> >>> requests and questions about the documentation to Denis Magda.
> >>> >>>
> >>> >>> -Artem
> >>> >>>
> >>> >>> [1] :
> >>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>> >>>
> >>> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >>> alexey.goncharuk@gmail.com> wrote:
> >>> >>>>
> >>> >>>> Alexey,
> >>> >>>>
> >>> >>>> I've tried to play with message factories locally, but
> >>> unfortunately, I
> >>> >>>> cannot spot the difference between old and new implementation in
> >>> >>>> distributed benchmarks. I pushed an implementation of
> >>> MessageFactoryImpl
> >>> >>>> with the old switch statement to the ignite-2.9-revert-12568
> branch
> >>> >>>> (discussed this with Andrey Gura, the change should be compatible
> >>> with the
> >>> >>>> new metrics as we still use the register() mechanics).
> >>> >>>>
> >>> >>>> Can you check if this change makes any difference performance-wise
> >>> in your
> >>> >>>> environment? If yes, we can go with runtime code generation in the
> >>> long
> >>> >>>> term: register classes and generate a dynamic message factory with
> >>> a switch
> >>> >>>> statement once all messages are registered (not in 2.9 though,
> >>> obviously).
> >>> >>>>
> >>> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <
> plehanov.alex@gmail.com
> >>> >:
> >>> >>>>
> >>> >>>> > Hello guys,
> >>> >>>> >
> >>> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
> >>> reduced the
> >>> >>>> > drop, but not completely removed it.
> >>> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
> >>> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against
> >>> 2.8.1
> >>> >>>> > release on your environment?
> >>> >>>> > With this patch on our environment, it's about a 3% drop left,
> >>> it's close
> >>> >>>> > to measurement error and I think such a drop is not a
> >>> showstopper. Guys,
> >>> >>>> > WDYT?
> >>> >>>> >
> >>> >>>> > Also, I found that compatibility is broken for JDBC thin driver
> >>> between 2.8
> >>> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should
> >>> be
> >>> >>>> > fixed in 2.9. I've prepared the patch.
> >>> >>>> > Taras Ledkov, can you please review this patch?
> >>> >>>> >
> >>> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO
> message
> >>> >>>> > send problem in some circumstances). I will cherry-pick it if
> >>> there is no
> >>> >>>> > objection.
> >>> >>>> >
> >>> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>> >>>> > [2] https://github.com/apache/ignite/pull/8223
> >>> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>> >>>> >
> >>> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mmuzaf@apache.org
> >:
> >>> >>>> >
> >>> >>>> > > Alexey,
> >>> >>>> > >
> >>> >>>> > > I propose to include [1] issue to the 2.9 release. Since this
> >>> issue is
> >>> >>>> > > related to the new master key change functionality which
> >>> haven't been
> >>> >>>> > > released yet I think it will be safe to cherry-pick commit to
> >>> the
> >>> >>>> > > release branch.
> >>> >>>> > >
> >>> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>> >>>> > >
> >>> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
> >>> nizhikov@apache.org>
> >>> >>>> > wrote:
> >>> >>>> > > >
> >>> >>>> > > > Hello, Igniters.
> >>> >>>> > > >
> >>> >>>> > > > Alexey, please, include one more Python thin client fix [1]
> >>> into the
> >>> >>>> > 2.9
> >>> >>>> > > release
> >>> >>>> > > > It fixes kinda major issue - "Python client returns fields
> in
> >>> wrong
> >>> >>>> > > order since the 2 row when fields_count>10"
> >>> >>>> > > >
> >>> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>> >>>> > > > [2]
> >>> >>>> > >
> >>> >>>> >
> >>>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>> >>>> > > >
> >>> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>> >>>> > alexey.goncharuk@gmail.com>
> >>> >>>> > > написал(а):
> >>> >>>> > > > >
> >>> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize
> >>> anything out of
> >>> >>>> > > the
> >>> >>>> > > > > message factory with suppliers (at least I have no ideas
> >>> right now),
> >>> >>>> > so
> >>> >>>> > > > > most likely the only move here is to switch back to the
> >>> switch
> >>> >>>> > approach
> >>> >>>> > > > > somehow preserving the metrics part. Probably, inlining
> the
> >>> Ignite
> >>> >>>> > > messages
> >>> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let
> me
> >>> explore
> >>> >>>> > the
> >>> >>>> > > > > code a bit.
> >>> >>>> > > > >
> >>> >>>> > > > > P.S. I am surprised by the impact this part makes for the
> >>> >>>> > performance.
> >>> >>>> > > > > Message creation is indeed on the hot path, but a single
> >>> virtual call
> >>> >>>> > > > > should not make that much of a difference given the amount
> >>> of other
> >>> >>>> > > work
> >>> >>>> > > > > happening during the message processing.
> >>> >>>> > > > >
> >>> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >>> plehanov.alex@gmail.com
> >>> >>>> > >:
> >>> >>>> > > > >
> >>> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark
> results.
> >>> >>>> > Actually,
> >>> >>>> > > we
> >>> >>>> > > > >> were looking for a drop using bi-sect in the range
> between
> >>> e6a7f93
> >>> >>>> > > (first
> >>> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and
> >>> 6592dfa5 (last
> >>> >>>> > > commit in
> >>> >>>> > > > >> the 2.9 branch). And we found these two problematic
> >>> commits.
> >>> >>>> > > > >>
> >>> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a
> >>> drop
> >>> >>>> > between
> >>> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
> >>> reverted
> >>> >>>> > > IGNITE-13060
> >>> >>>> > > > >> now and performance looks the same)
> >>> >>>> > > > >>
> >>> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >>> related to
> >>> >>>> > > drop
> >>> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
> >>> problem, and
> >>> >>>> > we
> >>> >>>> > > can
> >>> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
> >>> >>>> > > > >>
> >>> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave
> >>> it as is?
> >>> >>>> > > > >>
> >>> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
> >>> refactoring)?
> >>> >>>> > > > >>
> >>> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>> >>>> > > alexey.goncharuk@gmail.com
> >>> >>>> > > > >>> :
> >>> >>>> > > > >>
> >>> >>>> > > > >>> Alexey,
> >>> >>>> > > > >>>
> >>> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
> >>> incorrect fix
> >>> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch
> >>> [1], so it
> >>> >>>> > > cannot be
> >>> >>>> > > > >>> the source of the drop against 2.8.1.
> >>> >>>> > > > >>>
> >>> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate work
> >>> with fix
> >>> >>>> > > versions
> >>> >>>> > > > >>> or develop some sort of tooling to verify the fix
> >>> versions.
> >>> >>>> > > > >>>
> >>> >>>> > > > >>> --AG
> >>> >>>> > > > >>>
> >>> >>>> > > > >>> [1]
> >>> >>>> > > > >>>
> >>> >>>> > >
> >>> >>>> >
> >>>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>> >>>> > > > >>>
> >>> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>> >>>> > > alexey.goncharuk@gmail.com
> >>> >>>> > > > >>>> :
> >>> >>>> > > > >>>
> >>> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>> >>>> > plehanov.alex@gmail.com
> >>> >>>> > > >:
> >>> >>>> > > > >>>>
> >>> >>>> > > > >>>>> Guys,
> >>> >>>> > > > >>>>>
> >>> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >>> IGNITE-12568
> >>> >>>> > > (reverted
> >>> >>>> > > > >>>>> it
> >>> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
> >>> >>>> > > > >>>>>
> >>> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot
> >>> paths, to
> >>> >>>> > trace
> >>> >>>> > > > >>>>> these
> >>> >>>> > > > >>>>> hot paths, it's clear why we have performance drop
> here.
> >>> >>>> > > > >>>>>
> >>> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) -
> switch/case
> >>> block was
> >>> >>>> > > > >>>>> refactored to an array of message suppliers. The
> >>> message factory
> >>> >>>> > > is on
> >>> >>>> > > > >>>>> the
> >>> >>>> > > > >>>>> hot path, which explains why this commit has an impact
> >>> on total
> >>> >>>> > > > >>>>> performance.
> >>> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
> >>> microbenchmarks,
> >>> >>>> > > and
> >>> >>>> > > > >>>>> found
> >>> >>>> > > > >>>>> that old implementation of MessageFactory.create()
> >>> about 30-35%
> >>> >>>> > > faster
> >>> >>>> > > > >>>>> than
> >>> >>>> > > > >>>>> the new one. The reason - approach with switch/case
> can
> >>> >>>> > effectively
> >>> >>>> > > > >>>>> inline
> >>> >>>> > > > >>>>> message creation code, but with an array of suppliers
> >>> relatively
> >>> >>>> > > heavy
> >>> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to
> >>> rewrite the
> >>> >>>> > code
> >>> >>>> > > > >>>>> using
> >>> >>>> > > > >>>>> an abstract class for suppliers instead of an
> interface
> >>> (to
> >>> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"),
> >>> but it gives
> >>> >>>> > > back
> >>> >>>> > > > >>>>> only
> >>> >>>> > > > >>>>> 10% of method performance and in this case, code looks
> >>> ugly
> >>> >>>> > > (lambdas
> >>> >>>> > > > >>>>> can't
> >>> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
> >>> optimize the
> >>> >>>> > > current
> >>> >>>> > > > >>>>> approach (except return to the switch/case block).
> >>> Andrey Gura,
> >>> >>>> > as
> >>> >>>> > > the
> >>> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas
> about
> >>> >>>> > > optimization?
> >>> >>>> > > > >>>>>
> >>> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are
> >>> some metrics
> >>> >>>> > > > >>>>> already
> >>> >>>> > > > >>>>> created, which can't be rewritten using old message
> >>> factory
> >>> >>>> > > > >>>>> implementation
> >>> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
> >>> >>>> > > > >>>>>
> >>> >>>> > > > >>>>
> >>> >>>> > > > >>>> Alexey,
> >>> >>>> > > > >>>>
> >>> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is
> >>> already released
> >>> >>>> > > in
> >>> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
> >>> only present
> >>> >>>> > > in Ignite
> >>> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> >>> metrics
> >>> >>>> > > created for
> >>> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock
> >>> the release
> >>> >>>> > > and deal
> >>> >>>> > > > >>>> with the optimizations in 2.10?
> >>> >>>> > > > >>>>
> >>> >>>> > > > >>>
> >>> >>>> > > >
> >>> >>>> > >
> >>> >>>> >
> >>>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Igniters,
I've prepared release notes for the 2.9 release [1]. Can anyone review it,
please?
[1]: https://github.com/apache/ignite/pull/8273
вт, 22 сент. 2020 г. в 09:39, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> I've filled the ticket with reproducer [1] for the discovery bug. This bug
> caused by [2] ticket. We discussed with Vladimir Steshin privately and
> decided to revert this ticket. I will do it today (after TC bot visa) if
> there are no objections.
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-13465
> [2]: https://issues.apache.org/jira/browse/IGNITE-13134
>
> пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <pl...@gmail.com>:
>
>> Guys,
>>
>> During internal testing, we've found a critical bug with
>> discovery (cluster falls apart if two nodes segmented sequentially). This
>> problem is not reproduced in 2.8.1. I think we should fix it
>> before release. Under investigation now. I'll let you know when we get
>> something.
>>
>> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>>
>>> > So what do you think? Should we proceed with a 'hacked' version of the
>>> message factory in 2.9 and go for the runtime message generation in later
>>> release, or keep the code clean and fix the regression in the next releases?
>>> > Andrey, can you take a look at my change? I think it is fairly
>>> straightforward and does not change the semantics, just skips the factory
>>> closures for certain messages.
>>>
>>> IMHO 2.5% isn't too much especially because it isn't actual for all
>>> workloads (I didn't get any significant drops during benchmarking). So
>>> I prefer the runtime generation in later releases.
>>>
>>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>>> <al...@gmail.com> wrote:
>>> >
>>> > Alexey, Andrey, Igniters,
>>> >
>>> > So what do you think? Should we proceed with a 'hacked' version of the
>>> message factory in 2.9 and go for the runtime message generation in later
>>> release, or keep the code clean and fix the regression in the next releases?
>>> > Andrey, can you take a look at my change? I think it is fairly
>>> straightforward and does not change the semantics, just skips the factory
>>> closures for certain messages.
>>> >
>>> > Personally, I would prefer fixing the regression given that we also
>>> introduced tracing in this release.
>>> >
>>> >
>>> >
>>> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
>>> >>
>>> >> Alexey,
>>> >>
>>> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and
>>> the performance of ed52559eb95 is better for about 2.5% on average on our
>>> environment (about the same results we got benchmarking 65c30ec6 vs
>>> 0606f03d). We've made 24 runs for each commit of
>>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
>>> 50 threads each. Yardstick results for this configuration:
>>> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>>> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>> >>
>>> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>>> a.budnikov.ignite@gmail.com>:
>>> >>>
>>> >>> Hi Everyone,
>>> >>>
>>> >>> I posted an instruction on how to publish the docs on
>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>>> update the docs by following the instruction. Unfortunately, I won't be
>>> able to spend any time on this project any longer. You can send your pull
>>> requests and questions about the documentation to Denis Magda.
>>> >>>
>>> >>> -Artem
>>> >>>
>>> >>> [1] :
>>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>> >>>
>>> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com> wrote:
>>> >>>>
>>> >>>> Alexey,
>>> >>>>
>>> >>>> I've tried to play with message factories locally, but
>>> unfortunately, I
>>> >>>> cannot spot the difference between old and new implementation in
>>> >>>> distributed benchmarks. I pushed an implementation of
>>> MessageFactoryImpl
>>> >>>> with the old switch statement to the ignite-2.9-revert-12568 branch
>>> >>>> (discussed this with Andrey Gura, the change should be compatible
>>> with the
>>> >>>> new metrics as we still use the register() mechanics).
>>> >>>>
>>> >>>> Can you check if this change makes any difference performance-wise
>>> in your
>>> >>>> environment? If yes, we can go with runtime code generation in the
>>> long
>>> >>>> term: register classes and generate a dynamic message factory with
>>> a switch
>>> >>>> statement once all messages are registered (not in 2.9 though,
>>> obviously).
>>> >>>>
>>> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <plehanov.alex@gmail.com
>>> >:
>>> >>>>
>>> >>>> > Hello guys,
>>> >>>> >
>>> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
>>> reduced the
>>> >>>> > drop, but not completely removed it.
>>> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>>> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against
>>> 2.8.1
>>> >>>> > release on your environment?
>>> >>>> > With this patch on our environment, it's about a 3% drop left,
>>> it's close
>>> >>>> > to measurement error and I think such a drop is not a
>>> showstopper. Guys,
>>> >>>> > WDYT?
>>> >>>> >
>>> >>>> > Also, I found that compatibility is broken for JDBC thin driver
>>> between 2.8
>>> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should
>>> be
>>> >>>> > fixed in 2.9. I've prepared the patch.
>>> >>>> > Taras Ledkov, can you please review this patch?
>>> >>>> >
>>> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>>> >>>> > send problem in some circumstances). I will cherry-pick it if
>>> there is no
>>> >>>> > objection.
>>> >>>> >
>>> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>> >>>> > [2] https://github.com/apache/ignite/pull/8223
>>> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>> >>>> >
>>> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>>> >>>> >
>>> >>>> > > Alexey,
>>> >>>> > >
>>> >>>> > > I propose to include [1] issue to the 2.9 release. Since this
>>> issue is
>>> >>>> > > related to the new master key change functionality which
>>> haven't been
>>> >>>> > > released yet I think it will be safe to cherry-pick commit to
>>> the
>>> >>>> > > release branch.
>>> >>>> > >
>>> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>> >>>> > >
>>> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>>> nizhikov@apache.org>
>>> >>>> > wrote:
>>> >>>> > > >
>>> >>>> > > > Hello, Igniters.
>>> >>>> > > >
>>> >>>> > > > Alexey, please, include one more Python thin client fix [1]
>>> into the
>>> >>>> > 2.9
>>> >>>> > > release
>>> >>>> > > > It fixes kinda major issue - "Python client returns fields in
>>> wrong
>>> >>>> > > order since the 2 row when fields_count>10"
>>> >>>> > > >
>>> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>> >>>> > > > [2]
>>> >>>> > >
>>> >>>> >
>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>> >>>> > > >
>>> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>> >>>> > alexey.goncharuk@gmail.com>
>>> >>>> > > написал(а):
>>> >>>> > > > >
>>> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize
>>> anything out of
>>> >>>> > > the
>>> >>>> > > > > message factory with suppliers (at least I have no ideas
>>> right now),
>>> >>>> > so
>>> >>>> > > > > most likely the only move here is to switch back to the
>>> switch
>>> >>>> > approach
>>> >>>> > > > > somehow preserving the metrics part. Probably, inlining the
>>> Ignite
>>> >>>> > > messages
>>> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
>>> explore
>>> >>>> > the
>>> >>>> > > > > code a bit.
>>> >>>> > > > >
>>> >>>> > > > > P.S. I am surprised by the impact this part makes for the
>>> >>>> > performance.
>>> >>>> > > > > Message creation is indeed on the hot path, but a single
>>> virtual call
>>> >>>> > > > > should not make that much of a difference given the amount
>>> of other
>>> >>>> > > work
>>> >>>> > > > > happening during the message processing.
>>> >>>> > > > >
>>> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>> plehanov.alex@gmail.com
>>> >>>> > >:
>>> >>>> > > > >
>>> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>>> >>>> > Actually,
>>> >>>> > > we
>>> >>>> > > > >> were looking for a drop using bi-sect in the range between
>>> e6a7f93
>>> >>>> > > (first
>>> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and
>>> 6592dfa5 (last
>>> >>>> > > commit in
>>> >>>> > > > >> the 2.9 branch). And we found these two problematic
>>> commits.
>>> >>>> > > > >>
>>> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a
>>> drop
>>> >>>> > between
>>> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>>> reverted
>>> >>>> > > IGNITE-13060
>>> >>>> > > > >> now and performance looks the same)
>>> >>>> > > > >>
>>> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>> related to
>>> >>>> > > drop
>>> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
>>> problem, and
>>> >>>> > we
>>> >>>> > > can
>>> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
>>> >>>> > > > >>
>>> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave
>>> it as is?
>>> >>>> > > > >>
>>> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
>>> refactoring)?
>>> >>>> > > > >>
>>> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>> >>>> > > alexey.goncharuk@gmail.com
>>> >>>> > > > >>> :
>>> >>>> > > > >>
>>> >>>> > > > >>> Alexey,
>>> >>>> > > > >>>
>>> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
>>> incorrect fix
>>> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch
>>> [1], so it
>>> >>>> > > cannot be
>>> >>>> > > > >>> the source of the drop against 2.8.1.
>>> >>>> > > > >>>
>>> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate work
>>> with fix
>>> >>>> > > versions
>>> >>>> > > > >>> or develop some sort of tooling to verify the fix
>>> versions.
>>> >>>> > > > >>>
>>> >>>> > > > >>> --AG
>>> >>>> > > > >>>
>>> >>>> > > > >>> [1]
>>> >>>> > > > >>>
>>> >>>> > >
>>> >>>> >
>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>> >>>> > > > >>>
>>> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>> >>>> > > alexey.goncharuk@gmail.com
>>> >>>> > > > >>>> :
>>> >>>> > > > >>>
>>> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>> >>>> > plehanov.alex@gmail.com
>>> >>>> > > >:
>>> >>>> > > > >>>>
>>> >>>> > > > >>>>> Guys,
>>> >>>> > > > >>>>>
>>> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>> IGNITE-12568
>>> >>>> > > (reverted
>>> >>>> > > > >>>>> it
>>> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
>>> >>>> > > > >>>>>
>>> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot
>>> paths, to
>>> >>>> > trace
>>> >>>> > > > >>>>> these
>>> >>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>>> >>>> > > > >>>>>
>>> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
>>> block was
>>> >>>> > > > >>>>> refactored to an array of message suppliers. The
>>> message factory
>>> >>>> > > is on
>>> >>>> > > > >>>>> the
>>> >>>> > > > >>>>> hot path, which explains why this commit has an impact
>>> on total
>>> >>>> > > > >>>>> performance.
>>> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
>>> microbenchmarks,
>>> >>>> > > and
>>> >>>> > > > >>>>> found
>>> >>>> > > > >>>>> that old implementation of MessageFactory.create()
>>> about 30-35%
>>> >>>> > > faster
>>> >>>> > > > >>>>> than
>>> >>>> > > > >>>>> the new one. The reason - approach with switch/case can
>>> >>>> > effectively
>>> >>>> > > > >>>>> inline
>>> >>>> > > > >>>>> message creation code, but with an array of suppliers
>>> relatively
>>> >>>> > > heavy
>>> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to
>>> rewrite the
>>> >>>> > code
>>> >>>> > > > >>>>> using
>>> >>>> > > > >>>>> an abstract class for suppliers instead of an interface
>>> (to
>>> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"),
>>> but it gives
>>> >>>> > > back
>>> >>>> > > > >>>>> only
>>> >>>> > > > >>>>> 10% of method performance and in this case, code looks
>>> ugly
>>> >>>> > > (lambdas
>>> >>>> > > > >>>>> can't
>>> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
>>> optimize the
>>> >>>> > > current
>>> >>>> > > > >>>>> approach (except return to the switch/case block).
>>> Andrey Gura,
>>> >>>> > as
>>> >>>> > > the
>>> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>>> >>>> > > optimization?
>>> >>>> > > > >>>>>
>>> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are
>>> some metrics
>>> >>>> > > > >>>>> already
>>> >>>> > > > >>>>> created, which can't be rewritten using old message
>>> factory
>>> >>>> > > > >>>>> implementation
>>> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>>> >>>> > > > >>>>>
>>> >>>> > > > >>>>
>>> >>>> > > > >>>> Alexey,
>>> >>>> > > > >>>>
>>> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is
>>> already released
>>> >>>> > > in
>>> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is
>>> only present
>>> >>>> > > in Ignite
>>> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>>> metrics
>>> >>>> > > created for
>>> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock
>>> the release
>>> >>>> > > and deal
>>> >>>> > > > >>>> with the optimizations in 2.10?
>>> >>>> > > > >>>>
>>> >>>> > > > >>>
>>> >>>> > > >
>>> >>>> > >
>>> >>>> >
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've filled the ticket with reproducer [1] for the discovery bug. This bug
caused by [2] ticket. We discussed with Vladimir Steshin privately and
decided to revert this ticket. I will do it today (after TC bot visa) if
there are no objections.
[1]: https://issues.apache.org/jira/browse/IGNITE-13465
[2]: https://issues.apache.org/jira/browse/IGNITE-13134
пн, 21 сент. 2020 г. в 11:08, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> During internal testing, we've found a critical bug with
> discovery (cluster falls apart if two nodes segmented sequentially). This
> problem is not reproduced in 2.8.1. I think we should fix it
> before release. Under investigation now. I'll let you know when we get
> something.
>
> чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
>
>> > So what do you think? Should we proceed with a 'hacked' version of the
>> message factory in 2.9 and go for the runtime message generation in later
>> release, or keep the code clean and fix the regression in the next releases?
>> > Andrey, can you take a look at my change? I think it is fairly
>> straightforward and does not change the semantics, just skips the factory
>> closures for certain messages.
>>
>> IMHO 2.5% isn't too much especially because it isn't actual for all
>> workloads (I didn't get any significant drops during benchmarking). So
>> I prefer the runtime generation in later releases.
>>
>> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
>> <al...@gmail.com> wrote:
>> >
>> > Alexey, Andrey, Igniters,
>> >
>> > So what do you think? Should we proceed with a 'hacked' version of the
>> message factory in 2.9 and go for the runtime message generation in later
>> release, or keep the code clean and fix the regression in the next releases?
>> > Andrey, can you take a look at my change? I think it is fairly
>> straightforward and does not change the semantics, just skips the factory
>> closures for certain messages.
>> >
>> > Personally, I would prefer fixing the regression given that we also
>> introduced tracing in this release.
>> >
>> >
>> >
>> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
>> >>
>> >> Alexey,
>> >>
>> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and
>> the performance of ed52559eb95 is better for about 2.5% on average on our
>> environment (about the same results we got benchmarking 65c30ec6 vs
>> 0606f03d). We've made 24 runs for each commit of
>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
>> 50 threads each. Yardstick results for this configuration:
>> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>> >>
>> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
>> a.budnikov.ignite@gmail.com>:
>> >>>
>> >>> Hi Everyone,
>> >>>
>> >>> I posted an instruction on how to publish the docs on
>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>> update the docs by following the instruction. Unfortunately, I won't be
>> able to spend any time on this project any longer. You can send your pull
>> requests and questions about the documentation to Denis Magda.
>> >>>
>> >>> -Artem
>> >>>
>> >>> [1] :
>> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>> >>>
>> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> alexey.goncharuk@gmail.com> wrote:
>> >>>>
>> >>>> Alexey,
>> >>>>
>> >>>> I've tried to play with message factories locally, but
>> unfortunately, I
>> >>>> cannot spot the difference between old and new implementation in
>> >>>> distributed benchmarks. I pushed an implementation of
>> MessageFactoryImpl
>> >>>> with the old switch statement to the ignite-2.9-revert-12568 branch
>> >>>> (discussed this with Andrey Gura, the change should be compatible
>> with the
>> >>>> new metrics as we still use the register() mechanics).
>> >>>>
>> >>>> Can you check if this change makes any difference performance-wise
>> in your
>> >>>> environment? If yes, we can go with runtime code generation in the
>> long
>> >>>> term: register classes and generate a dynamic message factory with a
>> switch
>> >>>> statement once all messages are registered (not in 2.9 though,
>> obviously).
>> >>>>
>> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <plehanov.alex@gmail.com
>> >:
>> >>>>
>> >>>> > Hello guys,
>> >>>> >
>> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
>> reduced the
>> >>>> > drop, but not completely removed it.
>> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against
>> 2.8.1
>> >>>> > release on your environment?
>> >>>> > With this patch on our environment, it's about a 3% drop left,
>> it's close
>> >>>> > to measurement error and I think such a drop is not a showstopper.
>> Guys,
>> >>>> > WDYT?
>> >>>> >
>> >>>> > Also, I found that compatibility is broken for JDBC thin driver
>> between 2.8
>> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
>> >>>> > fixed in 2.9. I've prepared the patch.
>> >>>> > Taras Ledkov, can you please review this patch?
>> >>>> >
>> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>> >>>> > send problem in some circumstances). I will cherry-pick it if
>> there is no
>> >>>> > objection.
>> >>>> >
>> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> >>>> > [2] https://github.com/apache/ignite/pull/8223
>> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >>>> >
>> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>> >>>> >
>> >>>> > > Alexey,
>> >>>> > >
>> >>>> > > I propose to include [1] issue to the 2.9 release. Since this
>> issue is
>> >>>> > > related to the new master key change functionality which haven't
>> been
>> >>>> > > released yet I think it will be safe to cherry-pick commit to the
>> >>>> > > release branch.
>> >>>> > >
>> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> >>>> > >
>> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <
>> nizhikov@apache.org>
>> >>>> > wrote:
>> >>>> > > >
>> >>>> > > > Hello, Igniters.
>> >>>> > > >
>> >>>> > > > Alexey, please, include one more Python thin client fix [1]
>> into the
>> >>>> > 2.9
>> >>>> > > release
>> >>>> > > > It fixes kinda major issue - "Python client returns fields in
>> wrong
>> >>>> > > order since the 2 row when fields_count>10"
>> >>>> > > >
>> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> >>>> > > > [2]
>> >>>> > >
>> >>>> >
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> >>>> > > >
>> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> >>>> > alexey.goncharuk@gmail.com>
>> >>>> > > написал(а):
>> >>>> > > > >
>> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize
>> anything out of
>> >>>> > > the
>> >>>> > > > > message factory with suppliers (at least I have no ideas
>> right now),
>> >>>> > so
>> >>>> > > > > most likely the only move here is to switch back to the
>> switch
>> >>>> > approach
>> >>>> > > > > somehow preserving the metrics part. Probably, inlining the
>> Ignite
>> >>>> > > messages
>> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
>> explore
>> >>>> > the
>> >>>> > > > > code a bit.
>> >>>> > > > >
>> >>>> > > > > P.S. I am surprised by the impact this part makes for the
>> >>>> > performance.
>> >>>> > > > > Message creation is indeed on the hot path, but a single
>> virtual call
>> >>>> > > > > should not make that much of a difference given the amount
>> of other
>> >>>> > > work
>> >>>> > > > > happening during the message processing.
>> >>>> > > > >
>> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> plehanov.alex@gmail.com
>> >>>> > >:
>> >>>> > > > >
>> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>> >>>> > Actually,
>> >>>> > > we
>> >>>> > > > >> were looking for a drop using bi-sect in the range between
>> e6a7f93
>> >>>> > > (first
>> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5
>> (last
>> >>>> > > commit in
>> >>>> > > > >> the 2.9 branch). And we found these two problematic commits.
>> >>>> > > > >>
>> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a
>> drop
>> >>>> > between
>> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with
>> reverted
>> >>>> > > IGNITE-13060
>> >>>> > > > >> now and performance looks the same)
>> >>>> > > > >>
>> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>> related to
>> >>>> > > drop
>> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
>> problem, and
>> >>>> > we
>> >>>> > > can
>> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
>> >>>> > > > >>
>> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave
>> it as is?
>> >>>> > > > >>
>> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
>> refactoring)?
>> >>>> > > > >>
>> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> >>>> > > alexey.goncharuk@gmail.com
>> >>>> > > > >>> :
>> >>>> > > > >>
>> >>>> > > > >>> Alexey,
>> >>>> > > > >>>
>> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
>> incorrect fix
>> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch
>> [1], so it
>> >>>> > > cannot be
>> >>>> > > > >>> the source of the drop against 2.8.1.
>> >>>> > > > >>>
>> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate work
>> with fix
>> >>>> > > versions
>> >>>> > > > >>> or develop some sort of tooling to verify the fix versions.
>> >>>> > > > >>>
>> >>>> > > > >>> --AG
>> >>>> > > > >>>
>> >>>> > > > >>> [1]
>> >>>> > > > >>>
>> >>>> > >
>> >>>> >
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> >>>> > > > >>>
>> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> >>>> > > alexey.goncharuk@gmail.com
>> >>>> > > > >>>> :
>> >>>> > > > >>>
>> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> >>>> > plehanov.alex@gmail.com
>> >>>> > > >:
>> >>>> > > > >>>>
>> >>>> > > > >>>>> Guys,
>> >>>> > > > >>>>>
>> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
>> IGNITE-12568
>> >>>> > > (reverted
>> >>>> > > > >>>>> it
>> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
>> >>>> > > > >>>>>
>> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot
>> paths, to
>> >>>> > trace
>> >>>> > > > >>>>> these
>> >>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>> >>>> > > > >>>>>
>> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
>> block was
>> >>>> > > > >>>>> refactored to an array of message suppliers. The message
>> factory
>> >>>> > > is on
>> >>>> > > > >>>>> the
>> >>>> > > > >>>>> hot path, which explains why this commit has an impact
>> on total
>> >>>> > > > >>>>> performance.
>> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
>> microbenchmarks,
>> >>>> > > and
>> >>>> > > > >>>>> found
>> >>>> > > > >>>>> that old implementation of MessageFactory.create() about
>> 30-35%
>> >>>> > > faster
>> >>>> > > > >>>>> than
>> >>>> > > > >>>>> the new one. The reason - approach with switch/case can
>> >>>> > effectively
>> >>>> > > > >>>>> inline
>> >>>> > > > >>>>> message creation code, but with an array of suppliers
>> relatively
>> >>>> > > heavy
>> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to
>> rewrite the
>> >>>> > code
>> >>>> > > > >>>>> using
>> >>>> > > > >>>>> an abstract class for suppliers instead of an interface
>> (to
>> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but
>> it gives
>> >>>> > > back
>> >>>> > > > >>>>> only
>> >>>> > > > >>>>> 10% of method performance and in this case, code looks
>> ugly
>> >>>> > > (lambdas
>> >>>> > > > >>>>> can't
>> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
>> optimize the
>> >>>> > > current
>> >>>> > > > >>>>> approach (except return to the switch/case block).
>> Andrey Gura,
>> >>>> > as
>> >>>> > > the
>> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>> >>>> > > optimization?
>> >>>> > > > >>>>>
>> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are
>> some metrics
>> >>>> > > > >>>>> already
>> >>>> > > > >>>>> created, which can't be rewritten using old message
>> factory
>> >>>> > > > >>>>> implementation
>> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>> >>>> > > > >>>>>
>> >>>> > > > >>>>
>> >>>> > > > >>>> Alexey,
>> >>>> > > > >>>>
>> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
>> released
>> >>>> > > in
>> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
>> present
>> >>>> > > in Ignite
>> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
>> metrics
>> >>>> > > created for
>> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock the
>> release
>> >>>> > > and deal
>> >>>> > > > >>>> with the optimizations in 2.10?
>> >>>> > > > >>>>
>> >>>> > > > >>>
>> >>>> > > >
>> >>>> > >
>> >>>> >
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
During internal testing, we've found a critical bug with discovery (cluster
falls apart if two nodes segmented sequentially). This problem is not
reproduced in 2.8.1. I think we should fix it before release. Under
investigation now. I'll let you know when we get something.
чт, 17 сент. 2020 г. в 00:51, Andrey Gura <ag...@apache.org>:
> > So what do you think? Should we proceed with a 'hacked' version of the
> message factory in 2.9 and go for the runtime message generation in later
> release, or keep the code clean and fix the regression in the next releases?
> > Andrey, can you take a look at my change? I think it is fairly
> straightforward and does not change the semantics, just skips the factory
> closures for certain messages.
>
> IMHO 2.5% isn't too much especially because it isn't actual for all
> workloads (I didn't get any significant drops during benchmarking). So
> I prefer the runtime generation in later releases.
>
> On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
> <al...@gmail.com> wrote:
> >
> > Alexey, Andrey, Igniters,
> >
> > So what do you think? Should we proceed with a 'hacked' version of the
> message factory in 2.9 and go for the runtime message generation in later
> release, or keep the code clean and fix the regression in the next releases?
> > Andrey, can you take a look at my change? I think it is fairly
> straightforward and does not change the semantics, just skips the factory
> closures for certain messages.
> >
> > Personally, I would prefer fixing the regression given that we also
> introduced tracing in this release.
> >
> >
> >
> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
> >>
> >> Alexey,
> >>
> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and
> the performance of ed52559eb95 is better for about 2.5% on average on our
> environment (about the same results we got benchmarking 65c30ec6 vs
> 0606f03d). We've made 24 runs for each commit of
> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
> 50 threads each. Yardstick results for this configuration:
> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>
> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> a.budnikov.ignite@gmail.com>:
> >>>
> >>> Hi Everyone,
> >>>
> >>> I posted an instruction on how to publish the docs on
> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
> update the docs by following the instruction. Unfortunately, I won't be
> able to spend any time on this project any longer. You can send your pull
> requests and questions about the documentation to Denis Magda.
> >>>
> >>> -Artem
> >>>
> >>> [1] :
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>>
> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
> >>>>
> >>>> Alexey,
> >>>>
> >>>> I've tried to play with message factories locally, but unfortunately,
> I
> >>>> cannot spot the difference between old and new implementation in
> >>>> distributed benchmarks. I pushed an implementation of
> MessageFactoryImpl
> >>>> with the old switch statement to the ignite-2.9-revert-12568 branch
> >>>> (discussed this with Andrey Gura, the change should be compatible
> with the
> >>>> new metrics as we still use the register() mechanics).
> >>>>
> >>>> Can you check if this change makes any difference performance-wise in
> your
> >>>> environment? If yes, we can go with runtime code generation in the
> long
> >>>> term: register classes and generate a dynamic message factory with a
> switch
> >>>> statement once all messages are registered (not in 2.9 though,
> obviously).
> >>>>
> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
> >>>>
> >>>> > Hello guys,
> >>>> >
> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
> reduced the
> >>>> > drop, but not completely removed it.
> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
> >>>> > release on your environment?
> >>>> > With this patch on our environment, it's about a 3% drop left, it's
> close
> >>>> > to measurement error and I think such a drop is not a showstopper.
> Guys,
> >>>> > WDYT?
> >>>> >
> >>>> > Also, I found that compatibility is broken for JDBC thin driver
> between 2.8
> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
> >>>> > fixed in 2.9. I've prepared the patch.
> >>>> > Taras Ledkov, can you please review this patch?
> >>>> >
> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
> >>>> > send problem in some circumstances). I will cherry-pick it if there
> is no
> >>>> > objection.
> >>>> >
> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>>> > [2] https://github.com/apache/ignite/pull/8223
> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>>> >
> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
> >>>> >
> >>>> > > Alexey,
> >>>> > >
> >>>> > > I propose to include [1] issue to the 2.9 release. Since this
> issue is
> >>>> > > related to the new master key change functionality which haven't
> been
> >>>> > > released yet I think it will be safe to cherry-pick commit to the
> >>>> > > release branch.
> >>>> > >
> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>>> > >
> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <nizhikov@apache.org
> >
> >>>> > wrote:
> >>>> > > >
> >>>> > > > Hello, Igniters.
> >>>> > > >
> >>>> > > > Alexey, please, include one more Python thin client fix [1]
> into the
> >>>> > 2.9
> >>>> > > release
> >>>> > > > It fixes kinda major issue - "Python client returns fields in
> wrong
> >>>> > > order since the 2 row when fields_count>10"
> >>>> > > >
> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>>> > > > [2]
> >>>> > >
> >>>> >
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>>> > > >
> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>>> > alexey.goncharuk@gmail.com>
> >>>> > > написал(а):
> >>>> > > > >
> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize
> anything out of
> >>>> > > the
> >>>> > > > > message factory with suppliers (at least I have no ideas
> right now),
> >>>> > so
> >>>> > > > > most likely the only move here is to switch back to the switch
> >>>> > approach
> >>>> > > > > somehow preserving the metrics part. Probably, inlining the
> Ignite
> >>>> > > messages
> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
> explore
> >>>> > the
> >>>> > > > > code a bit.
> >>>> > > > >
> >>>> > > > > P.S. I am surprised by the impact this part makes for the
> >>>> > performance.
> >>>> > > > > Message creation is indeed on the hot path, but a single
> virtual call
> >>>> > > > > should not make that much of a difference given the amount of
> other
> >>>> > > work
> >>>> > > > > happening during the message processing.
> >>>> > > > >
> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> plehanov.alex@gmail.com
> >>>> > >:
> >>>> > > > >
> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
> >>>> > Actually,
> >>>> > > we
> >>>> > > > >> were looking for a drop using bi-sect in the range between
> e6a7f93
> >>>> > > (first
> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5
> (last
> >>>> > > commit in
> >>>> > > > >> the 2.9 branch). And we found these two problematic commits.
> >>>> > > > >>
> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
> >>>> > between
> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> >>>> > > IGNITE-13060
> >>>> > > > >> now and performance looks the same)
> >>>> > > > >>
> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> related to
> >>>> > > drop
> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
> problem, and
> >>>> > we
> >>>> > > can
> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
> >>>> > > > >>
> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it
> as is?
> >>>> > > > >>
> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
> refactoring)?
> >>>> > > > >>
> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>>> > > alexey.goncharuk@gmail.com
> >>>> > > > >>> :
> >>>> > > > >>
> >>>> > > > >>> Alexey,
> >>>> > > > >>>
> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
> incorrect fix
> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch [1],
> so it
> >>>> > > cannot be
> >>>> > > > >>> the source of the drop against 2.8.1.
> >>>> > > > >>>
> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate work
> with fix
> >>>> > > versions
> >>>> > > > >>> or develop some sort of tooling to verify the fix versions.
> >>>> > > > >>>
> >>>> > > > >>> --AG
> >>>> > > > >>>
> >>>> > > > >>> [1]
> >>>> > > > >>>
> >>>> > >
> >>>> >
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>> > > > >>>
> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>>> > > alexey.goncharuk@gmail.com
> >>>> > > > >>>> :
> >>>> > > > >>>
> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>>> > plehanov.alex@gmail.com
> >>>> > > >:
> >>>> > > > >>>>
> >>>> > > > >>>>> Guys,
> >>>> > > > >>>>>
> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
> IGNITE-12568
> >>>> > > (reverted
> >>>> > > > >>>>> it
> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
> >>>> > > > >>>>>
> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot
> paths, to
> >>>> > trace
> >>>> > > > >>>>> these
> >>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
> >>>> > > > >>>>>
> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
> block was
> >>>> > > > >>>>> refactored to an array of message suppliers. The message
> factory
> >>>> > > is on
> >>>> > > > >>>>> the
> >>>> > > > >>>>> hot path, which explains why this commit has an impact on
> total
> >>>> > > > >>>>> performance.
> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
> microbenchmarks,
> >>>> > > and
> >>>> > > > >>>>> found
> >>>> > > > >>>>> that old implementation of MessageFactory.create() about
> 30-35%
> >>>> > > faster
> >>>> > > > >>>>> than
> >>>> > > > >>>>> the new one. The reason - approach with switch/case can
> >>>> > effectively
> >>>> > > > >>>>> inline
> >>>> > > > >>>>> message creation code, but with an array of suppliers
> relatively
> >>>> > > heavy
> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to
> rewrite the
> >>>> > code
> >>>> > > > >>>>> using
> >>>> > > > >>>>> an abstract class for suppliers instead of an interface
> (to
> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but
> it gives
> >>>> > > back
> >>>> > > > >>>>> only
> >>>> > > > >>>>> 10% of method performance and in this case, code looks
> ugly
> >>>> > > (lambdas
> >>>> > > > >>>>> can't
> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
> optimize the
> >>>> > > current
> >>>> > > > >>>>> approach (except return to the switch/case block). Andrey
> Gura,
> >>>> > as
> >>>> > > the
> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
> >>>> > > optimization?
> >>>> > > > >>>>>
> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
> metrics
> >>>> > > > >>>>> already
> >>>> > > > >>>>> created, which can't be rewritten using old message
> factory
> >>>> > > > >>>>> implementation
> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
> >>>> > > > >>>>>
> >>>> > > > >>>>
> >>>> > > > >>>> Alexey,
> >>>> > > > >>>>
> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
> released
> >>>> > > in
> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
> present
> >>>> > > in Ignite
> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> metrics
> >>>> > > created for
> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock the
> release
> >>>> > > and deal
> >>>> > > > >>>> with the optimizations in 2.10?
> >>>> > > > >>>>
> >>>> > > > >>>
> >>>> > > >
> >>>> > >
> >>>> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Andrey Gura <ag...@apache.org>.
> So what do you think? Should we proceed with a 'hacked' version of the message factory in 2.9 and go for the runtime message generation in later release, or keep the code clean and fix the regression in the next releases?
> Andrey, can you take a look at my change? I think it is fairly straightforward and does not change the semantics, just skips the factory closures for certain messages.
IMHO 2.5% isn't too much especially because it isn't actual for all
workloads (I didn't get any significant drops during benchmarking). So
I prefer the runtime generation in later releases.
On Mon, Sep 14, 2020 at 12:41 PM Alexey Goncharuk
<al...@gmail.com> wrote:
>
> Alexey, Andrey, Igniters,
>
> So what do you think? Should we proceed with a 'hacked' version of the message factory in 2.9 and go for the runtime message generation in later release, or keep the code clean and fix the regression in the next releases?
> Andrey, can you take a look at my change? I think it is fairly straightforward and does not change the semantics, just skips the factory closures for certain messages.
>
> Personally, I would prefer fixing the regression given that we also introduced tracing in this release.
>
>
>
> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
>>
>> Alexey,
>>
>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and the performance of ed52559eb95 is better for about 2.5% on average on our environment (about the same results we got benchmarking 65c30ec6 vs 0606f03d). We've made 24 runs for each commit of IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients 50 threads each. Yardstick results for this configuration:
>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>
>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <a....@gmail.com>:
>>>
>>> Hi Everyone,
>>>
>>> I posted an instruction on how to publish the docs on ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can update the docs by following the instruction. Unfortunately, I won't be able to spend any time on this project any longer. You can send your pull requests and questions about the documentation to Denis Magda.
>>>
>>> -Artem
>>>
>>> [1] : https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>
>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <al...@gmail.com> wrote:
>>>>
>>>> Alexey,
>>>>
>>>> I've tried to play with message factories locally, but unfortunately, I
>>>> cannot spot the difference between old and new implementation in
>>>> distributed benchmarks. I pushed an implementation of MessageFactoryImpl
>>>> with the old switch statement to the ignite-2.9-revert-12568 branch
>>>> (discussed this with Andrey Gura, the change should be compatible with the
>>>> new metrics as we still use the register() mechanics).
>>>>
>>>> Can you check if this change makes any difference performance-wise in your
>>>> environment? If yes, we can go with runtime code generation in the long
>>>> term: register classes and generate a dynamic message factory with a switch
>>>> statement once all messages are registered (not in 2.9 though, obviously).
>>>>
>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
>>>>
>>>> > Hello guys,
>>>> >
>>>> > I've tried to optimize tracing implementation (ticket [1]), it reduced the
>>>> > drop, but not completely removed it.
>>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>>>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
>>>> > release on your environment?
>>>> > With this patch on our environment, it's about a 3% drop left, it's close
>>>> > to measurement error and I think such a drop is not a showstopper. Guys,
>>>> > WDYT?
>>>> >
>>>> > Also, I found that compatibility is broken for JDBC thin driver between 2.8
>>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
>>>> > fixed in 2.9. I've prepared the patch.
>>>> > Taras Ledkov, can you please review this patch?
>>>> >
>>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>>>> > send problem in some circumstances). I will cherry-pick it if there is no
>>>> > objection.
>>>> >
>>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>> > [2] https://github.com/apache/ignite/pull/8223
>>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>> >
>>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>>>> >
>>>> > > Alexey,
>>>> > >
>>>> > > I propose to include [1] issue to the 2.9 release. Since this issue is
>>>> > > related to the new master key change functionality which haven't been
>>>> > > released yet I think it will be safe to cherry-pick commit to the
>>>> > > release branch.
>>>> > >
>>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>> > >
>>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
>>>> > wrote:
>>>> > > >
>>>> > > > Hello, Igniters.
>>>> > > >
>>>> > > > Alexey, please, include one more Python thin client fix [1] into the
>>>> > 2.9
>>>> > > release
>>>> > > > It fixes kinda major issue - "Python client returns fields in wrong
>>>> > > order since the 2 row when fields_count>10"
>>>> > > >
>>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>> > > > [2]
>>>> > >
>>>> > https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>> > > >
>>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>> > alexey.goncharuk@gmail.com>
>>>> > > написал(а):
>>>> > > > >
>>>> > > > > Alexey, thanks, got it. I am not sure we can optimize anything out of
>>>> > > the
>>>> > > > > message factory with suppliers (at least I have no ideas right now),
>>>> > so
>>>> > > > > most likely the only move here is to switch back to the switch
>>>> > approach
>>>> > > > > somehow preserving the metrics part. Probably, inlining the Ignite
>>>> > > messages
>>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me explore
>>>> > the
>>>> > > > > code a bit.
>>>> > > > >
>>>> > > > > P.S. I am surprised by the impact this part makes for the
>>>> > performance.
>>>> > > > > Message creation is indeed on the hot path, but a single virtual call
>>>> > > > > should not make that much of a difference given the amount of other
>>>> > > work
>>>> > > > > happening during the message processing.
>>>> > > > >
>>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <plehanov.alex@gmail.com
>>>> > >:
>>>> > > > >
>>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>>>> > Actually,
>>>> > > we
>>>> > > > >> were looking for a drop using bi-sect in the range between e6a7f93
>>>> > > (first
>>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
>>>> > > commit in
>>>> > > > >> the 2.9 branch). And we found these two problematic commits.
>>>> > > > >>
>>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
>>>> > between
>>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
>>>> > > IGNITE-13060
>>>> > > > >> now and performance looks the same)
>>>> > > > >>
>>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to
>>>> > > drop
>>>> > > > >> between 2.8.1 and 2.9, but still has some performance problem, and
>>>> > we
>>>> > > can
>>>> > > > >> win back IGNITE-13060 drop by this ticket.
>>>> > > > >>
>>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it as is?
>>>> > > > >>
>>>> > > > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
>>>> > > > >>
>>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>> > > alexey.goncharuk@gmail.com
>>>> > > > >>> :
>>>> > > > >>
>>>> > > > >>> Alexey,
>>>> > > > >>>
>>>> > > > >>> While investigating, I found that IGNITE-12568 has an incorrect fix
>>>> > > > >>> version and is actually present in ignite-2.8.1 branch [1], so it
>>>> > > cannot be
>>>> > > > >>> the source of the drop against 2.8.1.
>>>> > > > >>>
>>>> > > > >>> P.S. Looks like we need to enforce a more accurate work with fix
>>>> > > versions
>>>> > > > >>> or develop some sort of tooling to verify the fix versions.
>>>> > > > >>>
>>>> > > > >>> --AG
>>>> > > > >>>
>>>> > > > >>> [1]
>>>> > > > >>>
>>>> > >
>>>> > https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>> > > > >>>
>>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>> > > alexey.goncharuk@gmail.com
>>>> > > > >>>> :
>>>> > > > >>>
>>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>> > plehanov.alex@gmail.com
>>>> > > >:
>>>> > > > >>>>
>>>> > > > >>>>> Guys,
>>>> > > > >>>>>
>>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
>>>> > > (reverted
>>>> > > > >>>>> it
>>>> > > > >>>>> locally) and got the same performance as on 2.8.1
>>>> > > > >>>>>
>>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to
>>>> > trace
>>>> > > > >>>>> these
>>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>>>> > > > >>>>>
>>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>>>> > > > >>>>> refactored to an array of message suppliers. The message factory
>>>> > > is on
>>>> > > > >>>>> the
>>>> > > > >>>>> hot path, which explains why this commit has an impact on total
>>>> > > > >>>>> performance.
>>>> > > > >>>>> I've checked JIT assembly output, done some JMH microbenchmarks,
>>>> > > and
>>>> > > > >>>>> found
>>>> > > > >>>>> that old implementation of MessageFactory.create() about 30-35%
>>>> > > faster
>>>> > > > >>>>> than
>>>> > > > >>>>> the new one. The reason - approach with switch/case can
>>>> > effectively
>>>> > > > >>>>> inline
>>>> > > > >>>>> message creation code, but with an array of suppliers relatively
>>>> > > heavy
>>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the
>>>> > code
>>>> > > > >>>>> using
>>>> > > > >>>>> an abstract class for suppliers instead of an interface (to
>>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it gives
>>>> > > back
>>>> > > > >>>>> only
>>>> > > > >>>>> 10% of method performance and in this case, code looks ugly
>>>> > > (lambdas
>>>> > > > >>>>> can't
>>>> > > > >>>>> be used). Currently, I can't find any more ways to optimize the
>>>> > > current
>>>> > > > >>>>> approach (except return to the switch/case block). Andrey Gura,
>>>> > as
>>>> > > the
>>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>>>> > > optimization?
>>>> > > > >>>>>
>>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
>>>> > > > >>>>> already
>>>> > > > >>>>> created, which can't be rewritten using old message factory
>>>> > > > >>>>> implementation
>>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>>>> > > > >>>>>
>>>> > > > >>>>
>>>> > > > >>>> Alexey,
>>>> > > > >>>>
>>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already released
>>>> > > in
>>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present
>>>> > > in Ignite
>>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
>>>> > > created for
>>>> > > > >>>> 2.9 that depend on the new message factory to unblock the release
>>>> > > and deal
>>>> > > > >>>> with the optimizations in 2.10?
>>>> > > > >>>>
>>>> > > > >>>
>>>> > > >
>>>> > >
>>>> >
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Aleksandr Shapkin <le...@gmail.com>.
Hi guys,
Alex,
Currently, we are working on the Apache Ignite Kubernetes Operator that
should help with packaging, deploying, and managing the application in the
k8s world. And it turned out that current docker images are not suitable
for running in k8s, cause they won't propagate the signals to the JVM
itself. For sample, you can’t just stop a k8s pod with a deployed Apache
Ignite instance gracefully.
I’d like to include the IGNITE-13453 [1] improvement into the release scope
I understand, that the most stabilization work has been done already, but
there are no changes in the codebase itself, the changes only affect the
docker deployment thus could be tested separately and won’t break the
existing benchmarks/tests, etc.
[1] - https://issues.apache.org/jira/browse/IGNITE-13453
вт, 15 сент. 2020 г. в 10:47, Alex Plehanov <pl...@gmail.com>:
> Alexey,
>
> We've benchmarked the whole set of tests tonight in both persistent and
> in-memory modes on the current 2.9 branch (with minor tracing optimization
> merged). And on our environment, we've got 0-2.5% drop for each benchmark
> compared to 2.8.1. I think it is an acceptable result.
> So, I'm ok with keeping the message factory as is and fix the regression in
> the next release.
> Guys, WDYT?
>
> пн, 14 сент. 2020 г. в 12:41, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
> > Alexey, Andrey, Igniters,
> >
> > So what do you think? Should we proceed with a 'hacked' version of the
> > message factory in 2.9 and go for the runtime message generation in later
> > release, or keep the code clean and fix the regression in the next
> > releases?
> > Andrey, can you take a look at my change? I think it is fairly
> > straightforward and does not change the semantics, just skips the factory
> > closures for certain messages.
> >
> > Personally, I would prefer fixing the regression given that we also
> > introduced tracing in this release.
> >
> >
> >
> > пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
> >
> >> Alexey,
> >>
> >> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and
> the
> >> performance of ed52559eb95 is better for about 2.5% on average on our
> >> environment (about the same results we got benchmarking 65c30ec6 vs
> >> 0606f03d). We've made 24 runs for each commit of
> >> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> >> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5
> clients
> >> 50 threads each. Yardstick results for this configuration:
> >> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> >> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
> >>
> >> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <
> a.budnikov.ignite@gmail.com
> >> >:
> >>
> >>> Hi Everyone,
> >>>
> >>> I posted an instruction on how to publish the docs on
> >>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
> >>> update the docs by following the instruction. Unfortunately, I won't be
> >>> able to spend any time on this project any longer. You can send your
> pull
> >>> requests and questions about the documentation to Denis Magda.
> >>>
> >>> -Artem
> >>>
> >>> [1] :
> https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
> >>>
> >>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> >>> alexey.goncharuk@gmail.com> wrote:
> >>>
> >>>> Alexey,
> >>>>
> >>>> I've tried to play with message factories locally, but unfortunately,
> I
> >>>> cannot spot the difference between old and new implementation in
> >>>> distributed benchmarks. I pushed an implementation of
> MessageFactoryImpl
> >>>> with the old switch statement to the ignite-2.9-revert-12568 branch
> >>>> (discussed this with Andrey Gura, the change should be compatible with
> >>>> the
> >>>> new metrics as we still use the register() mechanics).
> >>>>
> >>>> Can you check if this change makes any difference performance-wise in
> >>>> your
> >>>> environment? If yes, we can go with runtime code generation in the
> long
> >>>> term: register classes and generate a dynamic message factory with a
> >>>> switch
> >>>> statement once all messages are registered (not in 2.9 though,
> >>>> obviously).
> >>>>
> >>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
> >>>>
> >>>> > Hello guys,
> >>>> >
> >>>> > I've tried to optimize tracing implementation (ticket [1]), it
> >>>> reduced the
> >>>> > drop, but not completely removed it.
> >>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
> >>>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
> >>>> > release on your environment?
> >>>> > With this patch on our environment, it's about a 3% drop left, it's
> >>>> close
> >>>> > to measurement error and I think such a drop is not a showstopper.
> >>>> Guys,
> >>>> > WDYT?
> >>>> >
> >>>> > Also, I found that compatibility is broken for JDBC thin driver
> >>>> between 2.8
> >>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
> >>>> > fixed in 2.9. I've prepared the patch.
> >>>> > Taras Ledkov, can you please review this patch?
> >>>> >
> >>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
> >>>> > send problem in some circumstances). I will cherry-pick it if there
> >>>> is no
> >>>> > objection.
> >>>> >
> >>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
> >>>> > [2] https://github.com/apache/ignite/pull/8223
> >>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
> >>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >>>> >
> >>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
> >>>> >
> >>>> > > Alexey,
> >>>> > >
> >>>> > > I propose to include [1] issue to the 2.9 release. Since this
> issue
> >>>> is
> >>>> > > related to the new master key change functionality which haven't
> >>>> been
> >>>> > > released yet I think it will be safe to cherry-pick commit to the
> >>>> > > release branch.
> >>>> > >
> >>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >>>> > >
> >>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <nizhikov@apache.org
> >
> >>>> > wrote:
> >>>> > > >
> >>>> > > > Hello, Igniters.
> >>>> > > >
> >>>> > > > Alexey, please, include one more Python thin client fix [1] into
> >>>> the
> >>>> > 2.9
> >>>> > > release
> >>>> > > > It fixes kinda major issue - "Python client returns fields in
> >>>> wrong
> >>>> > > order since the 2 row when fields_count>10"
> >>>> > > >
> >>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> >>>> > > > [2]
> >>>> > >
> >>>> >
> >>>>
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >>>> > > >
> >>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> >>>> > alexey.goncharuk@gmail.com>
> >>>> > > написал(а):
> >>>> > > > >
> >>>> > > > > Alexey, thanks, got it. I am not sure we can optimize anything
> >>>> out of
> >>>> > > the
> >>>> > > > > message factory with suppliers (at least I have no ideas right
> >>>> now),
> >>>> > so
> >>>> > > > > most likely the only move here is to switch back to the switch
> >>>> > approach
> >>>> > > > > somehow preserving the metrics part. Probably, inlining the
> >>>> Ignite
> >>>> > > messages
> >>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
> >>>> explore
> >>>> > the
> >>>> > > > > code a bit.
> >>>> > > > >
> >>>> > > > > P.S. I am surprised by the impact this part makes for the
> >>>> > performance.
> >>>> > > > > Message creation is indeed on the hot path, but a single
> >>>> virtual call
> >>>> > > > > should not make that much of a difference given the amount of
> >>>> other
> >>>> > > work
> >>>> > > > > happening during the message processing.
> >>>> > > > >
> >>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> >>>> plehanov.alex@gmail.com
> >>>> > >:
> >>>> > > > >
> >>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
> >>>> > Actually,
> >>>> > > we
> >>>> > > > >> were looking for a drop using bi-sect in the range between
> >>>> e6a7f93
> >>>> > > (first
> >>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5
> >>>> (last
> >>>> > > commit in
> >>>> > > > >> the 2.9 branch). And we found these two problematic commits.
> >>>> > > > >>
> >>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
> >>>> > between
> >>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> >>>> > > IGNITE-13060
> >>>> > > > >> now and performance looks the same)
> >>>> > > > >>
> >>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
> >>>> related to
> >>>> > > drop
> >>>> > > > >> between 2.8.1 and 2.9, but still has some performance
> problem,
> >>>> and
> >>>> > we
> >>>> > > can
> >>>> > > > >> win back IGNITE-13060 drop by this ticket.
> >>>> > > > >>
> >>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it
> >>>> as is?
> >>>> > > > >>
> >>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
> >>>> refactoring)?
> >>>> > > > >>
> >>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> >>>> > > alexey.goncharuk@gmail.com
> >>>> > > > >>> :
> >>>> > > > >>
> >>>> > > > >>> Alexey,
> >>>> > > > >>>
> >>>> > > > >>> While investigating, I found that IGNITE-12568 has an
> >>>> incorrect fix
> >>>> > > > >>> version and is actually present in ignite-2.8.1 branch [1],
> >>>> so it
> >>>> > > cannot be
> >>>> > > > >>> the source of the drop against 2.8.1.
> >>>> > > > >>>
> >>>> > > > >>> P.S. Looks like we need to enforce a more accurate work with
> >>>> fix
> >>>> > > versions
> >>>> > > > >>> or develop some sort of tooling to verify the fix versions.
> >>>> > > > >>>
> >>>> > > > >>> --AG
> >>>> > > > >>>
> >>>> > > > >>> [1]
> >>>> > > > >>>
> >>>> > >
> >>>> >
> >>>>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>> > > > >>>
> >>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> >>>> > > alexey.goncharuk@gmail.com
> >>>> > > > >>>> :
> >>>> > > > >>>
> >>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> >>>> > plehanov.alex@gmail.com
> >>>> > > >:
> >>>> > > > >>>>
> >>>> > > > >>>>> Guys,
> >>>> > > > >>>>>
> >>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
> >>>> IGNITE-12568
> >>>> > > (reverted
> >>>> > > > >>>>> it
> >>>> > > > >>>>> locally) and got the same performance as on 2.8.1
> >>>> > > > >>>>>
> >>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths,
> >>>> to
> >>>> > trace
> >>>> > > > >>>>> these
> >>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
> >>>> > > > >>>>>
> >>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
> >>>> block was
> >>>> > > > >>>>> refactored to an array of message suppliers. The message
> >>>> factory
> >>>> > > is on
> >>>> > > > >>>>> the
> >>>> > > > >>>>> hot path, which explains why this commit has an impact on
> >>>> total
> >>>> > > > >>>>> performance.
> >>>> > > > >>>>> I've checked JIT assembly output, done some JMH
> >>>> microbenchmarks,
> >>>> > > and
> >>>> > > > >>>>> found
> >>>> > > > >>>>> that old implementation of MessageFactory.create() about
> >>>> 30-35%
> >>>> > > faster
> >>>> > > > >>>>> than
> >>>> > > > >>>>> the new one. The reason - approach with switch/case can
> >>>> > effectively
> >>>> > > > >>>>> inline
> >>>> > > > >>>>> message creation code, but with an array of suppliers
> >>>> relatively
> >>>> > > heavy
> >>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite
> >>>> the
> >>>> > code
> >>>> > > > >>>>> using
> >>>> > > > >>>>> an abstract class for suppliers instead of an interface
> (to
> >>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but
> it
> >>>> gives
> >>>> > > back
> >>>> > > > >>>>> only
> >>>> > > > >>>>> 10% of method performance and in this case, code looks
> ugly
> >>>> > > (lambdas
> >>>> > > > >>>>> can't
> >>>> > > > >>>>> be used). Currently, I can't find any more ways to
> optimize
> >>>> the
> >>>> > > current
> >>>> > > > >>>>> approach (except return to the switch/case block). Andrey
> >>>> Gura,
> >>>> > as
> >>>> > > the
> >>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
> >>>> > > optimization?
> >>>> > > > >>>>>
> >>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
> >>>> metrics
> >>>> > > > >>>>> already
> >>>> > > > >>>>> created, which can't be rewritten using old message
> factory
> >>>> > > > >>>>> implementation
> >>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
> >>>> > > > >>>>>
> >>>> > > > >>>>
> >>>> > > > >>>> Alexey,
> >>>> > > > >>>>
> >>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
> >>>> released
> >>>> > > in
> >>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
> >>>> present
> >>>> > > in Ignite
> >>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new
> metrics
> >>>> > > created for
> >>>> > > > >>>> 2.9 that depend on the new message factory to unblock the
> >>>> release
> >>>> > > and deal
> >>>> > > > >>>> with the optimizations in 2.10?
> >>>> > > > >>>>
> >>>> > > > >>>
> >>>> > > >
> >>>> > >
> >>>> >
> >>>>
> >>>
>
--
Alex.
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Alexey,
We've benchmarked the whole set of tests tonight in both persistent and
in-memory modes on the current 2.9 branch (with minor tracing optimization
merged). And on our environment, we've got 0-2.5% drop for each benchmark
compared to 2.8.1. I think it is an acceptable result.
So, I'm ok with keeping the message factory as is and fix the regression in
the next release.
Guys, WDYT?
пн, 14 сент. 2020 г. в 12:41, Alexey Goncharuk <al...@gmail.com>:
> Alexey, Andrey, Igniters,
>
> So what do you think? Should we proceed with a 'hacked' version of the
> message factory in 2.9 and go for the runtime message generation in later
> release, or keep the code clean and fix the regression in the next
> releases?
> Andrey, can you take a look at my change? I think it is fairly
> straightforward and does not change the semantics, just skips the factory
> closures for certain messages.
>
> Personally, I would prefer fixing the regression given that we also
> introduced tracing in this release.
>
>
>
> пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
>
>> Alexey,
>>
>> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and the
>> performance of ed52559eb95 is better for about 2.5% on average on our
>> environment (about the same results we got benchmarking 65c30ec6 vs
>> 0606f03d). We've made 24 runs for each commit of
>> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
>> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
>> 50 threads each. Yardstick results for this configuration:
>> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
>> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>>
>> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <a.budnikov.ignite@gmail.com
>> >:
>>
>>> Hi Everyone,
>>>
>>> I posted an instruction on how to publish the docs on
>>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>>> update the docs by following the instruction. Unfortunately, I won't be
>>> able to spend any time on this project any longer. You can send your pull
>>> requests and questions about the documentation to Denis Magda.
>>>
>>> -Artem
>>>
>>> [1] : https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>>
>>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>>> alexey.goncharuk@gmail.com> wrote:
>>>
>>>> Alexey,
>>>>
>>>> I've tried to play with message factories locally, but unfortunately, I
>>>> cannot spot the difference between old and new implementation in
>>>> distributed benchmarks. I pushed an implementation of MessageFactoryImpl
>>>> with the old switch statement to the ignite-2.9-revert-12568 branch
>>>> (discussed this with Andrey Gura, the change should be compatible with
>>>> the
>>>> new metrics as we still use the register() mechanics).
>>>>
>>>> Can you check if this change makes any difference performance-wise in
>>>> your
>>>> environment? If yes, we can go with runtime code generation in the long
>>>> term: register classes and generate a dynamic message factory with a
>>>> switch
>>>> statement once all messages are registered (not in 2.9 though,
>>>> obviously).
>>>>
>>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
>>>>
>>>> > Hello guys,
>>>> >
>>>> > I've tried to optimize tracing implementation (ticket [1]), it
>>>> reduced the
>>>> > drop, but not completely removed it.
>>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>>>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
>>>> > release on your environment?
>>>> > With this patch on our environment, it's about a 3% drop left, it's
>>>> close
>>>> > to measurement error and I think such a drop is not a showstopper.
>>>> Guys,
>>>> > WDYT?
>>>> >
>>>> > Also, I found that compatibility is broken for JDBC thin driver
>>>> between 2.8
>>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
>>>> > fixed in 2.9. I've prepared the patch.
>>>> > Taras Ledkov, can you please review this patch?
>>>> >
>>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>>>> > send problem in some circumstances). I will cherry-pick it if there
>>>> is no
>>>> > objection.
>>>> >
>>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>>> > [2] https://github.com/apache/ignite/pull/8223
>>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>>> >
>>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>>>> >
>>>> > > Alexey,
>>>> > >
>>>> > > I propose to include [1] issue to the 2.9 release. Since this issue
>>>> is
>>>> > > related to the new master key change functionality which haven't
>>>> been
>>>> > > released yet I think it will be safe to cherry-pick commit to the
>>>> > > release branch.
>>>> > >
>>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>>> > >
>>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
>>>> > wrote:
>>>> > > >
>>>> > > > Hello, Igniters.
>>>> > > >
>>>> > > > Alexey, please, include one more Python thin client fix [1] into
>>>> the
>>>> > 2.9
>>>> > > release
>>>> > > > It fixes kinda major issue - "Python client returns fields in
>>>> wrong
>>>> > > order since the 2 row when fields_count>10"
>>>> > > >
>>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>>> > > > [2]
>>>> > >
>>>> >
>>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>>> > > >
>>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>>> > alexey.goncharuk@gmail.com>
>>>> > > написал(а):
>>>> > > > >
>>>> > > > > Alexey, thanks, got it. I am not sure we can optimize anything
>>>> out of
>>>> > > the
>>>> > > > > message factory with suppliers (at least I have no ideas right
>>>> now),
>>>> > so
>>>> > > > > most likely the only move here is to switch back to the switch
>>>> > approach
>>>> > > > > somehow preserving the metrics part. Probably, inlining the
>>>> Ignite
>>>> > > messages
>>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
>>>> explore
>>>> > the
>>>> > > > > code a bit.
>>>> > > > >
>>>> > > > > P.S. I am surprised by the impact this part makes for the
>>>> > performance.
>>>> > > > > Message creation is indeed on the hot path, but a single
>>>> virtual call
>>>> > > > > should not make that much of a difference given the amount of
>>>> other
>>>> > > work
>>>> > > > > happening during the message processing.
>>>> > > > >
>>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>> > >:
>>>> > > > >
>>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>>>> > Actually,
>>>> > > we
>>>> > > > >> were looking for a drop using bi-sect in the range between
>>>> e6a7f93
>>>> > > (first
>>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5
>>>> (last
>>>> > > commit in
>>>> > > > >> the 2.9 branch). And we found these two problematic commits.
>>>> > > > >>
>>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
>>>> > between
>>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
>>>> > > IGNITE-13060
>>>> > > > >> now and performance looks the same)
>>>> > > > >>
>>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not
>>>> related to
>>>> > > drop
>>>> > > > >> between 2.8.1 and 2.9, but still has some performance problem,
>>>> and
>>>> > we
>>>> > > can
>>>> > > > >> win back IGNITE-13060 drop by this ticket.
>>>> > > > >>
>>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it
>>>> as is?
>>>> > > > >>
>>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
>>>> refactoring)?
>>>> > > > >>
>>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>>> > > alexey.goncharuk@gmail.com
>>>> > > > >>> :
>>>> > > > >>
>>>> > > > >>> Alexey,
>>>> > > > >>>
>>>> > > > >>> While investigating, I found that IGNITE-12568 has an
>>>> incorrect fix
>>>> > > > >>> version and is actually present in ignite-2.8.1 branch [1],
>>>> so it
>>>> > > cannot be
>>>> > > > >>> the source of the drop against 2.8.1.
>>>> > > > >>>
>>>> > > > >>> P.S. Looks like we need to enforce a more accurate work with
>>>> fix
>>>> > > versions
>>>> > > > >>> or develop some sort of tooling to verify the fix versions.
>>>> > > > >>>
>>>> > > > >>> --AG
>>>> > > > >>>
>>>> > > > >>> [1]
>>>> > > > >>>
>>>> > >
>>>> >
>>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>> > > > >>>
>>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>>> > > alexey.goncharuk@gmail.com
>>>> > > > >>>> :
>>>> > > > >>>
>>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>>> > plehanov.alex@gmail.com
>>>> > > >:
>>>> > > > >>>>
>>>> > > > >>>>> Guys,
>>>> > > > >>>>>
>>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and
>>>> IGNITE-12568
>>>> > > (reverted
>>>> > > > >>>>> it
>>>> > > > >>>>> locally) and got the same performance as on 2.8.1
>>>> > > > >>>>>
>>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths,
>>>> to
>>>> > trace
>>>> > > > >>>>> these
>>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>>>> > > > >>>>>
>>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
>>>> block was
>>>> > > > >>>>> refactored to an array of message suppliers. The message
>>>> factory
>>>> > > is on
>>>> > > > >>>>> the
>>>> > > > >>>>> hot path, which explains why this commit has an impact on
>>>> total
>>>> > > > >>>>> performance.
>>>> > > > >>>>> I've checked JIT assembly output, done some JMH
>>>> microbenchmarks,
>>>> > > and
>>>> > > > >>>>> found
>>>> > > > >>>>> that old implementation of MessageFactory.create() about
>>>> 30-35%
>>>> > > faster
>>>> > > > >>>>> than
>>>> > > > >>>>> the new one. The reason - approach with switch/case can
>>>> > effectively
>>>> > > > >>>>> inline
>>>> > > > >>>>> message creation code, but with an array of suppliers
>>>> relatively
>>>> > > heavy
>>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite
>>>> the
>>>> > code
>>>> > > > >>>>> using
>>>> > > > >>>>> an abstract class for suppliers instead of an interface (to
>>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it
>>>> gives
>>>> > > back
>>>> > > > >>>>> only
>>>> > > > >>>>> 10% of method performance and in this case, code looks ugly
>>>> > > (lambdas
>>>> > > > >>>>> can't
>>>> > > > >>>>> be used). Currently, I can't find any more ways to optimize
>>>> the
>>>> > > current
>>>> > > > >>>>> approach (except return to the switch/case block). Andrey
>>>> Gura,
>>>> > as
>>>> > > the
>>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>>>> > > optimization?
>>>> > > > >>>>>
>>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
>>>> metrics
>>>> > > > >>>>> already
>>>> > > > >>>>> created, which can't be rewritten using old message factory
>>>> > > > >>>>> implementation
>>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>>>> > > > >>>>>
>>>> > > > >>>>
>>>> > > > >>>> Alexey,
>>>> > > > >>>>
>>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
>>>> released
>>>> > > in
>>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
>>>> present
>>>> > > in Ignite
>>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
>>>> > > created for
>>>> > > > >>>> 2.9 that depend on the new message factory to unblock the
>>>> release
>>>> > > and deal
>>>> > > > >>>> with the optimizations in 2.10?
>>>> > > > >>>>
>>>> > > > >>>
>>>> > > >
>>>> > >
>>>> >
>>>>
>>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexey, Andrey, Igniters,
So what do you think? Should we proceed with a 'hacked' version of the
message factory in 2.9 and go for the runtime message generation in later
release, or keep the code clean and fix the regression in the next
releases?
Andrey, can you take a look at my change? I think it is fairly
straightforward and does not change the semantics, just skips the factory
closures for certain messages.
Personally, I would prefer fixing the regression given that we also
introduced tracing in this release.
пт, 11 сент. 2020 г. в 12:09, Alex Plehanov <pl...@gmail.com>:
> Alexey,
>
> We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and the
> performance of ed52559eb95 is better for about 2.5% on average on our
> environment (about the same results we got benchmarking 65c30ec6 vs
> 0606f03d). We've made 24 runs for each commit of
> IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
> benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
> 50 threads each. Yardstick results for this configuration:
> Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
> Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
>
> пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <a.budnikov.ignite@gmail.com
> >:
>
>> Hi Everyone,
>>
>> I posted an instruction on how to publish the docs on
>> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
>> update the docs by following the instruction. Unfortunately, I won't be
>> able to spend any time on this project any longer. You can send your pull
>> requests and questions about the documentation to Denis Magda.
>>
>> -Artem
>>
>> [1] : https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>>
>> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
>> alexey.goncharuk@gmail.com> wrote:
>>
>>> Alexey,
>>>
>>> I've tried to play with message factories locally, but unfortunately, I
>>> cannot spot the difference between old and new implementation in
>>> distributed benchmarks. I pushed an implementation of MessageFactoryImpl
>>> with the old switch statement to the ignite-2.9-revert-12568 branch
>>> (discussed this with Andrey Gura, the change should be compatible with
>>> the
>>> new metrics as we still use the register() mechanics).
>>>
>>> Can you check if this change makes any difference performance-wise in
>>> your
>>> environment? If yes, we can go with runtime code generation in the long
>>> term: register classes and generate a dynamic message factory with a
>>> switch
>>> statement once all messages are registered (not in 2.9 though,
>>> obviously).
>>>
>>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
>>>
>>> > Hello guys,
>>> >
>>> > I've tried to optimize tracing implementation (ticket [1]), it reduced
>>> the
>>> > drop, but not completely removed it.
>>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
>>> > release on your environment?
>>> > With this patch on our environment, it's about a 3% drop left, it's
>>> close
>>> > to measurement error and I think such a drop is not a showstopper.
>>> Guys,
>>> > WDYT?
>>> >
>>> > Also, I found that compatibility is broken for JDBC thin driver
>>> between 2.8
>>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
>>> > fixed in 2.9. I've prepared the patch.
>>> > Taras Ledkov, can you please review this patch?
>>> >
>>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>>> > send problem in some circumstances). I will cherry-pick it if there is
>>> no
>>> > objection.
>>> >
>>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>>> > [2] https://github.com/apache/ignite/pull/8223
>>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>>> >
>>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>>> >
>>> > > Alexey,
>>> > >
>>> > > I propose to include [1] issue to the 2.9 release. Since this issue
>>> is
>>> > > related to the new master key change functionality which haven't been
>>> > > released yet I think it will be safe to cherry-pick commit to the
>>> > > release branch.
>>> > >
>>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>>> > >
>>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
>>> > wrote:
>>> > > >
>>> > > > Hello, Igniters.
>>> > > >
>>> > > > Alexey, please, include one more Python thin client fix [1] into
>>> the
>>> > 2.9
>>> > > release
>>> > > > It fixes kinda major issue - "Python client returns fields in wrong
>>> > > order since the 2 row when fields_count>10"
>>> > > >
>>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>>> > > > [2]
>>> > >
>>> >
>>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>>> > > >
>>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>>> > alexey.goncharuk@gmail.com>
>>> > > написал(а):
>>> > > > >
>>> > > > > Alexey, thanks, got it. I am not sure we can optimize anything
>>> out of
>>> > > the
>>> > > > > message factory with suppliers (at least I have no ideas right
>>> now),
>>> > so
>>> > > > > most likely the only move here is to switch back to the switch
>>> > approach
>>> > > > > somehow preserving the metrics part. Probably, inlining the
>>> Ignite
>>> > > messages
>>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
>>> explore
>>> > the
>>> > > > > code a bit.
>>> > > > >
>>> > > > > P.S. I am surprised by the impact this part makes for the
>>> > performance.
>>> > > > > Message creation is indeed on the hot path, but a single virtual
>>> call
>>> > > > > should not make that much of a difference given the amount of
>>> other
>>> > > work
>>> > > > > happening during the message processing.
>>> > > > >
>>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>>> plehanov.alex@gmail.com
>>> > >:
>>> > > > >
>>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>>> > Actually,
>>> > > we
>>> > > > >> were looking for a drop using bi-sect in the range between
>>> e6a7f93
>>> > > (first
>>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5
>>> (last
>>> > > commit in
>>> > > > >> the 2.9 branch). And we found these two problematic commits.
>>> > > > >>
>>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
>>> > between
>>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
>>> > > IGNITE-13060
>>> > > > >> now and performance looks the same)
>>> > > > >>
>>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related
>>> to
>>> > > drop
>>> > > > >> between 2.8.1 and 2.9, but still has some performance problem,
>>> and
>>> > we
>>> > > can
>>> > > > >> win back IGNITE-13060 drop by this ticket.
>>> > > > >>
>>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it as
>>> is?
>>> > > > >>
>>> > > > >> What should we do with IGNITE-12568 (MessageFactory
>>> refactoring)?
>>> > > > >>
>>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>>> > > alexey.goncharuk@gmail.com
>>> > > > >>> :
>>> > > > >>
>>> > > > >>> Alexey,
>>> > > > >>>
>>> > > > >>> While investigating, I found that IGNITE-12568 has an
>>> incorrect fix
>>> > > > >>> version and is actually present in ignite-2.8.1 branch [1], so
>>> it
>>> > > cannot be
>>> > > > >>> the source of the drop against 2.8.1.
>>> > > > >>>
>>> > > > >>> P.S. Looks like we need to enforce a more accurate work with
>>> fix
>>> > > versions
>>> > > > >>> or develop some sort of tooling to verify the fix versions.
>>> > > > >>>
>>> > > > >>> --AG
>>> > > > >>>
>>> > > > >>> [1]
>>> > > > >>>
>>> > >
>>> >
>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>> > > > >>>
>>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>>> > > alexey.goncharuk@gmail.com
>>> > > > >>>> :
>>> > > > >>>
>>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>>> > plehanov.alex@gmail.com
>>> > > >:
>>> > > > >>>>
>>> > > > >>>>> Guys,
>>> > > > >>>>>
>>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
>>> > > (reverted
>>> > > > >>>>> it
>>> > > > >>>>> locally) and got the same performance as on 2.8.1
>>> > > > >>>>>
>>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to
>>> > trace
>>> > > > >>>>> these
>>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>>> > > > >>>>>
>>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case
>>> block was
>>> > > > >>>>> refactored to an array of message suppliers. The message
>>> factory
>>> > > is on
>>> > > > >>>>> the
>>> > > > >>>>> hot path, which explains why this commit has an impact on
>>> total
>>> > > > >>>>> performance.
>>> > > > >>>>> I've checked JIT assembly output, done some JMH
>>> microbenchmarks,
>>> > > and
>>> > > > >>>>> found
>>> > > > >>>>> that old implementation of MessageFactory.create() about
>>> 30-35%
>>> > > faster
>>> > > > >>>>> than
>>> > > > >>>>> the new one. The reason - approach with switch/case can
>>> > effectively
>>> > > > >>>>> inline
>>> > > > >>>>> message creation code, but with an array of suppliers
>>> relatively
>>> > > heavy
>>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite
>>> the
>>> > code
>>> > > > >>>>> using
>>> > > > >>>>> an abstract class for suppliers instead of an interface (to
>>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it
>>> gives
>>> > > back
>>> > > > >>>>> only
>>> > > > >>>>> 10% of method performance and in this case, code looks ugly
>>> > > (lambdas
>>> > > > >>>>> can't
>>> > > > >>>>> be used). Currently, I can't find any more ways to optimize
>>> the
>>> > > current
>>> > > > >>>>> approach (except return to the switch/case block). Andrey
>>> Gura,
>>> > as
>>> > > the
>>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>>> > > optimization?
>>> > > > >>>>>
>>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
>>> metrics
>>> > > > >>>>> already
>>> > > > >>>>> created, which can't be rewritten using old message factory
>>> > > > >>>>> implementation
>>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>>> > > > >>>>>
>>> > > > >>>>
>>> > > > >>>> Alexey,
>>> > > > >>>>
>>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
>>> released
>>> > > in
>>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
>>> present
>>> > > in Ignite
>>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
>>> > > created for
>>> > > > >>>> 2.9 that depend on the new message factory to unblock the
>>> release
>>> > > and deal
>>> > > > >>>> with the optimizations in 2.10?
>>> > > > >>>>
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Alexey,
We've benchmarked by yardstick commits 130376741bf vs ed52559eb95 and the
performance of ed52559eb95 is better for about 2.5% on average on our
environment (about the same results we got benchmarking 65c30ec6 vs
0606f03d). We've made 24 runs for each commit of
IgnitePutTxImplicitBenchmark (we got maximum drop for 2.9 on this
benchmark), 200 seconds warmup, 300 seconds benchmark, 6 servers, 5 clients
50 threads each. Yardstick results for this configuration:
Commit 130376741bf: avg TPS=164096, avg latency=9173464 ns
Commit ed52559eb95: avg TPS=168283, avg latency=8945908 ns
пт, 11 сент. 2020 г. в 09:51, Artem Budnikov <a....@gmail.com>:
> Hi Everyone,
>
> I posted an instruction on how to publish the docs on
> ignite.apache.org/docs [1]. When you finish with Ignite 2.9, you can
> update the docs by following the instruction. Unfortunately, I won't be
> able to spend any time on this project any longer. You can send your pull
> requests and questions about the documentation to Denis Magda.
>
> -Artem
>
> [1] : https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
>
> On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <
> alexey.goncharuk@gmail.com> wrote:
>
>> Alexey,
>>
>> I've tried to play with message factories locally, but unfortunately, I
>> cannot spot the difference between old and new implementation in
>> distributed benchmarks. I pushed an implementation of MessageFactoryImpl
>> with the old switch statement to the ignite-2.9-revert-12568 branch
>> (discussed this with Andrey Gura, the change should be compatible with the
>> new metrics as we still use the register() mechanics).
>>
>> Can you check if this change makes any difference performance-wise in your
>> environment? If yes, we can go with runtime code generation in the long
>> term: register classes and generate a dynamic message factory with a
>> switch
>> statement once all messages are registered (not in 2.9 though, obviously).
>>
>> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
>>
>> > Hello guys,
>> >
>> > I've tried to optimize tracing implementation (ticket [1]), it reduced
>> the
>> > drop, but not completely removed it.
>> > Ivan Rakov, Alexander Lapin, can you please review the patch?
>> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
>> > release on your environment?
>> > With this patch on our environment, it's about a 3% drop left, it's
>> close
>> > to measurement error and I think such a drop is not a showstopper. Guys,
>> > WDYT?
>> >
>> > Also, I found that compatibility is broken for JDBC thin driver between
>> 2.8
>> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
>> > fixed in 2.9. I've prepared the patch.
>> > Taras Ledkov, can you please review this patch?
>> >
>> > And one more ticket I propose to include into 2.9 [4] (NIO message
>> > send problem in some circumstances). I will cherry-pick it if there is
>> no
>> > objection.
>> >
>> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
>> > [2] https://github.com/apache/ignite/pull/8223
>> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
>> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
>> >
>> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>> >
>> > > Alexey,
>> > >
>> > > I propose to include [1] issue to the 2.9 release. Since this issue is
>> > > related to the new master key change functionality which haven't been
>> > > released yet I think it will be safe to cherry-pick commit to the
>> > > release branch.
>> > >
>> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
>> > >
>> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
>> > wrote:
>> > > >
>> > > > Hello, Igniters.
>> > > >
>> > > > Alexey, please, include one more Python thin client fix [1] into the
>> > 2.9
>> > > release
>> > > > It fixes kinda major issue - "Python client returns fields in wrong
>> > > order since the 2 row when fields_count>10"
>> > > >
>> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
>> > > > [2]
>> > >
>> >
>> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>> > > >
>> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
>> > alexey.goncharuk@gmail.com>
>> > > написал(а):
>> > > > >
>> > > > > Alexey, thanks, got it. I am not sure we can optimize anything
>> out of
>> > > the
>> > > > > message factory with suppliers (at least I have no ideas right
>> now),
>> > so
>> > > > > most likely the only move here is to switch back to the switch
>> > approach
>> > > > > somehow preserving the metrics part. Probably, inlining the Ignite
>> > > messages
>> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me
>> explore
>> > the
>> > > > > code a bit.
>> > > > >
>> > > > > P.S. I am surprised by the impact this part makes for the
>> > performance.
>> > > > > Message creation is indeed on the hot path, but a single virtual
>> call
>> > > > > should not make that much of a difference given the amount of
>> other
>> > > work
>> > > > > happening during the message processing.
>> > > > >
>> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
>> plehanov.alex@gmail.com
>> > >:
>> > > > >
>> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
>> > Actually,
>> > > we
>> > > > >> were looking for a drop using bi-sect in the range between
>> e6a7f93
>> > > (first
>> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
>> > > commit in
>> > > > >> the 2.9 branch). And we found these two problematic commits.
>> > > > >>
>> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
>> > between
>> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
>> > > IGNITE-13060
>> > > > >> now and performance looks the same)
>> > > > >>
>> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related
>> to
>> > > drop
>> > > > >> between 2.8.1 and 2.9, but still has some performance problem,
>> and
>> > we
>> > > can
>> > > > >> win back IGNITE-13060 drop by this ticket.
>> > > > >>
>> > > > >> Do we need more investigation on IGNITE-13060 or we leave it as
>> is?
>> > > > >>
>> > > > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
>> > > > >>
>> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
>> > > alexey.goncharuk@gmail.com
>> > > > >>> :
>> > > > >>
>> > > > >>> Alexey,
>> > > > >>>
>> > > > >>> While investigating, I found that IGNITE-12568 has an incorrect
>> fix
>> > > > >>> version and is actually present in ignite-2.8.1 branch [1], so
>> it
>> > > cannot be
>> > > > >>> the source of the drop against 2.8.1.
>> > > > >>>
>> > > > >>> P.S. Looks like we need to enforce a more accurate work with fix
>> > > versions
>> > > > >>> or develop some sort of tooling to verify the fix versions.
>> > > > >>>
>> > > > >>> --AG
>> > > > >>>
>> > > > >>> [1]
>> > > > >>>
>> > >
>> >
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>> > > > >>>
>> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
>> > > alexey.goncharuk@gmail.com
>> > > > >>>> :
>> > > > >>>
>> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
>> > plehanov.alex@gmail.com
>> > > >:
>> > > > >>>>
>> > > > >>>>> Guys,
>> > > > >>>>>
>> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
>> > > (reverted
>> > > > >>>>> it
>> > > > >>>>> locally) and got the same performance as on 2.8.1
>> > > > >>>>>
>> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to
>> > trace
>> > > > >>>>> these
>> > > > >>>>> hot paths, it's clear why we have performance drop here.
>> > > > >>>>>
>> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block
>> was
>> > > > >>>>> refactored to an array of message suppliers. The message
>> factory
>> > > is on
>> > > > >>>>> the
>> > > > >>>>> hot path, which explains why this commit has an impact on
>> total
>> > > > >>>>> performance.
>> > > > >>>>> I've checked JIT assembly output, done some JMH
>> microbenchmarks,
>> > > and
>> > > > >>>>> found
>> > > > >>>>> that old implementation of MessageFactory.create() about
>> 30-35%
>> > > faster
>> > > > >>>>> than
>> > > > >>>>> the new one. The reason - approach with switch/case can
>> > effectively
>> > > > >>>>> inline
>> > > > >>>>> message creation code, but with an array of suppliers
>> relatively
>> > > heavy
>> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the
>> > code
>> > > > >>>>> using
>> > > > >>>>> an abstract class for suppliers instead of an interface (to
>> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it
>> gives
>> > > back
>> > > > >>>>> only
>> > > > >>>>> 10% of method performance and in this case, code looks ugly
>> > > (lambdas
>> > > > >>>>> can't
>> > > > >>>>> be used). Currently, I can't find any more ways to optimize
>> the
>> > > current
>> > > > >>>>> approach (except return to the switch/case block). Andrey
>> Gura,
>> > as
>> > > the
>> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
>> > > optimization?
>> > > > >>>>>
>> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
>> metrics
>> > > > >>>>> already
>> > > > >>>>> created, which can't be rewritten using old message factory
>> > > > >>>>> implementation
>> > > > >>>>> (IGNITE-12756). Guys, WDYT?
>> > > > >>>>>
>> > > > >>>>
>> > > > >>>> Alexey,
>> > > > >>>>
>> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
>> released
>> > > in
>> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
>> present
>> > > in Ignite
>> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
>> > > created for
>> > > > >>>> 2.9 that depend on the new message factory to unblock the
>> release
>> > > and deal
>> > > > >>>> with the optimizations in 2.10?
>> > > > >>>>
>> > > > >>>
>> > > >
>> > >
>> >
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
Hi Everyone,
I posted an instruction on how to publish the docs on ignite.apache.org/docs
[1]. When you finish with Ignite 2.9, you can update the docs by following
the instruction. Unfortunately, I won't be able to spend any time on this
project any longer. You can send your pull requests and questions about the
documentation to Denis Magda.
-Artem
[1] : https://cwiki.apache.org/confluence/display/IGNITE/How+to+Document
On Thu, Sep 10, 2020 at 2:45 PM Alexey Goncharuk <al...@gmail.com>
wrote:
> Alexey,
>
> I've tried to play with message factories locally, but unfortunately, I
> cannot spot the difference between old and new implementation in
> distributed benchmarks. I pushed an implementation of MessageFactoryImpl
> with the old switch statement to the ignite-2.9-revert-12568 branch
> (discussed this with Andrey Gura, the change should be compatible with the
> new metrics as we still use the register() mechanics).
>
> Can you check if this change makes any difference performance-wise in your
> environment? If yes, we can go with runtime code generation in the long
> term: register classes and generate a dynamic message factory with a switch
> statement once all messages are registered (not in 2.9 though, obviously).
>
> ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
>
> > Hello guys,
> >
> > I've tried to optimize tracing implementation (ticket [1]), it reduced
> the
> > drop, but not completely removed it.
> > Ivan Rakov, Alexander Lapin, can you please review the patch?
> > Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
> > release on your environment?
> > With this patch on our environment, it's about a 3% drop left, it's close
> > to measurement error and I think such a drop is not a showstopper. Guys,
> > WDYT?
> >
> > Also, I found that compatibility is broken for JDBC thin driver between
> 2.8
> > and 2.9 versions (ticket [3]). I think it's a blocker and should be
> > fixed in 2.9. I've prepared the patch.
> > Taras Ledkov, can you please review this patch?
> >
> > And one more ticket I propose to include into 2.9 [4] (NIO message
> > send problem in some circumstances). I will cherry-pick it if there is no
> > objection.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-13411
> > [2] https://github.com/apache/ignite/pull/8223
> > [3] https://issues.apache.org/jira/browse/IGNITE-13414
> > [4] https://issues.apache.org/jira/browse/IGNITE-13361
> >
> > пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
> >
> > > Alexey,
> > >
> > > I propose to include [1] issue to the 2.9 release. Since this issue is
> > > related to the new master key change functionality which haven't been
> > > released yet I think it will be safe to cherry-pick commit to the
> > > release branch.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> > >
> > > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
> > wrote:
> > > >
> > > > Hello, Igniters.
> > > >
> > > > Alexey, please, include one more Python thin client fix [1] into the
> > 2.9
> > > release
> > > > It fixes kinda major issue - "Python client returns fields in wrong
> > > order since the 2 row when fields_count>10"
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> > > > [2]
> > >
> >
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> > > >
> > > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com>
> > > написал(а):
> > > > >
> > > > > Alexey, thanks, got it. I am not sure we can optimize anything out
> of
> > > the
> > > > > message factory with suppliers (at least I have no ideas right
> now),
> > so
> > > > > most likely the only move here is to switch back to the switch
> > approach
> > > > > somehow preserving the metrics part. Probably, inlining the Ignite
> > > messages
> > > > > to the IgniteMessageFactoryImpl should do the trick. Let me explore
> > the
> > > > > code a bit.
> > > > >
> > > > > P.S. I am surprised by the impact this part makes for the
> > performance.
> > > > > Message creation is indeed on the hot path, but a single virtual
> call
> > > > > should not make that much of a difference given the amount of other
> > > work
> > > > > happening during the message processing.
> > > > >
> > > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <
> plehanov.alex@gmail.com
> > >:
> > > > >
> > > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
> > Actually,
> > > we
> > > > >> were looking for a drop using bi-sect in the range between e6a7f93
> > > (first
> > > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
> > > commit in
> > > > >> the 2.9 branch). And we found these two problematic commits.
> > > > >>
> > > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
> > between
> > > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> > > IGNITE-13060
> > > > >> now and performance looks the same)
> > > > >>
> > > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to
> > > drop
> > > > >> between 2.8.1 and 2.9, but still has some performance problem, and
> > we
> > > can
> > > > >> win back IGNITE-13060 drop by this ticket.
> > > > >>
> > > > >> Do we need more investigation on IGNITE-13060 or we leave it as
> is?
> > > > >>
> > > > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> > > > >>
> > > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >>> :
> > > > >>
> > > > >>> Alexey,
> > > > >>>
> > > > >>> While investigating, I found that IGNITE-12568 has an incorrect
> fix
> > > > >>> version and is actually present in ignite-2.8.1 branch [1], so it
> > > cannot be
> > > > >>> the source of the drop against 2.8.1.
> > > > >>>
> > > > >>> P.S. Looks like we need to enforce a more accurate work with fix
> > > versions
> > > > >>> or develop some sort of tooling to verify the fix versions.
> > > > >>>
> > > > >>> --AG
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > >
> >
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > > > >>>
> > > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> > > alexey.goncharuk@gmail.com
> > > > >>>> :
> > > > >>>
> > > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> > plehanov.alex@gmail.com
> > > >:
> > > > >>>>
> > > > >>>>> Guys,
> > > > >>>>>
> > > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
> > > (reverted
> > > > >>>>> it
> > > > >>>>> locally) and got the same performance as on 2.8.1
> > > > >>>>>
> > > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to
> > trace
> > > > >>>>> these
> > > > >>>>> hot paths, it's clear why we have performance drop here.
> > > > >>>>>
> > > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block
> was
> > > > >>>>> refactored to an array of message suppliers. The message
> factory
> > > is on
> > > > >>>>> the
> > > > >>>>> hot path, which explains why this commit has an impact on total
> > > > >>>>> performance.
> > > > >>>>> I've checked JIT assembly output, done some JMH
> microbenchmarks,
> > > and
> > > > >>>>> found
> > > > >>>>> that old implementation of MessageFactory.create() about 30-35%
> > > faster
> > > > >>>>> than
> > > > >>>>> the new one. The reason - approach with switch/case can
> > effectively
> > > > >>>>> inline
> > > > >>>>> message creation code, but with an array of suppliers
> relatively
> > > heavy
> > > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the
> > code
> > > > >>>>> using
> > > > >>>>> an abstract class for suppliers instead of an interface (to
> > > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it
> gives
> > > back
> > > > >>>>> only
> > > > >>>>> 10% of method performance and in this case, code looks ugly
> > > (lambdas
> > > > >>>>> can't
> > > > >>>>> be used). Currently, I can't find any more ways to optimize the
> > > current
> > > > >>>>> approach (except return to the switch/case block). Andrey Gura,
> > as
> > > the
> > > > >>>>> author of IGNITE-12568, maybe you have some ideas about
> > > optimization?
> > > > >>>>>
> > > > >>>>> Perhaps we should revert IGNITE-12568, but there are some
> metrics
> > > > >>>>> already
> > > > >>>>> created, which can't be rewritten using old message factory
> > > > >>>>> implementation
> > > > >>>>> (IGNITE-12756). Guys, WDYT?
> > > > >>>>>
> > > > >>>>
> > > > >>>> Alexey,
> > > > >>>>
> > > > >>>> I see that IGNITE-12756 (metrics improvements) is already
> released
> > > in
> > > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only
> present
> > > in Ignite
> > > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
> > > created for
> > > > >>>> 2.9 that depend on the new message factory to unblock the
> release
> > > and deal
> > > > >>>> with the optimizations in 2.10?
> > > > >>>>
> > > > >>>
> > > >
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexey,
I've tried to play with message factories locally, but unfortunately, I
cannot spot the difference between old and new implementation in
distributed benchmarks. I pushed an implementation of MessageFactoryImpl
with the old switch statement to the ignite-2.9-revert-12568 branch
(discussed this with Andrey Gura, the change should be compatible with the
new metrics as we still use the register() mechanics).
Can you check if this change makes any difference performance-wise in your
environment? If yes, we can go with runtime code generation in the long
term: register classes and generate a dynamic message factory with a switch
statement once all messages are registered (not in 2.9 though, obviously).
ср, 9 сент. 2020 г. в 14:53, Alex Plehanov <pl...@gmail.com>:
> Hello guys,
>
> I've tried to optimize tracing implementation (ticket [1]), it reduced the
> drop, but not completely removed it.
> Ivan Rakov, Alexander Lapin, can you please review the patch?
> Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
> release on your environment?
> With this patch on our environment, it's about a 3% drop left, it's close
> to measurement error and I think such a drop is not a showstopper. Guys,
> WDYT?
>
> Also, I found that compatibility is broken for JDBC thin driver between 2.8
> and 2.9 versions (ticket [3]). I think it's a blocker and should be
> fixed in 2.9. I've prepared the patch.
> Taras Ledkov, can you please review this patch?
>
> And one more ticket I propose to include into 2.9 [4] (NIO message
> send problem in some circumstances). I will cherry-pick it if there is no
> objection.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13411
> [2] https://github.com/apache/ignite/pull/8223
> [3] https://issues.apache.org/jira/browse/IGNITE-13414
> [4] https://issues.apache.org/jira/browse/IGNITE-13361
>
> пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
>
> > Alexey,
> >
> > I propose to include [1] issue to the 2.9 release. Since this issue is
> > related to the new master key change functionality which haven't been
> > released yet I think it will be safe to cherry-pick commit to the
> > release branch.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-13390
> >
> > On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org>
> wrote:
> > >
> > > Hello, Igniters.
> > >
> > > Alexey, please, include one more Python thin client fix [1] into the
> 2.9
> > release
> > > It fixes kinda major issue - "Python client returns fields in wrong
> > order since the 2 row when fields_count>10"
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> > > [2]
> >
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> > >
> > > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <
> alexey.goncharuk@gmail.com>
> > написал(а):
> > > >
> > > > Alexey, thanks, got it. I am not sure we can optimize anything out of
> > the
> > > > message factory with suppliers (at least I have no ideas right now),
> so
> > > > most likely the only move here is to switch back to the switch
> approach
> > > > somehow preserving the metrics part. Probably, inlining the Ignite
> > messages
> > > > to the IgniteMessageFactoryImpl should do the trick. Let me explore
> the
> > > > code a bit.
> > > >
> > > > P.S. I am surprised by the impact this part makes for the
> performance.
> > > > Message creation is indeed on the hot path, but a single virtual call
> > > > should not make that much of a difference given the amount of other
> > work
> > > > happening during the message processing.
> > > >
> > > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <plehanov.alex@gmail.com
> >:
> > > >
> > > >> Alexey, sorry, I wrongly interpreted our benchmark results.
> Actually,
> > we
> > > >> were looking for a drop using bi-sect in the range between e6a7f93
> > (first
> > > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
> > commit in
> > > >> the 2.9 branch). And we found these two problematic commits.
> > > >>
> > > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop
> between
> > > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> > IGNITE-13060
> > > >> now and performance looks the same)
> > > >>
> > > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to
> > drop
> > > >> between 2.8.1 and 2.9, but still has some performance problem, and
> we
> > can
> > > >> win back IGNITE-13060 drop by this ticket.
> > > >>
> > > >> Do we need more investigation on IGNITE-13060 or we leave it as is?
> > > >>
> > > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> > > >>
> > > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >>> :
> > > >>
> > > >>> Alexey,
> > > >>>
> > > >>> While investigating, I found that IGNITE-12568 has an incorrect fix
> > > >>> version and is actually present in ignite-2.8.1 branch [1], so it
> > cannot be
> > > >>> the source of the drop against 2.8.1.
> > > >>>
> > > >>> P.S. Looks like we need to enforce a more accurate work with fix
> > versions
> > > >>> or develop some sort of tooling to verify the fix versions.
> > > >>>
> > > >>> --AG
> > > >>>
> > > >>> [1]
> > > >>>
> >
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > > >>>
> > > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> > alexey.goncharuk@gmail.com
> > > >>>> :
> > > >>>
> > > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <
> plehanov.alex@gmail.com
> > >:
> > > >>>>
> > > >>>>> Guys,
> > > >>>>>
> > > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
> > (reverted
> > > >>>>> it
> > > >>>>> locally) and got the same performance as on 2.8.1
> > > >>>>>
> > > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to
> trace
> > > >>>>> these
> > > >>>>> hot paths, it's clear why we have performance drop here.
> > > >>>>>
> > > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > > >>>>> refactored to an array of message suppliers. The message factory
> > is on
> > > >>>>> the
> > > >>>>> hot path, which explains why this commit has an impact on total
> > > >>>>> performance.
> > > >>>>> I've checked JIT assembly output, done some JMH microbenchmarks,
> > and
> > > >>>>> found
> > > >>>>> that old implementation of MessageFactory.create() about 30-35%
> > faster
> > > >>>>> than
> > > >>>>> the new one. The reason - approach with switch/case can
> effectively
> > > >>>>> inline
> > > >>>>> message creation code, but with an array of suppliers relatively
> > heavy
> > > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the
> code
> > > >>>>> using
> > > >>>>> an abstract class for suppliers instead of an interface (to
> > > >>>>> replace "invokeinterface" with the "invokevirtual"), but it gives
> > back
> > > >>>>> only
> > > >>>>> 10% of method performance and in this case, code looks ugly
> > (lambdas
> > > >>>>> can't
> > > >>>>> be used). Currently, I can't find any more ways to optimize the
> > current
> > > >>>>> approach (except return to the switch/case block). Andrey Gura,
> as
> > the
> > > >>>>> author of IGNITE-12568, maybe you have some ideas about
> > optimization?
> > > >>>>>
> > > >>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
> > > >>>>> already
> > > >>>>> created, which can't be rewritten using old message factory
> > > >>>>> implementation
> > > >>>>> (IGNITE-12756). Guys, WDYT?
> > > >>>>>
> > > >>>>
> > > >>>> Alexey,
> > > >>>>
> > > >>>> I see that IGNITE-12756 (metrics improvements) is already released
> > in
> > > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present
> > in Ignite
> > > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
> > created for
> > > >>>> 2.9 that depend on the new message factory to unblock the release
> > and deal
> > > >>>> with the optimizations in 2.10?
> > > >>>>
> > > >>>
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Hello guys,
I've tried to optimize tracing implementation (ticket [1]), it reduced the
drop, but not completely removed it.
Ivan Rakov, Alexander Lapin, can you please review the patch?
Ivan Artiukhov, can you please benchmark the patch [2] against 2.8.1
release on your environment?
With this patch on our environment, it's about a 3% drop left, it's close
to measurement error and I think such a drop is not a showstopper. Guys,
WDYT?
Also, I found that compatibility is broken for JDBC thin driver between 2.8
and 2.9 versions (ticket [3]). I think it's a blocker and should be
fixed in 2.9. I've prepared the patch.
Taras Ledkov, can you please review this patch?
And one more ticket I propose to include into 2.9 [4] (NIO message
send problem in some circumstances). I will cherry-pick it if there is no
objection.
[1] https://issues.apache.org/jira/browse/IGNITE-13411
[2] https://github.com/apache/ignite/pull/8223
[3] https://issues.apache.org/jira/browse/IGNITE-13414
[4] https://issues.apache.org/jira/browse/IGNITE-13361
пн, 7 сент. 2020 г. в 14:14, Maxim Muzafarov <mm...@apache.org>:
> Alexey,
>
> I propose to include [1] issue to the 2.9 release. Since this issue is
> related to the new master key change functionality which haven't been
> released yet I think it will be safe to cherry-pick commit to the
> release branch.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13390
>
> On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org> wrote:
> >
> > Hello, Igniters.
> >
> > Alexey, please, include one more Python thin client fix [1] into the 2.9
> release
> > It fixes kinda major issue - "Python client returns fields in wrong
> order since the 2 row when fields_count>10"
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12809
> > [2]
> https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> >
> > > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <al...@gmail.com>
> написал(а):
> > >
> > > Alexey, thanks, got it. I am not sure we can optimize anything out of
> the
> > > message factory with suppliers (at least I have no ideas right now), so
> > > most likely the only move here is to switch back to the switch approach
> > > somehow preserving the metrics part. Probably, inlining the Ignite
> messages
> > > to the IgniteMessageFactoryImpl should do the trick. Let me explore the
> > > code a bit.
> > >
> > > P.S. I am surprised by the impact this part makes for the performance.
> > > Message creation is indeed on the hot path, but a single virtual call
> > > should not make that much of a difference given the amount of other
> work
> > > happening during the message processing.
> > >
> > > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <pl...@gmail.com>:
> > >
> > >> Alexey, sorry, I wrongly interpreted our benchmark results. Actually,
> we
> > >> were looking for a drop using bi-sect in the range between e6a7f93
> (first
> > >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last
> commit in
> > >> the 2.9 branch). And we found these two problematic commits.
> > >>
> > >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
> > >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted
> IGNITE-13060
> > >> now and performance looks the same)
> > >>
> > >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to
> drop
> > >> between 2.8.1 and 2.9, but still has some performance problem, and we
> can
> > >> win back IGNITE-13060 drop by this ticket.
> > >>
> > >> Do we need more investigation on IGNITE-13060 or we leave it as is?
> > >>
> > >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> > >>
> > >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >>> :
> > >>
> > >>> Alexey,
> > >>>
> > >>> While investigating, I found that IGNITE-12568 has an incorrect fix
> > >>> version and is actually present in ignite-2.8.1 branch [1], so it
> cannot be
> > >>> the source of the drop against 2.8.1.
> > >>>
> > >>> P.S. Looks like we need to enforce a more accurate work with fix
> versions
> > >>> or develop some sort of tooling to verify the fix versions.
> > >>>
> > >>> --AG
> > >>>
> > >>> [1]
> > >>>
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> > >>>
> > >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <
> alexey.goncharuk@gmail.com
> > >>>> :
> > >>>
> > >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <plehanov.alex@gmail.com
> >:
> > >>>>
> > >>>>> Guys,
> > >>>>>
> > >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568
> (reverted
> > >>>>> it
> > >>>>> locally) and got the same performance as on 2.8.1
> > >>>>>
> > >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
> > >>>>> these
> > >>>>> hot paths, it's clear why we have performance drop here.
> > >>>>>
> > >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > >>>>> refactored to an array of message suppliers. The message factory
> is on
> > >>>>> the
> > >>>>> hot path, which explains why this commit has an impact on total
> > >>>>> performance.
> > >>>>> I've checked JIT assembly output, done some JMH microbenchmarks,
> and
> > >>>>> found
> > >>>>> that old implementation of MessageFactory.create() about 30-35%
> faster
> > >>>>> than
> > >>>>> the new one. The reason - approach with switch/case can effectively
> > >>>>> inline
> > >>>>> message creation code, but with an array of suppliers relatively
> heavy
> > >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code
> > >>>>> using
> > >>>>> an abstract class for suppliers instead of an interface (to
> > >>>>> replace "invokeinterface" with the "invokevirtual"), but it gives
> back
> > >>>>> only
> > >>>>> 10% of method performance and in this case, code looks ugly
> (lambdas
> > >>>>> can't
> > >>>>> be used). Currently, I can't find any more ways to optimize the
> current
> > >>>>> approach (except return to the switch/case block). Andrey Gura, as
> the
> > >>>>> author of IGNITE-12568, maybe you have some ideas about
> optimization?
> > >>>>>
> > >>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
> > >>>>> already
> > >>>>> created, which can't be rewritten using old message factory
> > >>>>> implementation
> > >>>>> (IGNITE-12756). Guys, WDYT?
> > >>>>>
> > >>>>
> > >>>> Alexey,
> > >>>>
> > >>>> I see that IGNITE-12756 (metrics improvements) is already released
> in
> > >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present
> in Ignite
> > >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics
> created for
> > >>>> 2.9 that depend on the new message factory to unblock the release
> and deal
> > >>>> with the optimizations in 2.10?
> > >>>>
> > >>>
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Maxim Muzafarov <mm...@apache.org>.
Alexey,
I propose to include [1] issue to the 2.9 release. Since this issue is
related to the new master key change functionality which haven't been
released yet I think it will be safe to cherry-pick commit to the
release branch.
[1] https://issues.apache.org/jira/browse/IGNITE-13390
On Tue, 1 Sep 2020 at 12:13, Nikolay Izhikov <ni...@apache.org> wrote:
>
> Hello, Igniters.
>
> Alexey, please, include one more Python thin client fix [1] into the 2.9 release
> It fixes kinda major issue - "Python client returns fields in wrong order since the 2 row when fields_count>10"
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12809
> [2] https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
>
> > 31 авг. 2020 г., в 19:23, Alexey Goncharuk <al...@gmail.com> написал(а):
> >
> > Alexey, thanks, got it. I am not sure we can optimize anything out of the
> > message factory with suppliers (at least I have no ideas right now), so
> > most likely the only move here is to switch back to the switch approach
> > somehow preserving the metrics part. Probably, inlining the Ignite messages
> > to the IgniteMessageFactoryImpl should do the trick. Let me explore the
> > code a bit.
> >
> > P.S. I am surprised by the impact this part makes for the performance.
> > Message creation is indeed on the hot path, but a single virtual call
> > should not make that much of a difference given the amount of other work
> > happening during the message processing.
> >
> > пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <pl...@gmail.com>:
> >
> >> Alexey, sorry, I wrongly interpreted our benchmark results. Actually, we
> >> were looking for a drop using bi-sect in the range between e6a7f93 (first
> >> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last commit in
> >> the 2.9 branch). And we found these two problematic commits.
> >>
> >> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
> >> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted IGNITE-13060
> >> now and performance looks the same)
> >>
> >> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to drop
> >> between 2.8.1 and 2.9, but still has some performance problem, and we can
> >> win back IGNITE-13060 drop by this ticket.
> >>
> >> Do we need more investigation on IGNITE-13060 or we leave it as is?
> >>
> >> What should we do with IGNITE-12568 (MessageFactory refactoring)?
> >>
> >> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >>> :
> >>
> >>> Alexey,
> >>>
> >>> While investigating, I found that IGNITE-12568 has an incorrect fix
> >>> version and is actually present in ignite-2.8.1 branch [1], so it cannot be
> >>> the source of the drop against 2.8.1.
> >>>
> >>> P.S. Looks like we need to enforce a more accurate work with fix versions
> >>> or develop some sort of tooling to verify the fix versions.
> >>>
> >>> --AG
> >>>
> >>> [1]
> >>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
> >>>
> >>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >>>> :
> >>>
> >>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
> >>>>
> >>>>> Guys,
> >>>>>
> >>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
> >>>>> it
> >>>>> locally) and got the same performance as on 2.8.1
> >>>>>
> >>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
> >>>>> these
> >>>>> hot paths, it's clear why we have performance drop here.
> >>>>>
> >>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> >>>>> refactored to an array of message suppliers. The message factory is on
> >>>>> the
> >>>>> hot path, which explains why this commit has an impact on total
> >>>>> performance.
> >>>>> I've checked JIT assembly output, done some JMH microbenchmarks, and
> >>>>> found
> >>>>> that old implementation of MessageFactory.create() about 30-35% faster
> >>>>> than
> >>>>> the new one. The reason - approach with switch/case can effectively
> >>>>> inline
> >>>>> message creation code, but with an array of suppliers relatively heavy
> >>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code
> >>>>> using
> >>>>> an abstract class for suppliers instead of an interface (to
> >>>>> replace "invokeinterface" with the "invokevirtual"), but it gives back
> >>>>> only
> >>>>> 10% of method performance and in this case, code looks ugly (lambdas
> >>>>> can't
> >>>>> be used). Currently, I can't find any more ways to optimize the current
> >>>>> approach (except return to the switch/case block). Andrey Gura, as the
> >>>>> author of IGNITE-12568, maybe you have some ideas about optimization?
> >>>>>
> >>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
> >>>>> already
> >>>>> created, which can't be rewritten using old message factory
> >>>>> implementation
> >>>>> (IGNITE-12756). Guys, WDYT?
> >>>>>
> >>>>
> >>>> Alexey,
> >>>>
> >>>> I see that IGNITE-12756 (metrics improvements) is already released in
> >>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
> >>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
> >>>> 2.9 that depend on the new message factory to unblock the release and deal
> >>>> with the optimizations in 2.10?
> >>>>
> >>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
Hello, Igniters.
Alexey, please, include one more Python thin client fix [1] into the 2.9 release
It fixes kinda major issue - "Python client returns fields in wrong order since the 2 row when fields_count>10"
[1] https://issues.apache.org/jira/browse/IGNITE-12809
[2] https://github.com/apache/ignite/commit/38025ee4167f05eaa2d6a2c5c2ab70c83a462cfc
> 31 авг. 2020 г., в 19:23, Alexey Goncharuk <al...@gmail.com> написал(а):
>
> Alexey, thanks, got it. I am not sure we can optimize anything out of the
> message factory with suppliers (at least I have no ideas right now), so
> most likely the only move here is to switch back to the switch approach
> somehow preserving the metrics part. Probably, inlining the Ignite messages
> to the IgniteMessageFactoryImpl should do the trick. Let me explore the
> code a bit.
>
> P.S. I am surprised by the impact this part makes for the performance.
> Message creation is indeed on the hot path, but a single virtual call
> should not make that much of a difference given the amount of other work
> happening during the message processing.
>
> пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <pl...@gmail.com>:
>
>> Alexey, sorry, I wrongly interpreted our benchmark results. Actually, we
>> were looking for a drop using bi-sect in the range between e6a7f93 (first
>> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last commit in
>> the 2.9 branch). And we found these two problematic commits.
>>
>> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
>> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted IGNITE-13060
>> now and performance looks the same)
>>
>> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to drop
>> between 2.8.1 and 2.9, but still has some performance problem, and we can
>> win back IGNITE-13060 drop by this ticket.
>>
>> Do we need more investigation on IGNITE-13060 or we leave it as is?
>>
>> What should we do with IGNITE-12568 (MessageFactory refactoring)?
>>
>> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <alexey.goncharuk@gmail.com
>>> :
>>
>>> Alexey,
>>>
>>> While investigating, I found that IGNITE-12568 has an incorrect fix
>>> version and is actually present in ignite-2.8.1 branch [1], so it cannot be
>>> the source of the drop against 2.8.1.
>>>
>>> P.S. Looks like we need to enforce a more accurate work with fix versions
>>> or develop some sort of tooling to verify the fix versions.
>>>
>>> --AG
>>>
>>> [1]
>>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>>
>>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <alexey.goncharuk@gmail.com
>>>> :
>>>
>>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
>>>>
>>>>> Guys,
>>>>>
>>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
>>>>> it
>>>>> locally) and got the same performance as on 2.8.1
>>>>>
>>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
>>>>> these
>>>>> hot paths, it's clear why we have performance drop here.
>>>>>
>>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>>>>> refactored to an array of message suppliers. The message factory is on
>>>>> the
>>>>> hot path, which explains why this commit has an impact on total
>>>>> performance.
>>>>> I've checked JIT assembly output, done some JMH microbenchmarks, and
>>>>> found
>>>>> that old implementation of MessageFactory.create() about 30-35% faster
>>>>> than
>>>>> the new one. The reason - approach with switch/case can effectively
>>>>> inline
>>>>> message creation code, but with an array of suppliers relatively heavy
>>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code
>>>>> using
>>>>> an abstract class for suppliers instead of an interface (to
>>>>> replace "invokeinterface" with the "invokevirtual"), but it gives back
>>>>> only
>>>>> 10% of method performance and in this case, code looks ugly (lambdas
>>>>> can't
>>>>> be used). Currently, I can't find any more ways to optimize the current
>>>>> approach (except return to the switch/case block). Andrey Gura, as the
>>>>> author of IGNITE-12568, maybe you have some ideas about optimization?
>>>>>
>>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
>>>>> already
>>>>> created, which can't be rewritten using old message factory
>>>>> implementation
>>>>> (IGNITE-12756). Guys, WDYT?
>>>>>
>>>>
>>>> Alexey,
>>>>
>>>> I see that IGNITE-12756 (metrics improvements) is already released in
>>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
>>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
>>>> 2.9 that depend on the new message factory to unblock the release and deal
>>>> with the optimizations in 2.10?
>>>>
>>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexey, thanks, got it. I am not sure we can optimize anything out of the
message factory with suppliers (at least I have no ideas right now), so
most likely the only move here is to switch back to the switch approach
somehow preserving the metrics part. Probably, inlining the Ignite messages
to the IgniteMessageFactoryImpl should do the trick. Let me explore the
code a bit.
P.S. I am surprised by the impact this part makes for the performance.
Message creation is indeed on the hot path, but a single virtual call
should not make that much of a difference given the amount of other work
happening during the message processing.
пн, 31 авг. 2020 г. в 18:33, Alex Plehanov <pl...@gmail.com>:
> Alexey, sorry, I wrongly interpreted our benchmark results. Actually, we
> were looking for a drop using bi-sect in the range between e6a7f93 (first
> commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last commit in
> the 2.9 branch). And we found these two problematic commits.
>
> Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between
> 2.8.1 and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted IGNITE-13060
> now and performance looks the same)
>
> Ticket IGNITE-12568 (MessageFactory refactoring) is not related to drop
> between 2.8.1 and 2.9, but still has some performance problem, and we can
> win back IGNITE-13060 drop by this ticket.
>
> Do we need more investigation on IGNITE-13060 or we leave it as is?
>
> What should we do with IGNITE-12568 (MessageFactory refactoring)?
>
> пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
>> Alexey,
>>
>> While investigating, I found that IGNITE-12568 has an incorrect fix
>> version and is actually present in ignite-2.8.1 branch [1], so it cannot be
>> the source of the drop against 2.8.1.
>>
>> P.S. Looks like we need to enforce a more accurate work with fix versions
>> or develop some sort of tooling to verify the fix versions.
>>
>> --AG
>>
>> [1]
>> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>>
>> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <alexey.goncharuk@gmail.com
>> >:
>>
>>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
>>>
>>>> Guys,
>>>>
>>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
>>>> it
>>>> locally) and got the same performance as on 2.8.1
>>>>
>>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace
>>>> these
>>>> hot paths, it's clear why we have performance drop here.
>>>>
>>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>>>> refactored to an array of message suppliers. The message factory is on
>>>> the
>>>> hot path, which explains why this commit has an impact on total
>>>> performance.
>>>> I've checked JIT assembly output, done some JMH microbenchmarks, and
>>>> found
>>>> that old implementation of MessageFactory.create() about 30-35% faster
>>>> than
>>>> the new one. The reason - approach with switch/case can effectively
>>>> inline
>>>> message creation code, but with an array of suppliers relatively heavy
>>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code
>>>> using
>>>> an abstract class for suppliers instead of an interface (to
>>>> replace "invokeinterface" with the "invokevirtual"), but it gives back
>>>> only
>>>> 10% of method performance and in this case, code looks ugly (lambdas
>>>> can't
>>>> be used). Currently, I can't find any more ways to optimize the current
>>>> approach (except return to the switch/case block). Andrey Gura, as the
>>>> author of IGNITE-12568, maybe you have some ideas about optimization?
>>>>
>>>> Perhaps we should revert IGNITE-12568, but there are some metrics
>>>> already
>>>> created, which can't be rewritten using old message factory
>>>> implementation
>>>> (IGNITE-12756). Guys, WDYT?
>>>>
>>>
>>> Alexey,
>>>
>>> I see that IGNITE-12756 (metrics improvements) is already released in
>>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
>>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
>>> 2.9 that depend on the new message factory to unblock the release and deal
>>> with the optimizations in 2.10?
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Alexey, sorry, I wrongly interpreted our benchmark results. Actually, we
were looking for a drop using bi-sect in the range between e6a7f93 (first
commit in the 2.9 branch after 2.8 branch cut) and 6592dfa5 (last commit in
the 2.9 branch). And we found these two problematic commits.
Perhaps only IGNITE-13060 (Tracing) is responsible for a drop between 2.8.1
and 2.9 (we have benchmarked 2.8.1 vs 2.9 with reverted IGNITE-13060 now
and performance looks the same)
Ticket IGNITE-12568 (MessageFactory refactoring) is not related to drop
between 2.8.1 and 2.9, but still has some performance problem, and we can
win back IGNITE-13060 drop by this ticket.
Do we need more investigation on IGNITE-13060 or we leave it as is?
What should we do with IGNITE-12568 (MessageFactory refactoring)?
пн, 31 авг. 2020 г. в 13:25, Alexey Goncharuk <al...@gmail.com>:
> Alexey,
>
> While investigating, I found that IGNITE-12568 has an incorrect fix
> version and is actually present in ignite-2.8.1 branch [1], so it cannot be
> the source of the drop against 2.8.1.
>
> P.S. Looks like we need to enforce a more accurate work with fix versions
> or develop some sort of tooling to verify the fix versions.
>
> --AG
>
> [1]
> https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
>
> пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <alexey.goncharuk@gmail.com
> >:
>
>> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
>>
>>> Guys,
>>>
>>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
>>> it
>>> locally) and got the same performance as on 2.8.1
>>>
>>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
>>> hot paths, it's clear why we have performance drop here.
>>>
>>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>>> refactored to an array of message suppliers. The message factory is on
>>> the
>>> hot path, which explains why this commit has an impact on total
>>> performance.
>>> I've checked JIT assembly output, done some JMH microbenchmarks, and
>>> found
>>> that old implementation of MessageFactory.create() about 30-35% faster
>>> than
>>> the new one. The reason - approach with switch/case can effectively
>>> inline
>>> message creation code, but with an array of suppliers relatively heavy
>>> "invokeinterface" cannot be skipped. I've tried to rewrite the code using
>>> an abstract class for suppliers instead of an interface (to
>>> replace "invokeinterface" with the "invokevirtual"), but it gives back
>>> only
>>> 10% of method performance and in this case, code looks ugly (lambdas
>>> can't
>>> be used). Currently, I can't find any more ways to optimize the current
>>> approach (except return to the switch/case block). Andrey Gura, as the
>>> author of IGNITE-12568, maybe you have some ideas about optimization?
>>>
>>> Perhaps we should revert IGNITE-12568, but there are some metrics already
>>> created, which can't be rewritten using old message factory
>>> implementation
>>> (IGNITE-12756). Guys, WDYT?
>>>
>>
>> Alexey,
>>
>> I see that IGNITE-12756 (metrics improvements) is already released in
>> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
>> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
>> 2.9 that depend on the new message factory to unblock the release and deal
>> with the optimizations in 2.10?
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
Alexey,
While investigating, I found that IGNITE-12568 has an incorrect fix version
and is actually present in ignite-2.8.1 branch [1], so it cannot be the
source of the drop against 2.8.1.
P.S. Looks like we need to enforce a more accurate work with fix versions
or develop some sort of tooling to verify the fix versions.
--AG
[1]
https://github.com/apache/ignite/commit/3e492bd23851856bbd0385c6a419892d0bba2a34
пн, 31 авг. 2020 г. в 12:42, Alexey Goncharuk <al...@gmail.com>:
> пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
>
>> Guys,
>>
>> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted it
>> locally) and got the same performance as on 2.8.1
>>
>> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
>> hot paths, it's clear why we have performance drop here.
>>
>> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
>> refactored to an array of message suppliers. The message factory is on the
>> hot path, which explains why this commit has an impact on total
>> performance.
>> I've checked JIT assembly output, done some JMH microbenchmarks, and found
>> that old implementation of MessageFactory.create() about 30-35% faster
>> than
>> the new one. The reason - approach with switch/case can effectively inline
>> message creation code, but with an array of suppliers relatively heavy
>> "invokeinterface" cannot be skipped. I've tried to rewrite the code using
>> an abstract class for suppliers instead of an interface (to
>> replace "invokeinterface" with the "invokevirtual"), but it gives back
>> only
>> 10% of method performance and in this case, code looks ugly (lambdas can't
>> be used). Currently, I can't find any more ways to optimize the current
>> approach (except return to the switch/case block). Andrey Gura, as the
>> author of IGNITE-12568, maybe you have some ideas about optimization?
>>
>> Perhaps we should revert IGNITE-12568, but there are some metrics already
>> created, which can't be rewritten using old message factory implementation
>> (IGNITE-12756). Guys, WDYT?
>>
>
> Alexey,
>
> I see that IGNITE-12756 (metrics improvements) is already released in
> Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
> 2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
> 2.9 that depend on the new message factory to unblock the release and deal
> with the optimizations in 2.10?
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Goncharuk <al...@gmail.com>.
пт, 28 авг. 2020 г. в 11:16, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted it
> locally) and got the same performance as on 2.8.1
>
> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
> hot paths, it's clear why we have performance drop here.
>
> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> refactored to an array of message suppliers. The message factory is on the
> hot path, which explains why this commit has an impact on total
> performance.
> I've checked JIT assembly output, done some JMH microbenchmarks, and found
> that old implementation of MessageFactory.create() about 30-35% faster than
> the new one. The reason - approach with switch/case can effectively inline
> message creation code, but with an array of suppliers relatively heavy
> "invokeinterface" cannot be skipped. I've tried to rewrite the code using
> an abstract class for suppliers instead of an interface (to
> replace "invokeinterface" with the "invokevirtual"), but it gives back only
> 10% of method performance and in this case, code looks ugly (lambdas can't
> be used). Currently, I can't find any more ways to optimize the current
> approach (except return to the switch/case block). Andrey Gura, as the
> author of IGNITE-12568, maybe you have some ideas about optimization?
>
> Perhaps we should revert IGNITE-12568, but there are some metrics already
> created, which can't be rewritten using old message factory implementation
> (IGNITE-12756). Guys, WDYT?
>
Alexey,
I see that IGNITE-12756 (metrics improvements) is already released in
Ignite 2.8.1 while IGNITE-12568 (message factory) is only present in Ignite
2.9. Let's revert both IGNITE-12568 and whichever new metrics created for
2.9 that depend on the new message factory to unblock the release and deal
with the optimizations in 2.10?
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Daschinsky <iv...@gmail.com>.
Artem, in ignite 2.9 a way to build C++ for linux/mac os x was changed
(autotools to cmake). As an author of this change, I want to contribute in
documentation.
As far as I understand, now it should be done through PR to specific
repository. Could you please help me with this?
пт, 28 авг. 2020 г. в 16:33, Anton Kalashnikov <ka...@yandex.ru>:
> Hi Guys,
>
> As I understand we will be merging some tickets to release. May I suggest
> also add ticket [1] to 2.9 release.
>
> There are not a lot of changes in code but It's a critical fix for the
> ability to launch ignite in lamba on Azure(There are not any workaround).
>
> So if nobody minds let's merge it to 2.9.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13013
>
> --
> Best regards,
> Anton Kalashnikov
>
>
>
> 28.08.2020, 11:16, "Alex Plehanov" <pl...@gmail.com>:
> > Guys,
> >
> > We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted
> it
> > locally) and got the same performance as on 2.8.1
> >
> > IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
> > hot paths, it's clear why we have performance drop here.
> >
> > IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> > refactored to an array of message suppliers. The message factory is on
> the
> > hot path, which explains why this commit has an impact on total
> > performance.
> > I've checked JIT assembly output, done some JMH microbenchmarks, and
> found
> > that old implementation of MessageFactory.create() about 30-35% faster
> than
> > the new one. The reason - approach with switch/case can effectively
> inline
> > message creation code, but with an array of suppliers relatively heavy
> > "invokeinterface" cannot be skipped. I've tried to rewrite the code using
> > an abstract class for suppliers instead of an interface (to
> > replace "invokeinterface" with the "invokevirtual"), but it gives back
> only
> > 10% of method performance and in this case, code looks ugly (lambdas
> can't
> > be used). Currently, I can't find any more ways to optimize the current
> > approach (except return to the switch/case block). Andrey Gura, as the
> > author of IGNITE-12568, maybe you have some ideas about optimization?
> >
> > Perhaps we should revert IGNITE-12568, but there are some metrics already
> > created, which can't be rewritten using old message factory
> implementation
> > (IGNITE-12756). Guys, WDYT?
> >
> > пт, 28 авг. 2020 г. в 01:52, Denis Magda <dm...@apache.org>:
> >
> >> Looks beautiful and easy to use, thanks, Artem! Could you please add
> the
> >> following copyright to the footer of the pages?
> >>
> >> *© 2020 The Apache Software Foundation.*
> >> *Apache, Apache Ignite, the Apache feather and the Apache Ignite logo
> are
> >> either registered trademarks or trademarks of The Apache Software
> >> Foundation. *
> >> *Privacy Policy*
> >>
> >> -
> >> Denis
> >>
> >> On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov <
> >> a.budnikov.ignite@gmail.com> wrote:
> >>
> >>> Hi everyone,
> >>>
> >>> We published the draft of Ignite 2.9 documentation on the Apache
> Ignite
> >>> web-site. The docs are available via the following link:
> >>>
> >>>
> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
> >>>
> >>> Alex,
> >>>
> >>> Is there an estimate for the release date?
> >>>
> >>> -Artem
> >>>
> >>> On 26.08.2020 17:47, Alex Plehanov wrote:
> >>> > Denis,
> >>> >
> >>> > Currently, we are running mostly IgnitePutTxImplicitBenchmark
> without
> >>> > persistence. For other benchmarks drop is lower and it's harder to
> find
> >>> > problematic commit.
> >>> >
> >>> > ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
> >>> >
> >>> >> Alex,
> >>> >>
> >>> >> Thanks for sending an update. The drop is quite big. What are the
> >>> types of
> >>> >> benchmarks you are observing the degradation for (atomic puts,
> >>> >> transactions, sql, etc.)?
> >>> >>
> >>> >> Let us know if any help by particular committers is required.
> >>> >>
> >>> >> -
> >>> >> Denis
> >>> >>
> >>> >>
> >>> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <
> >>> plehanov.alex@gmail.com>
> >>> >> wrote:
> >>> >>
> >>> >>> Hello, guys!
> >>> >>>
> >>> >>> We finally have some benchmark results. Looks like there is more
> than
> >>> one
> >>> >>> commit with a performance drop. Detected drops for those commits
> only
> >>> >>> slightly higher than measurement error, so it was hard to find
> them
> >>> and
> >>> >> we
> >>> >>> are not completely sure we found them all and found them right.
> >>> >>>
> >>> >>> Drops detected:
> >>> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
> >>> >>> implementation)
> >>> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
> >>> >> refactored
> >>> >>> in order to detect registration of message with the same direct
> type)
> >>> >>>
> >>> >>> The total drop we have on our environment - 7-8% and perhaps
> there is
> >>> >>> something else here (benchmarks still in progress, I will write
> if we
> >>> >> find
> >>> >>> more suspected commits).
> >>> >>>
> >>> >>> Ivan Artiukhov, can you please recheck mentioned above commits on
> your
> >>> >>> environment?
> >>> >>>
> >>> >>>
> >>> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
> >>> ilya.kasnacheev@gmail.com
> >>> >>> :
> >>> >>>
> >>> >>>> Hello!
> >>> >>>>
> >>> >>>> Readme.io uses blue book :)
> >>> >>>>
> >>> >>>> https://apacheignite.readme.io/docs/performance-tips
> >>> >>>>
> >>> >>>> I was thinking of something along a blue circle with `i' in it,
> for
> >>> >>>> information items.
> >>> >>>>
> >>> >>>> Regards,
> >>> >>>> --
> >>> >>>> Ilya Kasnacheev
> >>> >>>>
> >>> >>>>
> >>> >>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
> >>> >> a.budnikov.ignite@gmail.com
> >>> >>>> :
> >>> >>>>
> >>> >>>>>> Search does not seem to work.
> >>> >>>>> It uses mockups right now, but it should be ready when the docs
> are
> >>> >>>>> released.
> >>> >>>>>
> >>> >>>>>> I can see that note blocks are just annotated with "Note." Can
> we
> >>> >>> have
> >>> >>>>> some
> >>> >>>>>> image there?
> >>> >>>>> Do you have a preference as to which image you would like to see
> >>> >> there?
> >>> >>>>> -Artem
> >>> >>>>>
> >>> >>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> >>> >>>>>> Hello!
> >>> >>>>>>
> >>> >>>>>> Search does not seem to work. Are we going to have a proper
> search
> >>> >>>>> results
> >>> >>>>>> page? It is often the case that there's none.
> >>> >>>>>>
> >>> >>>>>> I can see that note blocks are just annotated with "Note." Can
> we
> >>> >>> have
> >>> >>>>> some
> >>> >>>>>> image there? Example is
> >>> >>>>>> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> >>> >>>>>>
> >>> >>>>>> Regards,
>
--
Sincerely yours, Ivan Daschinskiy
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Anton Kalashnikov <ka...@yandex.ru>.
Hi Guys,
As I understand we will be merging some tickets to release. May I suggest also add ticket [1] to 2.9 release.
There are not a lot of changes in code but It's a critical fix for the ability to launch ignite in lamba on Azure(There are not any workaround).
So if nobody minds let's merge it to 2.9.
[1] https://issues.apache.org/jira/browse/IGNITE-13013
--
Best regards,
Anton Kalashnikov
28.08.2020, 11:16, "Alex Plehanov" <pl...@gmail.com>:
> Guys,
>
> We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted it
> locally) and got the same performance as on 2.8.1
>
> IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
> hot paths, it's clear why we have performance drop here.
>
> IGNITE-12568 (MessageFactory refactoring) - switch/case block was
> refactored to an array of message suppliers. The message factory is on the
> hot path, which explains why this commit has an impact on total
> performance.
> I've checked JIT assembly output, done some JMH microbenchmarks, and found
> that old implementation of MessageFactory.create() about 30-35% faster than
> the new one. The reason - approach with switch/case can effectively inline
> message creation code, but with an array of suppliers relatively heavy
> "invokeinterface" cannot be skipped. I've tried to rewrite the code using
> an abstract class for suppliers instead of an interface (to
> replace "invokeinterface" with the "invokevirtual"), but it gives back only
> 10% of method performance and in this case, code looks ugly (lambdas can't
> be used). Currently, I can't find any more ways to optimize the current
> approach (except return to the switch/case block). Andrey Gura, as the
> author of IGNITE-12568, maybe you have some ideas about optimization?
>
> Perhaps we should revert IGNITE-12568, but there are some metrics already
> created, which can't be rewritten using old message factory implementation
> (IGNITE-12756). Guys, WDYT?
>
> пт, 28 авг. 2020 г. в 01:52, Denis Magda <dm...@apache.org>:
>
>> Looks beautiful and easy to use, thanks, Artem! Could you please add the
>> following copyright to the footer of the pages?
>>
>> *© 2020 The Apache Software Foundation.*
>> *Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are
>> either registered trademarks or trademarks of The Apache Software
>> Foundation. *
>> *Privacy Policy*
>>
>> -
>> Denis
>>
>> On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov <
>> a.budnikov.ignite@gmail.com> wrote:
>>
>>> Hi everyone,
>>>
>>> We published the draft of Ignite 2.9 documentation on the Apache Ignite
>>> web-site. The docs are available via the following link:
>>>
>>> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
>>>
>>> Alex,
>>>
>>> Is there an estimate for the release date?
>>>
>>> -Artem
>>>
>>> On 26.08.2020 17:47, Alex Plehanov wrote:
>>> > Denis,
>>> >
>>> > Currently, we are running mostly IgnitePutTxImplicitBenchmark without
>>> > persistence. For other benchmarks drop is lower and it's harder to find
>>> > problematic commit.
>>> >
>>> > ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
>>> >
>>> >> Alex,
>>> >>
>>> >> Thanks for sending an update. The drop is quite big. What are the
>>> types of
>>> >> benchmarks you are observing the degradation for (atomic puts,
>>> >> transactions, sql, etc.)?
>>> >>
>>> >> Let us know if any help by particular committers is required.
>>> >>
>>> >> -
>>> >> Denis
>>> >>
>>> >>
>>> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <
>>> plehanov.alex@gmail.com>
>>> >> wrote:
>>> >>
>>> >>> Hello, guys!
>>> >>>
>>> >>> We finally have some benchmark results. Looks like there is more than
>>> one
>>> >>> commit with a performance drop. Detected drops for those commits only
>>> >>> slightly higher than measurement error, so it was hard to find them
>>> and
>>> >> we
>>> >>> are not completely sure we found them all and found them right.
>>> >>>
>>> >>> Drops detected:
>>> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
>>> >>> implementation)
>>> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
>>> >> refactored
>>> >>> in order to detect registration of message with the same direct type)
>>> >>>
>>> >>> The total drop we have on our environment - 7-8% and perhaps there is
>>> >>> something else here (benchmarks still in progress, I will write if we
>>> >> find
>>> >>> more suspected commits).
>>> >>>
>>> >>> Ivan Artiukhov, can you please recheck mentioned above commits on your
>>> >>> environment?
>>> >>>
>>> >>>
>>> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
>>> ilya.kasnacheev@gmail.com
>>> >>> :
>>> >>>
>>> >>>> Hello!
>>> >>>>
>>> >>>> Readme.io uses blue book :)
>>> >>>>
>>> >>>> https://apacheignite.readme.io/docs/performance-tips
>>> >>>>
>>> >>>> I was thinking of something along a blue circle with `i' in it, for
>>> >>>> information items.
>>> >>>>
>>> >>>> Regards,
>>> >>>> --
>>> >>>> Ilya Kasnacheev
>>> >>>>
>>> >>>>
>>> >>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
>>> >> a.budnikov.ignite@gmail.com
>>> >>>> :
>>> >>>>
>>> >>>>>> Search does not seem to work.
>>> >>>>> It uses mockups right now, but it should be ready when the docs are
>>> >>>>> released.
>>> >>>>>
>>> >>>>>> I can see that note blocks are just annotated with "Note." Can we
>>> >>> have
>>> >>>>> some
>>> >>>>>> image there?
>>> >>>>> Do you have a preference as to which image you would like to see
>>> >> there?
>>> >>>>> -Artem
>>> >>>>>
>>> >>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
>>> >>>>>> Hello!
>>> >>>>>>
>>> >>>>>> Search does not seem to work. Are we going to have a proper search
>>> >>>>> results
>>> >>>>>> page? It is often the case that there's none.
>>> >>>>>>
>>> >>>>>> I can see that note blocks are just annotated with "Note." Can we
>>> >>> have
>>> >>>>> some
>>> >>>>>> image there? Example is
>>> >>>>>> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
>>> >>>>>>
>>> >>>>>> Regards,
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
We have benchmarked 2.9 without IGNITE-13060 and IGNITE-12568 (reverted it
locally) and got the same performance as on 2.8.1
IGNITE-13060 (Tracing) - some code was added to hot paths, to trace these
hot paths, it's clear why we have performance drop here.
IGNITE-12568 (MessageFactory refactoring) - switch/case block was
refactored to an array of message suppliers. The message factory is on the
hot path, which explains why this commit has an impact on total
performance.
I've checked JIT assembly output, done some JMH microbenchmarks, and found
that old implementation of MessageFactory.create() about 30-35% faster than
the new one. The reason - approach with switch/case can effectively inline
message creation code, but with an array of suppliers relatively heavy
"invokeinterface" cannot be skipped. I've tried to rewrite the code using
an abstract class for suppliers instead of an interface (to
replace "invokeinterface" with the "invokevirtual"), but it gives back only
10% of method performance and in this case, code looks ugly (lambdas can't
be used). Currently, I can't find any more ways to optimize the current
approach (except return to the switch/case block). Andrey Gura, as the
author of IGNITE-12568, maybe you have some ideas about optimization?
Perhaps we should revert IGNITE-12568, but there are some metrics already
created, which can't be rewritten using old message factory implementation
(IGNITE-12756). Guys, WDYT?
пт, 28 авг. 2020 г. в 01:52, Denis Magda <dm...@apache.org>:
> Looks beautiful and easy to use, thanks, Artem! Could you please add the
> following copyright to the footer of the pages?
>
> *© 2020 The Apache Software Foundation.*
> *Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are
> either registered trademarks or trademarks of The Apache Software
> Foundation. *
> *Privacy Policy*
>
> -
> Denis
>
>
> On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov <
> a.budnikov.ignite@gmail.com> wrote:
>
>> Hi everyone,
>>
>> We published the draft of Ignite 2.9 documentation on the Apache Ignite
>> web-site. The docs are available via the following link:
>>
>> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
>>
>> Alex,
>>
>> Is there an estimate for the release date?
>>
>> -Artem
>>
>> On 26.08.2020 17:47, Alex Plehanov wrote:
>> > Denis,
>> >
>> > Currently, we are running mostly IgnitePutTxImplicitBenchmark without
>> > persistence. For other benchmarks drop is lower and it's harder to find
>> > problematic commit.
>> >
>> > ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
>> >
>> >> Alex,
>> >>
>> >> Thanks for sending an update. The drop is quite big. What are the
>> types of
>> >> benchmarks you are observing the degradation for (atomic puts,
>> >> transactions, sql, etc.)?
>> >>
>> >> Let us know if any help by particular committers is required.
>> >>
>> >> -
>> >> Denis
>> >>
>> >>
>> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <
>> plehanov.alex@gmail.com>
>> >> wrote:
>> >>
>> >>> Hello, guys!
>> >>>
>> >>> We finally have some benchmark results. Looks like there is more than
>> one
>> >>> commit with a performance drop. Detected drops for those commits only
>> >>> slightly higher than measurement error, so it was hard to find them
>> and
>> >> we
>> >>> are not completely sure we found them all and found them right.
>> >>>
>> >>> Drops detected:
>> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
>> >>> implementation)
>> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
>> >> refactored
>> >>> in order to detect registration of message with the same direct type)
>> >>>
>> >>> The total drop we have on our environment - 7-8% and perhaps there is
>> >>> something else here (benchmarks still in progress, I will write if we
>> >> find
>> >>> more suspected commits).
>> >>>
>> >>> Ivan Artiukhov, can you please recheck mentioned above commits on your
>> >>> environment?
>> >>>
>> >>>
>> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
>> ilya.kasnacheev@gmail.com
>> >>> :
>> >>>
>> >>>> Hello!
>> >>>>
>> >>>> Readme.io uses blue book :)
>> >>>>
>> >>>> https://apacheignite.readme.io/docs/performance-tips
>> >>>>
>> >>>> I was thinking of something along a blue circle with `i' in it, for
>> >>>> information items.
>> >>>>
>> >>>> Regards,
>> >>>> --
>> >>>> Ilya Kasnacheev
>> >>>>
>> >>>>
>> >>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
>> >> a.budnikov.ignite@gmail.com
>> >>>> :
>> >>>>
>> >>>>>> Search does not seem to work.
>> >>>>> It uses mockups right now, but it should be ready when the docs are
>> >>>>> released.
>> >>>>>
>> >>>>>> I can see that note blocks are just annotated with "Note." Can we
>> >>> have
>> >>>>> some
>> >>>>>> image there?
>> >>>>> Do you have a preference as to which image you would like to see
>> >> there?
>> >>>>> -Artem
>> >>>>>
>> >>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
>> >>>>>> Hello!
>> >>>>>>
>> >>>>>> Search does not seem to work. Are we going to have a proper search
>> >>>>> results
>> >>>>>> page? It is often the case that there's none.
>> >>>>>>
>> >>>>>> I can see that note blocks are just annotated with "Note." Can we
>> >>> have
>> >>>>> some
>> >>>>>> image there? Example is
>> >>>>>> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
>> >>>>>>
>> >>>>>> Regards,
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Looks beautiful and easy to use, thanks, Artem! Could you please add the
following copyright to the footer of the pages?
*© 2020 The Apache Software Foundation.*
*Apache, Apache Ignite, the Apache feather and the Apache Ignite logo are
either registered trademarks or trademarks of The Apache Software
Foundation. *
*Privacy Policy*
-
Denis
On Thu, Aug 27, 2020 at 5:20 AM Artem Budnikov <a....@gmail.com>
wrote:
> Hi everyone,
>
> We published the draft of Ignite 2.9 documentation on the Apache Ignite
> web-site. The docs are available via the following link:
>
> https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
>
> Alex,
>
> Is there an estimate for the release date?
>
> -Artem
>
> On 26.08.2020 17:47, Alex Plehanov wrote:
> > Denis,
> >
> > Currently, we are running mostly IgnitePutTxImplicitBenchmark without
> > persistence. For other benchmarks drop is lower and it's harder to find
> > problematic commit.
> >
> > ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
> >
> >> Alex,
> >>
> >> Thanks for sending an update. The drop is quite big. What are the types
> of
> >> benchmarks you are observing the degradation for (atomic puts,
> >> transactions, sql, etc.)?
> >>
> >> Let us know if any help by particular committers is required.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <plehanov.alex@gmail.com
> >
> >> wrote:
> >>
> >>> Hello, guys!
> >>>
> >>> We finally have some benchmark results. Looks like there is more than
> one
> >>> commit with a performance drop. Detected drops for those commits only
> >>> slightly higher than measurement error, so it was hard to find them and
> >> we
> >>> are not completely sure we found them all and found them right.
> >>>
> >>> Drops detected:
> >>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
> >>> implementation)
> >>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
> >> refactored
> >>> in order to detect registration of message with the same direct type)
> >>>
> >>> The total drop we have on our environment - 7-8% and perhaps there is
> >>> something else here (benchmarks still in progress, I will write if we
> >> find
> >>> more suspected commits).
> >>>
> >>> Ivan Artiukhov, can you please recheck mentioned above commits on your
> >>> environment?
> >>>
> >>>
> >>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com
> >>> :
> >>>
> >>>> Hello!
> >>>>
> >>>> Readme.io uses blue book :)
> >>>>
> >>>> https://apacheignite.readme.io/docs/performance-tips
> >>>>
> >>>> I was thinking of something along a blue circle with `i' in it, for
> >>>> information items.
> >>>>
> >>>> Regards,
> >>>> --
> >>>> Ilya Kasnacheev
> >>>>
> >>>>
> >>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
> >> a.budnikov.ignite@gmail.com
> >>>> :
> >>>>
> >>>>>> Search does not seem to work.
> >>>>> It uses mockups right now, but it should be ready when the docs are
> >>>>> released.
> >>>>>
> >>>>>> I can see that note blocks are just annotated with "Note." Can we
> >>> have
> >>>>> some
> >>>>>> image there?
> >>>>> Do you have a preference as to which image you would like to see
> >> there?
> >>>>> -Artem
> >>>>>
> >>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> >>>>>> Hello!
> >>>>>>
> >>>>>> Search does not seem to work. Are we going to have a proper search
> >>>>> results
> >>>>>> page? It is often the case that there's none.
> >>>>>>
> >>>>>> I can see that note blocks are just annotated with "Note." Can we
> >>> have
> >>>>> some
> >>>>>> image there? Example is
> >>>>>> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> >>>>>>
> >>>>>> Regards,
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
Hi everyone,
We published the draft of Ignite 2.9 documentation on the Apache Ignite
web-site. The docs are available via the following link:
https://ignite.apache.org/docs/2.9.0/installation/installing-using-docker
Alex,
Is there an estimate for the release date?
-Artem
On 26.08.2020 17:47, Alex Plehanov wrote:
> Denis,
>
> Currently, we are running mostly IgnitePutTxImplicitBenchmark without
> persistence. For other benchmarks drop is lower and it's harder to find
> problematic commit.
>
> ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
>
>> Alex,
>>
>> Thanks for sending an update. The drop is quite big. What are the types of
>> benchmarks you are observing the degradation for (atomic puts,
>> transactions, sql, etc.)?
>>
>> Let us know if any help by particular committers is required.
>>
>> -
>> Denis
>>
>>
>> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <pl...@gmail.com>
>> wrote:
>>
>>> Hello, guys!
>>>
>>> We finally have some benchmark results. Looks like there is more than one
>>> commit with a performance drop. Detected drops for those commits only
>>> slightly higher than measurement error, so it was hard to find them and
>> we
>>> are not completely sure we found them all and found them right.
>>>
>>> Drops detected:
>>> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
>>> implementation)
>>> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
>> refactored
>>> in order to detect registration of message with the same direct type)
>>>
>>> The total drop we have on our environment - 7-8% and perhaps there is
>>> something else here (benchmarks still in progress, I will write if we
>> find
>>> more suspected commits).
>>>
>>> Ivan Artiukhov, can you please recheck mentioned above commits on your
>>> environment?
>>>
>>>
>>> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
>>> :
>>>
>>>> Hello!
>>>>
>>>> Readme.io uses blue book :)
>>>>
>>>> https://apacheignite.readme.io/docs/performance-tips
>>>>
>>>> I was thinking of something along a blue circle with `i' in it, for
>>>> information items.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
>> a.budnikov.ignite@gmail.com
>>>> :
>>>>
>>>>>> Search does not seem to work.
>>>>> It uses mockups right now, but it should be ready when the docs are
>>>>> released.
>>>>>
>>>>>> I can see that note blocks are just annotated with "Note." Can we
>>> have
>>>>> some
>>>>>> image there?
>>>>> Do you have a preference as to which image you would like to see
>> there?
>>>>> -Artem
>>>>>
>>>>> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
>>>>>> Hello!
>>>>>>
>>>>>> Search does not seem to work. Are we going to have a proper search
>>>>> results
>>>>>> page? It is often the case that there's none.
>>>>>>
>>>>>> I can see that note blocks are just annotated with "Note." Can we
>>> have
>>>>> some
>>>>>> image there? Example is
>>>>>> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
>>>>>>
>>>>>> Regards,
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Denis,
Currently, we are running mostly IgnitePutTxImplicitBenchmark without
persistence. For other benchmarks drop is lower and it's harder to find
problematic commit.
ср, 26 авг. 2020 г. в 17:34, Denis Magda <dm...@apache.org>:
> Alex,
>
> Thanks for sending an update. The drop is quite big. What are the types of
> benchmarks you are observing the degradation for (atomic puts,
> transactions, sql, etc.)?
>
> Let us know if any help by particular committers is required.
>
> -
> Denis
>
>
> On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
> > Hello, guys!
> >
> > We finally have some benchmark results. Looks like there is more than one
> > commit with a performance drop. Detected drops for those commits only
> > slightly higher than measurement error, so it was hard to find them and
> we
> > are not completely sure we found them all and found them right.
> >
> > Drops detected:
> > 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
> > implementation)
> > 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is
> refactored
> > in order to detect registration of message with the same direct type)
> >
> > The total drop we have on our environment - 7-8% and perhaps there is
> > something else here (benchmarks still in progress, I will write if we
> find
> > more suspected commits).
> >
> > Ivan Artiukhov, can you please recheck mentioned above commits on your
> > environment?
> >
> >
> > чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >:
> >
> > > Hello!
> > >
> > > Readme.io uses blue book :)
> > >
> > > https://apacheignite.readme.io/docs/performance-tips
> > >
> > > I was thinking of something along a blue circle with `i' in it, for
> > > information items.
> > >
> > > Regards,
> > > --
> > > Ilya Kasnacheev
> > >
> > >
> > > ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <
> a.budnikov.ignite@gmail.com
> > >:
> > >
> > > > > Search does not seem to work.
> > > > It uses mockups right now, but it should be ready when the docs are
> > > > released.
> > > >
> > > > > I can see that note blocks are just annotated with "Note." Can we
> > have
> > > > some
> > > > > image there?
> > > > Do you have a preference as to which image you would like to see
> there?
> > > >
> > > > -Artem
> > > >
> > > > On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> > > > > Hello!
> > > > >
> > > > > Search does not seem to work. Are we going to have a proper search
> > > > results
> > > > > page? It is often the case that there's none.
> > > > >
> > > > > I can see that note blocks are just annotated with "Note." Can we
> > have
> > > > some
> > > > > image there? Example is
> > > > > http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> > > > >
> > > > > Regards,
> > > >
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Alex,
Thanks for sending an update. The drop is quite big. What are the types of
benchmarks you are observing the degradation for (atomic puts,
transactions, sql, etc.)?
Let us know if any help by particular committers is required.
-
Denis
On Wed, Aug 26, 2020 at 12:26 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Hello, guys!
>
> We finally have some benchmark results. Looks like there is more than one
> commit with a performance drop. Detected drops for those commits only
> slightly higher than measurement error, so it was hard to find them and we
> are not completely sure we found them all and found them right.
>
> Drops detected:
> 2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
> implementation)
> 2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is refactored
> in order to detect registration of message with the same direct type)
>
> The total drop we have on our environment - 7-8% and perhaps there is
> something else here (benchmarks still in progress, I will write if we find
> more suspected commits).
>
> Ivan Artiukhov, can you please recheck mentioned above commits on your
> environment?
>
>
> чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <il...@gmail.com>:
>
> > Hello!
> >
> > Readme.io uses blue book :)
> >
> > https://apacheignite.readme.io/docs/performance-tips
> >
> > I was thinking of something along a blue circle with `i' in it, for
> > information items.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <a.budnikov.ignite@gmail.com
> >:
> >
> > > > Search does not seem to work.
> > > It uses mockups right now, but it should be ready when the docs are
> > > released.
> > >
> > > > I can see that note blocks are just annotated with "Note." Can we
> have
> > > some
> > > > image there?
> > > Do you have a preference as to which image you would like to see there?
> > >
> > > -Artem
> > >
> > > On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> > > > Hello!
> > > >
> > > > Search does not seem to work. Are we going to have a proper search
> > > results
> > > > page? It is often the case that there's none.
> > > >
> > > > I can see that note blocks are just annotated with "Note." Can we
> have
> > > some
> > > > image there? Example is
> > > > http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> > > >
> > > > Regards,
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Hello, guys!
We finally have some benchmark results. Looks like there is more than one
commit with a performance drop. Detected drops for those commits only
slightly higher than measurement error, so it was hard to find them and we
are not completely sure we found them all and found them right.
Drops detected:
2-3% drop on commit 99b0e0143e0 (IGNITE-13060 Tracing: initial
implementation)
2-3% drop on commit 65c30ec6947 (IGNITE-12568 MessageFactory is refactored
in order to detect registration of message with the same direct type)
The total drop we have on our environment - 7-8% and perhaps there is
something else here (benchmarks still in progress, I will write if we find
more suspected commits).
Ivan Artiukhov, can you please recheck mentioned above commits on your
environment?
чт, 20 авг. 2020 г. в 11:43, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> Readme.io uses blue book :)
>
> https://apacheignite.readme.io/docs/performance-tips
>
> I was thinking of something along a blue circle with `i' in it, for
> information items.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <a....@gmail.com>:
>
> > > Search does not seem to work.
> > It uses mockups right now, but it should be ready when the docs are
> > released.
> >
> > > I can see that note blocks are just annotated with "Note." Can we have
> > some
> > > image there?
> > Do you have a preference as to which image you would like to see there?
> >
> > -Artem
> >
> > On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> > > Hello!
> > >
> > > Search does not seem to work. Are we going to have a proper search
> > results
> > > page? It is often the case that there's none.
> > >
> > > I can see that note blocks are just annotated with "Note." Can we have
> > some
> > > image there? Example is
> > > http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> > >
> > > Regards,
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Readme.io uses blue book :)
https://apacheignite.readme.io/docs/performance-tips
I was thinking of something along a blue circle with `i' in it, for
information items.
Regards,
--
Ilya Kasnacheev
ср, 19 авг. 2020 г. в 18:29, Artem Budnikov <a....@gmail.com>:
> > Search does not seem to work.
> It uses mockups right now, but it should be ready when the docs are
> released.
>
> > I can see that note blocks are just annotated with "Note." Can we have
> some
> > image there?
> Do you have a preference as to which image you would like to see there?
>
> -Artem
>
> On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> > Hello!
> >
> > Search does not seem to work. Are we going to have a proper search
> results
> > page? It is often the case that there's none.
> >
> > I can see that note blocks are just annotated with "Note." Can we have
> some
> > image there? Example is
> > http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
> >
> > Regards,
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
> Search does not seem to work.
It uses mockups right now, but it should be ready when the docs are
released.
> I can see that note blocks are just annotated with "Note." Can we have some
> image there?
Do you have a preference as to which image you would like to see there?
-Artem
On 19.08.2020 17:37, Ilya Kasnacheev wrote:
> Hello!
>
> Search does not seem to work. Are we going to have a proper search results
> page? It is often the case that there's none.
>
> I can see that note blocks are just annotated with "Note." Can we have some
> image there? Example is
> http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
>
> Regards,
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Search does not seem to work. Are we going to have a proper search results
page? It is often the case that there's none.
I can see that note blocks are just annotated with "Note." Can we have some
image there? Example is
http://64.227.57.229/docs/2.9.0/persistence/persistence-tuning
Regards,
--
Ilya Kasnacheev
ср, 19 авг. 2020 г. в 13:48, Artem Budnikov <a....@gmail.com>:
> Hi everyone,
>
> We've set up a staging site for the Ignite 2.9 docs:
> http://64.227.57.229/docs/2.9.0/installation/installing-using-docker
>
> user/pass: ignite/apache2020
>
> More content is being added, but that's basically what it will look like.
>
> Your comments are welcome.
>
> -Artem
>
> On 18.08.2020 22:01, Pavel Tupitsyn wrote:
> > Ok, I'm just asking, there is no rush for this ticket.
> > If the release is postponed anyway, I thought that we can include more
> > fixes.
> >
> > On Tue, Aug 18, 2020 at 8:10 PM Alex Plehanov <pl...@gmail.com>
> > wrote:
> >
> >> Pavel,
> >>
> >> We still can't find the root cause of performance drop.
> >>
> >> Ticket IGNITE-13369 still in progress. When it will be resolved? Is it
> >> really critical bug? According to the user-list thread attached to the
> >> ticket, there is a workaround exists for this problem and looks like
> it's
> >> not so critical.
> >>
> >> вт, 18 авг. 2020 г. в 12:30, Pavel Tupitsyn <pt...@apache.org>:
> >>
> >>> Alex,
> >>>
> >>> What's the status of the release?
> >>> Can we include a bug fix there [1]?
> >>>
> >>> [1] https://issues.apache.org/jira/browse/IGNITE-13369
> >>>
> >>> On Fri, Aug 14, 2020 at 11:14 AM Alex Plehanov <
> plehanov.alex@gmail.com>
> >>> wrote:
> >>>
> >>>> Hello,
> >>>>
> >>>>> What is the release date for 2.9, from the cwiki it still says
> >> August 7
> >>>> We have a performance drop on the new release and still trying to find
> >>> the
> >>>> problematic commit. I hope we will find it in a week + there are one
> or
> >>> two
> >>>> weeks needed for voting and further release steps.
> >>>>
> >>>> пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <
> >>>> gaurav.agg83@gmail.com
> >>>>> :
> >>>>> What is the release date for 2.9, from the cwiki it still says August
> >>> 7 :
> >>>>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >>>>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
Hi everyone,
We've set up a staging site for the Ignite 2.9 docs:
http://64.227.57.229/docs/2.9.0/installation/installing-using-docker
user/pass: ignite/apache2020
More content is being added, but that's basically what it will look like.
Your comments are welcome.
-Artem
On 18.08.2020 22:01, Pavel Tupitsyn wrote:
> Ok, I'm just asking, there is no rush for this ticket.
> If the release is postponed anyway, I thought that we can include more
> fixes.
>
> On Tue, Aug 18, 2020 at 8:10 PM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Pavel,
>>
>> We still can't find the root cause of performance drop.
>>
>> Ticket IGNITE-13369 still in progress. When it will be resolved? Is it
>> really critical bug? According to the user-list thread attached to the
>> ticket, there is a workaround exists for this problem and looks like it's
>> not so critical.
>>
>> вт, 18 авг. 2020 г. в 12:30, Pavel Tupitsyn <pt...@apache.org>:
>>
>>> Alex,
>>>
>>> What's the status of the release?
>>> Can we include a bug fix there [1]?
>>>
>>> [1] https://issues.apache.org/jira/browse/IGNITE-13369
>>>
>>> On Fri, Aug 14, 2020 at 11:14 AM Alex Plehanov <pl...@gmail.com>
>>> wrote:
>>>
>>>> Hello,
>>>>
>>>>> What is the release date for 2.9, from the cwiki it still says
>> August 7
>>>> We have a performance drop on the new release and still trying to find
>>> the
>>>> problematic commit. I hope we will find it in a week + there are one or
>>> two
>>>> weeks needed for voting and further release steps.
>>>>
>>>> пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <
>>>> gaurav.agg83@gmail.com
>>>>> :
>>>>> What is the release date for 2.9, from the cwiki it still says August
>>> 7 :
>>>>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>>>>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Pavel Tupitsyn <pt...@apache.org>.
Ok, I'm just asking, there is no rush for this ticket.
If the release is postponed anyway, I thought that we can include more
fixes.
On Tue, Aug 18, 2020 at 8:10 PM Alex Plehanov <pl...@gmail.com>
wrote:
> Pavel,
>
> We still can't find the root cause of performance drop.
>
> Ticket IGNITE-13369 still in progress. When it will be resolved? Is it
> really critical bug? According to the user-list thread attached to the
> ticket, there is a workaround exists for this problem and looks like it's
> not so critical.
>
> вт, 18 авг. 2020 г. в 12:30, Pavel Tupitsyn <pt...@apache.org>:
>
> > Alex,
> >
> > What's the status of the release?
> > Can we include a bug fix there [1]?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-13369
> >
> > On Fri, Aug 14, 2020 at 11:14 AM Alex Plehanov <pl...@gmail.com>
> > wrote:
> >
> > > Hello,
> > >
> > > > What is the release date for 2.9, from the cwiki it still says
> August 7
> > > We have a performance drop on the new release and still trying to find
> > the
> > > problematic commit. I hope we will find it in a week + there are one or
> > two
> > > weeks needed for voting and further release steps.
> > >
> > > пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <
> > > gaurav.agg83@gmail.com
> > > >:
> > >
> > > > What is the release date for 2.9, from the cwiki it still says August
> > 7 :
> > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> > > >
> > > >
> > > >
> > > > --
> > > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > > >
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Pavel,
We still can't find the root cause of performance drop.
Ticket IGNITE-13369 still in progress. When it will be resolved? Is it
really critical bug? According to the user-list thread attached to the
ticket, there is a workaround exists for this problem and looks like it's
not so critical.
вт, 18 авг. 2020 г. в 12:30, Pavel Tupitsyn <pt...@apache.org>:
> Alex,
>
> What's the status of the release?
> Can we include a bug fix there [1]?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13369
>
> On Fri, Aug 14, 2020 at 11:14 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
> > Hello,
> >
> > > What is the release date for 2.9, from the cwiki it still says August 7
> > We have a performance drop on the new release and still trying to find
> the
> > problematic commit. I hope we will find it in a week + there are one or
> two
> > weeks needed for voting and further release steps.
> >
> > пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <
> > gaurav.agg83@gmail.com
> > >:
> >
> > > What is the release date for 2.9, from the cwiki it still says August
> 7 :
> > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> > >
> > >
> > >
> > > --
> > > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Pavel Tupitsyn <pt...@apache.org>.
Alex,
What's the status of the release?
Can we include a bug fix there [1]?
[1] https://issues.apache.org/jira/browse/IGNITE-13369
On Fri, Aug 14, 2020 at 11:14 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Hello,
>
> > What is the release date for 2.9, from the cwiki it still says August 7
> We have a performance drop on the new release and still trying to find the
> problematic commit. I hope we will find it in a week + there are one or two
> weeks needed for voting and further release steps.
>
> пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <
> gaurav.agg83@gmail.com
> >:
>
> > What is the release date for 2.9, from the cwiki it still says August 7 :
> > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Hello,
> What is the release date for 2.9, from the cwiki it still says August 7
We have a performance drop on the new release and still trying to find the
problematic commit. I hope we will find it in a week + there are one or two
weeks needed for voting and further release steps.
пт, 14 авг. 2020 г. в 11:01, gaurav.agg83@gmail.com <gaurav.agg83@gmail.com
>:
> What is the release date for 2.9, from the cwiki it still says August 7 :
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>
>
>
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by "gaurav.agg83@gmail.com" <ga...@gmail.com>.
What is the release date for 2.9, from the cwiki it still says August 7 :
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Artem Budnikov <a....@gmail.com>.
Hi Everyone,
I looked at the documentation tasks for Ignite 2.9 [1] and noticed that
some of them were being worked on. I'll take a look at the other tickets
(unassigned or assigned to me). I hope I'll have enough time to finish
them by the time of the release.
-Artem
[1]:
https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9#ApacheIgnite2.9-Documentationtasksforimportantfeaturesimplementedin2.9
On 06.08.2020 15:05, Alex Plehanov wrote:
> Ivan,
>
> Thank you. We've got performance drop on our benchmarks too.
> We are trying to bisect changes and find problematic commit now.
>
> чт, 6 авг. 2020 г. в 14:51, Ivan Artiukhov <m....@gmail.com>:
>
>> Hello,
>>
>> I've compared performances of 2.9 and 2.8.1 using Yardstick and some
>> atomic/transactional/SQL operations. Ignite 2.9 is up to 7% slower in my
>> configuration. Please see
>> https://issues.apache.org/jira/browse/IGNITE-13337 for
>> details. Is this performance drop a blocker for 2.9 release?
>>
>> --
>> Regards,
>> Ivan Artiukhov
>>
>> чт, 6 авг. 2020 г. в 11:10, Ivan Daschinsky <iv...@gmail.com>:
>>
>>> I recently found, that control.sh is broken since 2.8.0 a little bit.
>>> Script returns always 0 code, despite the fact, that CommandHandler
>> returns
>>> code correctly.
>>>
>>> Here is the issue https://issues.apache.org/jira/browse/IGNITE-13328,
>>> patch
>>> is available.
>>>
>>> пт, 31 июл. 2020 г. в 14:09, Alex Plehanov <pl...@gmail.com>:
>>>
>>>> Ivan,
>>>>
>>>> IGNITE-13306 cherry-picked to 2.9
>>>>
>>>> пт, 31 июл. 2020 г. в 13:43, Ivan Rakov <iv...@gmail.com>:
>>>>
>>>>> Hi Alex,
>>>>>
>>>>> https://issues.apache.org/jira/browse/IGNITE-13306 is merged to
>>> master.
>>>>> Can you please cherry-pick to 2.9?
>>>>>
>>>>> On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <
>>>> ilya.kasnacheev@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hello!
>>>>>>
>>>>>> I don't think that IGNITE-13006
>>>>>> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker
>> in
>>>> any
>>>>>> way. It is a good candidate for 3.0.
>>>>>>
>>>>>> ignite-spring will work with 4.x Spring as well as 5.x and the user
>> is
>>>>>> free
>>>>>> to bump Spring version. I think bumping this dependency explicitly
>> is
>>>>>> infeasible since it may break existing code.
>>>>>>
>>>>>> Regards,
>>>>>> --
>>>>>> Ilya Kasnacheev
>>>>>>
>>>>>>
>>>>>> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <plehanov.alex@gmail.com
>>> :
>>>>>>> Guys,
>>>>>>>
>>>>>>> We are in code-freeze phase now. I've moved almost all non-blocker
>>>>>>> unresolved tickets from 2.9 to the next release. If you think that
>>>>>>> some ticket is a blocker and should be included into 2.9 release,
>>>> please
>>>>>>> write a note in this thread.
>>>>>>>
>>>>>>> There are some tickets with "blocker" priority targeted to 2.9,
>> some
>>>> of
>>>>>>> them in "open" state and still unassigned, and I'm not sure we
>> need
>>>> all
>>>>>> of
>>>>>>> these tickets in 2.9:
>>>>>>>
>>>>>>> IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version
>> 4x
>>> to
>>>>>>> spring 5.2 version or later) - Is it really a blocker for 2.9
>>> release?
>>>>>> If
>>>>>>> yes, can somebody help with resolving this ticket?
>>>>>>>
>>>>>>> IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) -
>>>> ticket
>>>>>> in
>>>>>>> "Patch available" state. There is a thread on dev-list related to
>>> this
>>>>>>> ticket ([6]), but as far as I understand we still don't have
>>> consensus
>>>>>>> about version for this patch (2.9, 2.10, 3.0).
>>>>>>>
>>>>>>> IGNITE-12489 [3] (Error during purges by expiration: Unknown page
>>>> type)
>>>>>> -
>>>>>>> perhaps issue is already resolved by some related tickets, there
>> is
>>>>>> still
>>>>>>> no reproducer, no additional details and no work in progress. I
>>>> propose
>>>>>> to
>>>>>>> move this ticket to the next release.
>>>>>>>
>>>>>>> IGNITE-12911 [4] (B+Tree Corrupted exception when using a key
>>>> extracted
>>>>>>> from a BinaryObject value object --- and SQL enabled) - ticket in
>>>> "Patch
>>>>>>> available" state, but there is no activity since May 2020. Anton
>>>>>>> Kalashnikov, Ilya Kasnacheev, do we have any updates on this
>> ticket?
>>>> Is
>>>>>> it
>>>>>>> still in progress?
>>>>>>>
>>>>>>> IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
>>>>>> metrics
>>>>>>> framework is already released in 2.8 and it's still marked with
>>>>>>> @IgniteExperemental annotation, I think this ticket is not a
>>> blocker.
>>>> I
>>>>>>> propose to change the ticket priority and move it to the next
>>> release.
>>>>>>>
>>>>>>> [1]: https://issues.apache.org/jira/browse/IGNITE-13006
>>>>>>> [2]: https://issues.apache.org/jira/browse/IGNITE-11942
>>>>>>> [3]: https://issues.apache.org/jira/browse/IGNITE-12489
>>>>>>> [4]: https://issues.apache.org/jira/browse/IGNITE-12911
>>>>>>> [5]: https://issues.apache.org/jira/browse/IGNITE-12553
>>>>>>> [6]:
>>>>>>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
>>>>>>> пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <
>> plehanov.alex@gmail.com
>>>> :
>>>>>>>> Ivan,
>>>>>>>>
>>>>>>>> Merged to 2.9.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
>>>>>>>>
>>>>>>>>> Alex,
>>>>>>>>>
>>>>>>>>> Tracing is merged to master:
>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13060
>>>>>>>>>
>>>>>>>>> Can you please port it to 2.9?
>>>>>>>>> For you convenience, there's PR versus 2.9 with conflicts
>>> resolved:
>>>>>>>>> https://github.com/apache/ignite/pull/8046/files
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Ivan Rakov
>>>>>>>>>
>>>>>>>>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
>>>>>> plehanov.alex@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Ivan,
>>>>>>>>>>
>>>>>>>>>> Looks like master is broken after IGNITE-13246 (but everything
>> is
>>>> ok
>>>>>> in
>>>>>>>>>> 2.9
>>>>>>>>>> branch)
>>>>>>>>>>
>>>>>>>>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <
>>>> plehanov.alex@gmail.com
>>>>>>> :
>>>>>>>>>>> Zhenya, Ivan,
>>>>>>>>>>>
>>>>>>>>>>> I've cherry-picked IGNITE-13229 and IGNITE-13246 to
>> ignite-2.9
>>>>>> branch.
>>>>>>>>>>> Thank you.
>>>>>>>>>>>
>>>>>>>>>>> ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <
>>>> bessonov.ip@gmail.com
>>>>>>> :
>>>>>>>>>>>> Guys,
>>>>>>>>>>>>
>>>>>>>>>>>> can you please backport
>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13246
>>>>>>>>>>>> to ignite-2.9? Me and Alexey Kuznetsov really want these new
>>>>>> events
>>>>>>>>>> in
>>>>>>>>>>>> release.
>>>>>>>>>>>>
>>>>>>>>>>>> This time I prepared PR with resolved conflicts:
>>>>>>>>>>>> https://github.com/apache/ignite/pull/8042
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you!
>>>>>>>>>>>>
>>>>>>>>>>>> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>>>>>>>>>>>> <arzamas123@mail.ru.invalid
>>>>>>>>>>>>> :
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Alex, i also suggest to merge this
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13229 too,
>>>>>> GridClient
>>>>>>>>>>>>> leakage and further TC OOM preventing.
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ivan,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> It was already in release scope as discussed in this
>>> thread.
>>>>>>>>>>>>>> вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
>>>>>> ivan.glukos@gmail.com
>>>>>>>>>>> :
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> We are still waiting for a final review of Tracing
>>>>>>>>>> functionality [1]
>>>>>>>>>>>>> until
>>>>>>>>>>>>>>> the end of tomorrow (July 15).
>>>>>>>>>>>>>>> We anticipate that it will be merged to Ignite master
>> no
>>>>>> later
>>>>>>>>>> than
>>>>>>>>>>>> July
>>>>>>>>>>>>>>> 16.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sorry for being a bit late here. Alex P., can you
>> include
>>>> [1]
>>>>>>>>>> to the
>>>>>>>>>>>>>>> release scope?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1]:
>> https://issues.apache.org/jira/browse/IGNITE-13060
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Best Regards,
>>>>>>>>>>>>>>> Ivan Rakov
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>>>>>>>>>>>>> akuznetsov@gridgain.com >
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Alex,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Can you cherry-pick to Ignite 2.9 this issue:
>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> This issue is about BASELINE events and it is very
>>> useful
>>>>>> for
>>>>>>>>>>>>> notification
>>>>>>>>>>>>>>>> external tools about changes in baseline.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thank you!
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ---
>>>>>>>>>>>>>>>> Alexey Kuznetsov
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Sincerely yours,
>>>>>>>>>>>> Ivan Bessonov
>>>>>>>>>>>>
>>>
>>> --
>>> Sincerely yours, Ivan Daschinskiy
>>>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ivan,
Thank you. We've got performance drop on our benchmarks too.
We are trying to bisect changes and find problematic commit now.
чт, 6 авг. 2020 г. в 14:51, Ivan Artiukhov <m....@gmail.com>:
> Hello,
>
> I've compared performances of 2.9 and 2.8.1 using Yardstick and some
> atomic/transactional/SQL operations. Ignite 2.9 is up to 7% slower in my
> configuration. Please see
> https://issues.apache.org/jira/browse/IGNITE-13337 for
> details. Is this performance drop a blocker for 2.9 release?
>
> --
> Regards,
> Ivan Artiukhov
>
> чт, 6 авг. 2020 г. в 11:10, Ivan Daschinsky <iv...@gmail.com>:
>
> > I recently found, that control.sh is broken since 2.8.0 a little bit.
> > Script returns always 0 code, despite the fact, that CommandHandler
> returns
> > code correctly.
> >
> > Here is the issue https://issues.apache.org/jira/browse/IGNITE-13328,
> > patch
> > is available.
> >
> > пт, 31 июл. 2020 г. в 14:09, Alex Plehanov <pl...@gmail.com>:
> >
> > > Ivan,
> > >
> > > IGNITE-13306 cherry-picked to 2.9
> > >
> > > пт, 31 июл. 2020 г. в 13:43, Ivan Rakov <iv...@gmail.com>:
> > >
> > > > Hi Alex,
> > > >
> > > > https://issues.apache.org/jira/browse/IGNITE-13306 is merged to
> > master.
> > > > Can you please cherry-pick to 2.9?
> > > >
> > > > On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <
> > > ilya.kasnacheev@gmail.com>
> > > > wrote:
> > > >
> > > >> Hello!
> > > >>
> > > >> I don't think that IGNITE-13006
> > > >> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker
> in
> > > any
> > > >> way. It is a good candidate for 3.0.
> > > >>
> > > >> ignite-spring will work with 4.x Spring as well as 5.x and the user
> is
> > > >> free
> > > >> to bump Spring version. I think bumping this dependency explicitly
> is
> > > >> infeasible since it may break existing code.
> > > >>
> > > >> Regards,
> > > >> --
> > > >> Ilya Kasnacheev
> > > >>
> > > >>
> > > >> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <plehanov.alex@gmail.com
> >:
> > > >>
> > > >> > Guys,
> > > >> >
> > > >> > We are in code-freeze phase now. I've moved almost all non-blocker
> > > >> > unresolved tickets from 2.9 to the next release. If you think that
> > > >> > some ticket is a blocker and should be included into 2.9 release,
> > > please
> > > >> > write a note in this thread.
> > > >> >
> > > >> > There are some tickets with "blocker" priority targeted to 2.9,
> some
> > > of
> > > >> > them in "open" state and still unassigned, and I'm not sure we
> need
> > > all
> > > >> of
> > > >> > these tickets in 2.9:
> > > >> >
> > > >> > IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version
> 4x
> > to
> > > >> > spring 5.2 version or later) - Is it really a blocker for 2.9
> > release?
> > > >> If
> > > >> > yes, can somebody help with resolving this ticket?
> > > >> >
> > > >> > IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) -
> > > ticket
> > > >> in
> > > >> > "Patch available" state. There is a thread on dev-list related to
> > this
> > > >> > ticket ([6]), but as far as I understand we still don't have
> > consensus
> > > >> > about version for this patch (2.9, 2.10, 3.0).
> > > >> >
> > > >> > IGNITE-12489 [3] (Error during purges by expiration: Unknown page
> > > type)
> > > >> -
> > > >> > perhaps issue is already resolved by some related tickets, there
> is
> > > >> still
> > > >> > no reproducer, no additional details and no work in progress. I
> > > propose
> > > >> to
> > > >> > move this ticket to the next release.
> > > >> >
> > > >> > IGNITE-12911 [4] (B+Tree Corrupted exception when using a key
> > > extracted
> > > >> > from a BinaryObject value object --- and SQL enabled) - ticket in
> > > "Patch
> > > >> > available" state, but there is no activity since May 2020. Anton
> > > >> > Kalashnikov, Ilya Kasnacheev, do we have any updates on this
> ticket?
> > > Is
> > > >> it
> > > >> > still in progress?
> > > >> >
> > > >> > IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
> > > >> metrics
> > > >> > framework is already released in 2.8 and it's still marked with
> > > >> > @IgniteExperemental annotation, I think this ticket is not a
> > blocker.
> > > I
> > > >> > propose to change the ticket priority and move it to the next
> > release.
> > > >> >
> > > >> >
> > > >> > [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> > > >> > [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> > > >> > [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> > > >> > [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> > > >> > [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> > > >> > [6]:
> > > >> >
> > > >>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
> > > >> >
> > > >> > пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <
> plehanov.alex@gmail.com
> > >:
> > > >> >
> > > >> >> Ivan,
> > > >> >>
> > > >> >> Merged to 2.9.
> > > >> >>
> > > >> >> Thanks
> > > >> >>
> > > >> >> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> > > >> >>
> > > >> >>> Alex,
> > > >> >>>
> > > >> >>> Tracing is merged to master:
> > > >> >>> https://issues.apache.org/jira/browse/IGNITE-13060
> > > >> >>>
> > > >> >>> Can you please port it to 2.9?
> > > >> >>> For you convenience, there's PR versus 2.9 with conflicts
> > resolved:
> > > >> >>> https://github.com/apache/ignite/pull/8046/files
> > > >> >>>
> > > >> >>> --
> > > >> >>> Best Regards,
> > > >> >>> Ivan Rakov
> > > >> >>>
> > > >> >>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
> > > >> plehanov.alex@gmail.com>
> > > >> >>> wrote:
> > > >> >>>
> > > >> >>>> Ivan,
> > > >> >>>>
> > > >> >>>> Looks like master is broken after IGNITE-13246 (but everything
> is
> > > ok
> > > >> in
> > > >> >>>> 2.9
> > > >> >>>> branch)
> > > >> >>>>
> > > >> >>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <
> > > plehanov.alex@gmail.com
> > > >> >:
> > > >> >>>>
> > > >> >>>> > Zhenya, Ivan,
> > > >> >>>> >
> > > >> >>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to
> ignite-2.9
> > > >> branch.
> > > >> >>>> > Thank you.
> > > >> >>>> >
> > > >> >>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <
> > > bessonov.ip@gmail.com
> > > >> >:
> > > >> >>>> >
> > > >> >>>> >> Guys,
> > > >> >>>> >>
> > > >> >>>> >> can you please backport
> > > >> >>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> > > >> >>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
> > > >> events
> > > >> >>>> in
> > > >> >>>> >> release.
> > > >> >>>> >>
> > > >> >>>> >> This time I prepared PR with resolved conflicts:
> > > >> >>>> >> https://github.com/apache/ignite/pull/8042
> > > >> >>>> >>
> > > >> >>>> >> Thank you!
> > > >> >>>> >>
> > > >> >>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> > > >> >>>> >> <arzamas123@mail.ru.invalid
> > > >> >>>> >> >:
> > > >> >>>> >>
> > > >> >>>> >> >
> > > >> >>>> >> >
> > > >> >>>> >> >
> > > >> >>>> >> > Alex, i also suggest to merge this
> > > >> >>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> > > >> GridClient
> > > >> >>>> >> > leakage and further TC OOM preventing.
> > > >> >>>> >> >
> > > >> >>>> >> > >Ivan,
> > > >> >>>> >> > >
> > > >> >>>> >> > >It was already in release scope as discussed in this
> > thread.
> > > >> >>>> >> > >
> > > >> >>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
> > > >> ivan.glukos@gmail.com
> > > >> >>>> >:
> > > >> >>>> >> > >
> > > >> >>>> >> > >> Hi,
> > > >> >>>> >> > >>
> > > >> >>>> >> > >> We are still waiting for a final review of Tracing
> > > >> >>>> functionality [1]
> > > >> >>>> >> > until
> > > >> >>>> >> > >> the end of tomorrow (July 15).
> > > >> >>>> >> > >> We anticipate that it will be merged to Ignite master
> no
> > > >> later
> > > >> >>>> than
> > > >> >>>> >> July
> > > >> >>>> >> > >> 16.
> > > >> >>>> >> > >>
> > > >> >>>> >> > >> Sorry for being a bit late here. Alex P., can you
> include
> > > [1]
> > > >> >>>> to the
> > > >> >>>> >> > >> release scope?
> > > >> >>>> >> > >>
> > > >> >>>> >> > >> [1]:
> https://issues.apache.org/jira/browse/IGNITE-13060
> > > >> >>>> >> > >>
> > > >> >>>> >> > >> --
> > > >> >>>> >> > >> Best Regards,
> > > >> >>>> >> > >> Ivan Rakov
> > > >> >>>> >> > >>
> > > >> >>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> > > >> >>>> >> > akuznetsov@gridgain.com >
> > > >> >>>> >> > >> wrote:
> > > >> >>>> >> > >>
> > > >> >>>> >> > >>> Alex,
> > > >> >>>> >> > >>>
> > > >> >>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> > > >> >>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> > > >> >>>> >> > >>>
> > > >> >>>> >> > >>> This issue is about BASELINE events and it is very
> > useful
> > > >> for
> > > >> >>>> >> > notification
> > > >> >>>> >> > >>> external tools about changes in baseline.
> > > >> >>>> >> > >>>
> > > >> >>>> >> > >>> Thank you!
> > > >> >>>> >> > >>>
> > > >> >>>> >> > >>> ---
> > > >> >>>> >> > >>> Alexey Kuznetsov
> > > >> >>>> >> > >>>
> > > >> >>>> >> > >>
> > > >> >>>> >> >
> > > >> >>>> >> >
> > > >> >>>> >> >
> > > >> >>>> >> >
> > > >> >>>> >>
> > > >> >>>> >>
> > > >> >>>> >>
> > > >> >>>> >> --
> > > >> >>>> >> Sincerely yours,
> > > >> >>>> >> Ivan Bessonov
> > > >> >>>> >>
> > > >> >>>> >
> > > >> >>>>
> > > >> >>>
> > > >>
> > > >
> > >
> >
> >
> > --
> > Sincerely yours, Ivan Daschinskiy
> >
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Artiukhov <m....@gmail.com>.
Hello,
I've compared performances of 2.9 and 2.8.1 using Yardstick and some
atomic/transactional/SQL operations. Ignite 2.9 is up to 7% slower in my
configuration. Please see
https://issues.apache.org/jira/browse/IGNITE-13337 for
details. Is this performance drop a blocker for 2.9 release?
--
Regards,
Ivan Artiukhov
чт, 6 авг. 2020 г. в 11:10, Ivan Daschinsky <iv...@gmail.com>:
> I recently found, that control.sh is broken since 2.8.0 a little bit.
> Script returns always 0 code, despite the fact, that CommandHandler returns
> code correctly.
>
> Here is the issue https://issues.apache.org/jira/browse/IGNITE-13328,
> patch
> is available.
>
> пт, 31 июл. 2020 г. в 14:09, Alex Plehanov <pl...@gmail.com>:
>
> > Ivan,
> >
> > IGNITE-13306 cherry-picked to 2.9
> >
> > пт, 31 июл. 2020 г. в 13:43, Ivan Rakov <iv...@gmail.com>:
> >
> > > Hi Alex,
> > >
> > > https://issues.apache.org/jira/browse/IGNITE-13306 is merged to
> master.
> > > Can you please cherry-pick to 2.9?
> > >
> > > On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <
> > ilya.kasnacheev@gmail.com>
> > > wrote:
> > >
> > >> Hello!
> > >>
> > >> I don't think that IGNITE-13006
> > >> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker in
> > any
> > >> way. It is a good candidate for 3.0.
> > >>
> > >> ignite-spring will work with 4.x Spring as well as 5.x and the user is
> > >> free
> > >> to bump Spring version. I think bumping this dependency explicitly is
> > >> infeasible since it may break existing code.
> > >>
> > >> Regards,
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >>
> > >> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <pl...@gmail.com>:
> > >>
> > >> > Guys,
> > >> >
> > >> > We are in code-freeze phase now. I've moved almost all non-blocker
> > >> > unresolved tickets from 2.9 to the next release. If you think that
> > >> > some ticket is a blocker and should be included into 2.9 release,
> > please
> > >> > write a note in this thread.
> > >> >
> > >> > There are some tickets with "blocker" priority targeted to 2.9, some
> > of
> > >> > them in "open" state and still unassigned, and I'm not sure we need
> > all
> > >> of
> > >> > these tickets in 2.9:
> > >> >
> > >> > IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x
> to
> > >> > spring 5.2 version or later) - Is it really a blocker for 2.9
> release?
> > >> If
> > >> > yes, can somebody help with resolving this ticket?
> > >> >
> > >> > IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) -
> > ticket
> > >> in
> > >> > "Patch available" state. There is a thread on dev-list related to
> this
> > >> > ticket ([6]), but as far as I understand we still don't have
> consensus
> > >> > about version for this patch (2.9, 2.10, 3.0).
> > >> >
> > >> > IGNITE-12489 [3] (Error during purges by expiration: Unknown page
> > type)
> > >> -
> > >> > perhaps issue is already resolved by some related tickets, there is
> > >> still
> > >> > no reproducer, no additional details and no work in progress. I
> > propose
> > >> to
> > >> > move this ticket to the next release.
> > >> >
> > >> > IGNITE-12911 [4] (B+Tree Corrupted exception when using a key
> > extracted
> > >> > from a BinaryObject value object --- and SQL enabled) - ticket in
> > "Patch
> > >> > available" state, but there is no activity since May 2020. Anton
> > >> > Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket?
> > Is
> > >> it
> > >> > still in progress?
> > >> >
> > >> > IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
> > >> metrics
> > >> > framework is already released in 2.8 and it's still marked with
> > >> > @IgniteExperemental annotation, I think this ticket is not a
> blocker.
> > I
> > >> > propose to change the ticket priority and move it to the next
> release.
> > >> >
> > >> >
> > >> > [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> > >> > [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> > >> > [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> > >> > [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> > >> > [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> > >> > [6]:
> > >> >
> > >>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
> > >> >
> > >> > пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <plehanov.alex@gmail.com
> >:
> > >> >
> > >> >> Ivan,
> > >> >>
> > >> >> Merged to 2.9.
> > >> >>
> > >> >> Thanks
> > >> >>
> > >> >> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> > >> >>
> > >> >>> Alex,
> > >> >>>
> > >> >>> Tracing is merged to master:
> > >> >>> https://issues.apache.org/jira/browse/IGNITE-13060
> > >> >>>
> > >> >>> Can you please port it to 2.9?
> > >> >>> For you convenience, there's PR versus 2.9 with conflicts
> resolved:
> > >> >>> https://github.com/apache/ignite/pull/8046/files
> > >> >>>
> > >> >>> --
> > >> >>> Best Regards,
> > >> >>> Ivan Rakov
> > >> >>>
> > >> >>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
> > >> plehanov.alex@gmail.com>
> > >> >>> wrote:
> > >> >>>
> > >> >>>> Ivan,
> > >> >>>>
> > >> >>>> Looks like master is broken after IGNITE-13246 (but everything is
> > ok
> > >> in
> > >> >>>> 2.9
> > >> >>>> branch)
> > >> >>>>
> > >> >>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <
> > plehanov.alex@gmail.com
> > >> >:
> > >> >>>>
> > >> >>>> > Zhenya, Ivan,
> > >> >>>> >
> > >> >>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
> > >> branch.
> > >> >>>> > Thank you.
> > >> >>>> >
> > >> >>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <
> > bessonov.ip@gmail.com
> > >> >:
> > >> >>>> >
> > >> >>>> >> Guys,
> > >> >>>> >>
> > >> >>>> >> can you please backport
> > >> >>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> > >> >>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
> > >> events
> > >> >>>> in
> > >> >>>> >> release.
> > >> >>>> >>
> > >> >>>> >> This time I prepared PR with resolved conflicts:
> > >> >>>> >> https://github.com/apache/ignite/pull/8042
> > >> >>>> >>
> > >> >>>> >> Thank you!
> > >> >>>> >>
> > >> >>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> > >> >>>> >> <arzamas123@mail.ru.invalid
> > >> >>>> >> >:
> > >> >>>> >>
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> > Alex, i also suggest to merge this
> > >> >>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> > >> GridClient
> > >> >>>> >> > leakage and further TC OOM preventing.
> > >> >>>> >> >
> > >> >>>> >> > >Ivan,
> > >> >>>> >> > >
> > >> >>>> >> > >It was already in release scope as discussed in this
> thread.
> > >> >>>> >> > >
> > >> >>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
> > >> ivan.glukos@gmail.com
> > >> >>>> >:
> > >> >>>> >> > >
> > >> >>>> >> > >> Hi,
> > >> >>>> >> > >>
> > >> >>>> >> > >> We are still waiting for a final review of Tracing
> > >> >>>> functionality [1]
> > >> >>>> >> > until
> > >> >>>> >> > >> the end of tomorrow (July 15).
> > >> >>>> >> > >> We anticipate that it will be merged to Ignite master no
> > >> later
> > >> >>>> than
> > >> >>>> >> July
> > >> >>>> >> > >> 16.
> > >> >>>> >> > >>
> > >> >>>> >> > >> Sorry for being a bit late here. Alex P., can you include
> > [1]
> > >> >>>> to the
> > >> >>>> >> > >> release scope?
> > >> >>>> >> > >>
> > >> >>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> > >> >>>> >> > >>
> > >> >>>> >> > >> --
> > >> >>>> >> > >> Best Regards,
> > >> >>>> >> > >> Ivan Rakov
> > >> >>>> >> > >>
> > >> >>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> > >> >>>> >> > akuznetsov@gridgain.com >
> > >> >>>> >> > >> wrote:
> > >> >>>> >> > >>
> > >> >>>> >> > >>> Alex,
> > >> >>>> >> > >>>
> > >> >>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> > >> >>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> > >> >>>> >> > >>>
> > >> >>>> >> > >>> This issue is about BASELINE events and it is very
> useful
> > >> for
> > >> >>>> >> > notification
> > >> >>>> >> > >>> external tools about changes in baseline.
> > >> >>>> >> > >>>
> > >> >>>> >> > >>> Thank you!
> > >> >>>> >> > >>>
> > >> >>>> >> > >>> ---
> > >> >>>> >> > >>> Alexey Kuznetsov
> > >> >>>> >> > >>>
> > >> >>>> >> > >>
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >> >
> > >> >>>> >>
> > >> >>>> >>
> > >> >>>> >>
> > >> >>>> >> --
> > >> >>>> >> Sincerely yours,
> > >> >>>> >> Ivan Bessonov
> > >> >>>> >>
> > >> >>>> >
> > >> >>>>
> > >> >>>
> > >>
> > >
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Daschinsky <iv...@gmail.com>.
I recently found, that control.sh is broken since 2.8.0 a little bit.
Script returns always 0 code, despite the fact, that CommandHandler returns
code correctly.
Here is the issue https://issues.apache.org/jira/browse/IGNITE-13328, patch
is available.
пт, 31 июл. 2020 г. в 14:09, Alex Plehanov <pl...@gmail.com>:
> Ivan,
>
> IGNITE-13306 cherry-picked to 2.9
>
> пт, 31 июл. 2020 г. в 13:43, Ivan Rakov <iv...@gmail.com>:
>
> > Hi Alex,
> >
> > https://issues.apache.org/jira/browse/IGNITE-13306 is merged to master.
> > Can you please cherry-pick to 2.9?
> >
> > On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <
> ilya.kasnacheev@gmail.com>
> > wrote:
> >
> >> Hello!
> >>
> >> I don't think that IGNITE-13006
> >> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker in
> any
> >> way. It is a good candidate for 3.0.
> >>
> >> ignite-spring will work with 4.x Spring as well as 5.x and the user is
> >> free
> >> to bump Spring version. I think bumping this dependency explicitly is
> >> infeasible since it may break existing code.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <pl...@gmail.com>:
> >>
> >> > Guys,
> >> >
> >> > We are in code-freeze phase now. I've moved almost all non-blocker
> >> > unresolved tickets from 2.9 to the next release. If you think that
> >> > some ticket is a blocker and should be included into 2.9 release,
> please
> >> > write a note in this thread.
> >> >
> >> > There are some tickets with "blocker" priority targeted to 2.9, some
> of
> >> > them in "open" state and still unassigned, and I'm not sure we need
> all
> >> of
> >> > these tickets in 2.9:
> >> >
> >> > IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
> >> > spring 5.2 version or later) - Is it really a blocker for 2.9 release?
> >> If
> >> > yes, can somebody help with resolving this ticket?
> >> >
> >> > IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) -
> ticket
> >> in
> >> > "Patch available" state. There is a thread on dev-list related to this
> >> > ticket ([6]), but as far as I understand we still don't have consensus
> >> > about version for this patch (2.9, 2.10, 3.0).
> >> >
> >> > IGNITE-12489 [3] (Error during purges by expiration: Unknown page
> type)
> >> -
> >> > perhaps issue is already resolved by some related tickets, there is
> >> still
> >> > no reproducer, no additional details and no work in progress. I
> propose
> >> to
> >> > move this ticket to the next release.
> >> >
> >> > IGNITE-12911 [4] (B+Tree Corrupted exception when using a key
> extracted
> >> > from a BinaryObject value object --- and SQL enabled) - ticket in
> "Patch
> >> > available" state, but there is no activity since May 2020. Anton
> >> > Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket?
> Is
> >> it
> >> > still in progress?
> >> >
> >> > IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
> >> metrics
> >> > framework is already released in 2.8 and it's still marked with
> >> > @IgniteExperemental annotation, I think this ticket is not a blocker.
> I
> >> > propose to change the ticket priority and move it to the next release.
> >> >
> >> >
> >> > [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> >> > [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> >> > [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> >> > [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> >> > [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> >> > [6]:
> >> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
> >> >
> >> > пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
> >> >
> >> >> Ivan,
> >> >>
> >> >> Merged to 2.9.
> >> >>
> >> >> Thanks
> >> >>
> >> >> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> >> >>
> >> >>> Alex,
> >> >>>
> >> >>> Tracing is merged to master:
> >> >>> https://issues.apache.org/jira/browse/IGNITE-13060
> >> >>>
> >> >>> Can you please port it to 2.9?
> >> >>> For you convenience, there's PR versus 2.9 with conflicts resolved:
> >> >>> https://github.com/apache/ignite/pull/8046/files
> >> >>>
> >> >>> --
> >> >>> Best Regards,
> >> >>> Ivan Rakov
> >> >>>
> >> >>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
> >> plehanov.alex@gmail.com>
> >> >>> wrote:
> >> >>>
> >> >>>> Ivan,
> >> >>>>
> >> >>>> Looks like master is broken after IGNITE-13246 (but everything is
> ok
> >> in
> >> >>>> 2.9
> >> >>>> branch)
> >> >>>>
> >> >>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <
> plehanov.alex@gmail.com
> >> >:
> >> >>>>
> >> >>>> > Zhenya, Ivan,
> >> >>>> >
> >> >>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
> >> branch.
> >> >>>> > Thank you.
> >> >>>> >
> >> >>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <
> bessonov.ip@gmail.com
> >> >:
> >> >>>> >
> >> >>>> >> Guys,
> >> >>>> >>
> >> >>>> >> can you please backport
> >> >>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> >> >>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
> >> events
> >> >>>> in
> >> >>>> >> release.
> >> >>>> >>
> >> >>>> >> This time I prepared PR with resolved conflicts:
> >> >>>> >> https://github.com/apache/ignite/pull/8042
> >> >>>> >>
> >> >>>> >> Thank you!
> >> >>>> >>
> >> >>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> >> >>>> >> <arzamas123@mail.ru.invalid
> >> >>>> >> >:
> >> >>>> >>
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > Alex, i also suggest to merge this
> >> >>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> >> GridClient
> >> >>>> >> > leakage and further TC OOM preventing.
> >> >>>> >> >
> >> >>>> >> > >Ivan,
> >> >>>> >> > >
> >> >>>> >> > >It was already in release scope as discussed in this thread.
> >> >>>> >> > >
> >> >>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
> >> ivan.glukos@gmail.com
> >> >>>> >:
> >> >>>> >> > >
> >> >>>> >> > >> Hi,
> >> >>>> >> > >>
> >> >>>> >> > >> We are still waiting for a final review of Tracing
> >> >>>> functionality [1]
> >> >>>> >> > until
> >> >>>> >> > >> the end of tomorrow (July 15).
> >> >>>> >> > >> We anticipate that it will be merged to Ignite master no
> >> later
> >> >>>> than
> >> >>>> >> July
> >> >>>> >> > >> 16.
> >> >>>> >> > >>
> >> >>>> >> > >> Sorry for being a bit late here. Alex P., can you include
> [1]
> >> >>>> to the
> >> >>>> >> > >> release scope?
> >> >>>> >> > >>
> >> >>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >> >>>> >> > >>
> >> >>>> >> > >> --
> >> >>>> >> > >> Best Regards,
> >> >>>> >> > >> Ivan Rakov
> >> >>>> >> > >>
> >> >>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> >> >>>> >> > akuznetsov@gridgain.com >
> >> >>>> >> > >> wrote:
> >> >>>> >> > >>
> >> >>>> >> > >>> Alex,
> >> >>>> >> > >>>
> >> >>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> >> >>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >> >>>> >> > >>>
> >> >>>> >> > >>> This issue is about BASELINE events and it is very useful
> >> for
> >> >>>> >> > notification
> >> >>>> >> > >>> external tools about changes in baseline.
> >> >>>> >> > >>>
> >> >>>> >> > >>> Thank you!
> >> >>>> >> > >>>
> >> >>>> >> > >>> ---
> >> >>>> >> > >>> Alexey Kuznetsov
> >> >>>> >> > >>>
> >> >>>> >> > >>
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >> --
> >> >>>> >> Sincerely yours,
> >> >>>> >> Ivan Bessonov
> >> >>>> >>
> >> >>>> >
> >> >>>>
> >> >>>
> >>
> >
>
--
Sincerely yours, Ivan Daschinskiy
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ivan,
IGNITE-13306 cherry-picked to 2.9
пт, 31 июл. 2020 г. в 13:43, Ivan Rakov <iv...@gmail.com>:
> Hi Alex,
>
> https://issues.apache.org/jira/browse/IGNITE-13306 is merged to master.
> Can you please cherry-pick to 2.9?
>
> On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <il...@gmail.com>
> wrote:
>
>> Hello!
>>
>> I don't think that IGNITE-13006
>> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker in any
>> way. It is a good candidate for 3.0.
>>
>> ignite-spring will work with 4.x Spring as well as 5.x and the user is
>> free
>> to bump Spring version. I think bumping this dependency explicitly is
>> infeasible since it may break existing code.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <pl...@gmail.com>:
>>
>> > Guys,
>> >
>> > We are in code-freeze phase now. I've moved almost all non-blocker
>> > unresolved tickets from 2.9 to the next release. If you think that
>> > some ticket is a blocker and should be included into 2.9 release, please
>> > write a note in this thread.
>> >
>> > There are some tickets with "blocker" priority targeted to 2.9, some of
>> > them in "open" state and still unassigned, and I'm not sure we need all
>> of
>> > these tickets in 2.9:
>> >
>> > IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
>> > spring 5.2 version or later) - Is it really a blocker for 2.9 release?
>> If
>> > yes, can somebody help with resolving this ticket?
>> >
>> > IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket
>> in
>> > "Patch available" state. There is a thread on dev-list related to this
>> > ticket ([6]), but as far as I understand we still don't have consensus
>> > about version for this patch (2.9, 2.10, 3.0).
>> >
>> > IGNITE-12489 [3] (Error during purges by expiration: Unknown page type)
>> -
>> > perhaps issue is already resolved by some related tickets, there is
>> still
>> > no reproducer, no additional details and no work in progress. I propose
>> to
>> > move this ticket to the next release.
>> >
>> > IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
>> > from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
>> > available" state, but there is no activity since May 2020. Anton
>> > Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is
>> it
>> > still in progress?
>> >
>> > IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
>> metrics
>> > framework is already released in 2.8 and it's still marked with
>> > @IgniteExperemental annotation, I think this ticket is not a blocker. I
>> > propose to change the ticket priority and move it to the next release.
>> >
>> >
>> > [1]: https://issues.apache.org/jira/browse/IGNITE-13006
>> > [2]: https://issues.apache.org/jira/browse/IGNITE-11942
>> > [3]: https://issues.apache.org/jira/browse/IGNITE-12489
>> > [4]: https://issues.apache.org/jira/browse/IGNITE-12911
>> > [5]: https://issues.apache.org/jira/browse/IGNITE-12553
>> > [6]:
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
>> >
>> > пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
>> >
>> >> Ivan,
>> >>
>> >> Merged to 2.9.
>> >>
>> >> Thanks
>> >>
>> >> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
>> >>
>> >>> Alex,
>> >>>
>> >>> Tracing is merged to master:
>> >>> https://issues.apache.org/jira/browse/IGNITE-13060
>> >>>
>> >>> Can you please port it to 2.9?
>> >>> For you convenience, there's PR versus 2.9 with conflicts resolved:
>> >>> https://github.com/apache/ignite/pull/8046/files
>> >>>
>> >>> --
>> >>> Best Regards,
>> >>> Ivan Rakov
>> >>>
>> >>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
>> plehanov.alex@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> Ivan,
>> >>>>
>> >>>> Looks like master is broken after IGNITE-13246 (but everything is ok
>> in
>> >>>> 2.9
>> >>>> branch)
>> >>>>
>> >>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <plehanov.alex@gmail.com
>> >:
>> >>>>
>> >>>> > Zhenya, Ivan,
>> >>>> >
>> >>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
>> branch.
>> >>>> > Thank you.
>> >>>> >
>> >>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <bessonov.ip@gmail.com
>> >:
>> >>>> >
>> >>>> >> Guys,
>> >>>> >>
>> >>>> >> can you please backport
>> >>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
>> >>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
>> events
>> >>>> in
>> >>>> >> release.
>> >>>> >>
>> >>>> >> This time I prepared PR with resolved conflicts:
>> >>>> >> https://github.com/apache/ignite/pull/8042
>> >>>> >>
>> >>>> >> Thank you!
>> >>>> >>
>> >>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>> >>>> >> <arzamas123@mail.ru.invalid
>> >>>> >> >:
>> >>>> >>
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >> > Alex, i also suggest to merge this
>> >>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
>> GridClient
>> >>>> >> > leakage and further TC OOM preventing.
>> >>>> >> >
>> >>>> >> > >Ivan,
>> >>>> >> > >
>> >>>> >> > >It was already in release scope as discussed in this thread.
>> >>>> >> > >
>> >>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
>> ivan.glukos@gmail.com
>> >>>> >:
>> >>>> >> > >
>> >>>> >> > >> Hi,
>> >>>> >> > >>
>> >>>> >> > >> We are still waiting for a final review of Tracing
>> >>>> functionality [1]
>> >>>> >> > until
>> >>>> >> > >> the end of tomorrow (July 15).
>> >>>> >> > >> We anticipate that it will be merged to Ignite master no
>> later
>> >>>> than
>> >>>> >> July
>> >>>> >> > >> 16.
>> >>>> >> > >>
>> >>>> >> > >> Sorry for being a bit late here. Alex P., can you include [1]
>> >>>> to the
>> >>>> >> > >> release scope?
>> >>>> >> > >>
>> >>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>> >>>> >> > >>
>> >>>> >> > >> --
>> >>>> >> > >> Best Regards,
>> >>>> >> > >> Ivan Rakov
>> >>>> >> > >>
>> >>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>> >>>> >> > akuznetsov@gridgain.com >
>> >>>> >> > >> wrote:
>> >>>> >> > >>
>> >>>> >> > >>> Alex,
>> >>>> >> > >>>
>> >>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>> >>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>> >>>> >> > >>>
>> >>>> >> > >>> This issue is about BASELINE events and it is very useful
>> for
>> >>>> >> > notification
>> >>>> >> > >>> external tools about changes in baseline.
>> >>>> >> > >>>
>> >>>> >> > >>> Thank you!
>> >>>> >> > >>>
>> >>>> >> > >>> ---
>> >>>> >> > >>> Alexey Kuznetsov
>> >>>> >> > >>>
>> >>>> >> > >>
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >> >
>> >>>> >>
>> >>>> >>
>> >>>> >>
>> >>>> >> --
>> >>>> >> Sincerely yours,
>> >>>> >> Ivan Bessonov
>> >>>> >>
>> >>>> >
>> >>>>
>> >>>
>>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Rakov <iv...@gmail.com>.
Hi Alex,
https://issues.apache.org/jira/browse/IGNITE-13306 is merged to master.
Can you please cherry-pick to 2.9?
On Thu, Jul 30, 2020 at 7:42 PM Ilya Kasnacheev <il...@gmail.com>
wrote:
> Hello!
>
> I don't think that IGNITE-13006
> <https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker in any
> way. It is a good candidate for 3.0.
>
> ignite-spring will work with 4.x Spring as well as 5.x and the user is free
> to bump Spring version. I think bumping this dependency explicitly is
> infeasible since it may break existing code.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <pl...@gmail.com>:
>
> > Guys,
> >
> > We are in code-freeze phase now. I've moved almost all non-blocker
> > unresolved tickets from 2.9 to the next release. If you think that
> > some ticket is a blocker and should be included into 2.9 release, please
> > write a note in this thread.
> >
> > There are some tickets with "blocker" priority targeted to 2.9, some of
> > them in "open" state and still unassigned, and I'm not sure we need all
> of
> > these tickets in 2.9:
> >
> > IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
> > spring 5.2 version or later) - Is it really a blocker for 2.9 release? If
> > yes, can somebody help with resolving this ticket?
> >
> > IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket
> in
> > "Patch available" state. There is a thread on dev-list related to this
> > ticket ([6]), but as far as I understand we still don't have consensus
> > about version for this patch (2.9, 2.10, 3.0).
> >
> > IGNITE-12489 [3] (Error during purges by expiration: Unknown page type) -
> > perhaps issue is already resolved by some related tickets, there is still
> > no reproducer, no additional details and no work in progress. I propose
> to
> > move this ticket to the next release.
> >
> > IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
> > from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
> > available" state, but there is no activity since May 2020. Anton
> > Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is
> it
> > still in progress?
> >
> > IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
> metrics
> > framework is already released in 2.8 and it's still marked with
> > @IgniteExperemental annotation, I think this ticket is not a blocker. I
> > propose to change the ticket priority and move it to the next release.
> >
> >
> > [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> > [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> > [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> > [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> > [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> > [6]:
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
> >
> > пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
> >
> >> Ivan,
> >>
> >> Merged to 2.9.
> >>
> >> Thanks
> >>
> >> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> >>
> >>> Alex,
> >>>
> >>> Tracing is merged to master:
> >>> https://issues.apache.org/jira/browse/IGNITE-13060
> >>>
> >>> Can you please port it to 2.9?
> >>> For you convenience, there's PR versus 2.9 with conflicts resolved:
> >>> https://github.com/apache/ignite/pull/8046/files
> >>>
> >>> --
> >>> Best Regards,
> >>> Ivan Rakov
> >>>
> >>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <plehanov.alex@gmail.com
> >
> >>> wrote:
> >>>
> >>>> Ivan,
> >>>>
> >>>> Looks like master is broken after IGNITE-13246 (but everything is ok
> in
> >>>> 2.9
> >>>> branch)
> >>>>
> >>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
> >>>>
> >>>> > Zhenya, Ivan,
> >>>> >
> >>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
> branch.
> >>>> > Thank you.
> >>>> >
> >>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
> >>>> >
> >>>> >> Guys,
> >>>> >>
> >>>> >> can you please backport
> >>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> >>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events
> >>>> in
> >>>> >> release.
> >>>> >>
> >>>> >> This time I prepared PR with resolved conflicts:
> >>>> >> https://github.com/apache/ignite/pull/8042
> >>>> >>
> >>>> >> Thank you!
> >>>> >>
> >>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> >>>> >> <arzamas123@mail.ru.invalid
> >>>> >> >:
> >>>> >>
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> > Alex, i also suggest to merge this
> >>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> GridClient
> >>>> >> > leakage and further TC OOM preventing.
> >>>> >> >
> >>>> >> > >Ivan,
> >>>> >> > >
> >>>> >> > >It was already in release scope as discussed in this thread.
> >>>> >> > >
> >>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com
> >>>> >:
> >>>> >> > >
> >>>> >> > >> Hi,
> >>>> >> > >>
> >>>> >> > >> We are still waiting for a final review of Tracing
> >>>> functionality [1]
> >>>> >> > until
> >>>> >> > >> the end of tomorrow (July 15).
> >>>> >> > >> We anticipate that it will be merged to Ignite master no later
> >>>> than
> >>>> >> July
> >>>> >> > >> 16.
> >>>> >> > >>
> >>>> >> > >> Sorry for being a bit late here. Alex P., can you include [1]
> >>>> to the
> >>>> >> > >> release scope?
> >>>> >> > >>
> >>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >>>> >> > >>
> >>>> >> > >> --
> >>>> >> > >> Best Regards,
> >>>> >> > >> Ivan Rakov
> >>>> >> > >>
> >>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> >>>> >> > akuznetsov@gridgain.com >
> >>>> >> > >> wrote:
> >>>> >> > >>
> >>>> >> > >>> Alex,
> >>>> >> > >>>
> >>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> >>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >>>> >> > >>>
> >>>> >> > >>> This issue is about BASELINE events and it is very useful for
> >>>> >> > notification
> >>>> >> > >>> external tools about changes in baseline.
> >>>> >> > >>>
> >>>> >> > >>> Thank you!
> >>>> >> > >>>
> >>>> >> > >>> ---
> >>>> >> > >>> Alexey Kuznetsov
> >>>> >> > >>>
> >>>> >> > >>
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >> >
> >>>> >>
> >>>> >>
> >>>> >>
> >>>> >> --
> >>>> >> Sincerely yours,
> >>>> >> Ivan Bessonov
> >>>> >>
> >>>> >
> >>>>
> >>>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I don't think that IGNITE-13006
<https://issues.apache.org/jira/browse/IGNITE-13006> is a blocker in any
way. It is a good candidate for 3.0.
ignite-spring will work with 4.x Spring as well as 5.x and the user is free
to bump Spring version. I think bumping this dependency explicitly is
infeasible since it may break existing code.
Regards,
--
Ilya Kasnacheev
ср, 22 июл. 2020 г. в 10:22, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> We are in code-freeze phase now. I've moved almost all non-blocker
> unresolved tickets from 2.9 to the next release. If you think that
> some ticket is a blocker and should be included into 2.9 release, please
> write a note in this thread.
>
> There are some tickets with "blocker" priority targeted to 2.9, some of
> them in "open" state and still unassigned, and I'm not sure we need all of
> these tickets in 2.9:
>
> IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
> spring 5.2 version or later) - Is it really a blocker for 2.9 release? If
> yes, can somebody help with resolving this ticket?
>
> IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket in
> "Patch available" state. There is a thread on dev-list related to this
> ticket ([6]), but as far as I understand we still don't have consensus
> about version for this patch (2.9, 2.10, 3.0).
>
> IGNITE-12489 [3] (Error during purges by expiration: Unknown page type) -
> perhaps issue is already resolved by some related tickets, there is still
> no reproducer, no additional details and no work in progress. I propose to
> move this ticket to the next release.
>
> IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
> from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
> available" state, but there is no activity since May 2020. Anton
> Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is it
> still in progress?
>
> IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new metrics
> framework is already released in 2.8 and it's still marked with
> @IgniteExperemental annotation, I think this ticket is not a blocker. I
> propose to change the ticket priority and move it to the next release.
>
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> [6]:
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
>
> пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
>
>> Ivan,
>>
>> Merged to 2.9.
>>
>> Thanks
>>
>> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
>>
>>> Alex,
>>>
>>> Tracing is merged to master:
>>> https://issues.apache.org/jira/browse/IGNITE-13060
>>>
>>> Can you please port it to 2.9?
>>> For you convenience, there's PR versus 2.9 with conflicts resolved:
>>> https://github.com/apache/ignite/pull/8046/files
>>>
>>> --
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <pl...@gmail.com>
>>> wrote:
>>>
>>>> Ivan,
>>>>
>>>> Looks like master is broken after IGNITE-13246 (but everything is ok in
>>>> 2.9
>>>> branch)
>>>>
>>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
>>>>
>>>> > Zhenya, Ivan,
>>>> >
>>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
>>>> > Thank you.
>>>> >
>>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
>>>> >
>>>> >> Guys,
>>>> >>
>>>> >> can you please backport
>>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
>>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events
>>>> in
>>>> >> release.
>>>> >>
>>>> >> This time I prepared PR with resolved conflicts:
>>>> >> https://github.com/apache/ignite/pull/8042
>>>> >>
>>>> >> Thank you!
>>>> >>
>>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>>>> >> <arzamas123@mail.ru.invalid
>>>> >> >:
>>>> >>
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > Alex, i also suggest to merge this
>>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
>>>> >> > leakage and further TC OOM preventing.
>>>> >> >
>>>> >> > >Ivan,
>>>> >> > >
>>>> >> > >It was already in release scope as discussed in this thread.
>>>> >> > >
>>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com
>>>> >:
>>>> >> > >
>>>> >> > >> Hi,
>>>> >> > >>
>>>> >> > >> We are still waiting for a final review of Tracing
>>>> functionality [1]
>>>> >> > until
>>>> >> > >> the end of tomorrow (July 15).
>>>> >> > >> We anticipate that it will be merged to Ignite master no later
>>>> than
>>>> >> July
>>>> >> > >> 16.
>>>> >> > >>
>>>> >> > >> Sorry for being a bit late here. Alex P., can you include [1]
>>>> to the
>>>> >> > >> release scope?
>>>> >> > >>
>>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>>>> >> > >>
>>>> >> > >> --
>>>> >> > >> Best Regards,
>>>> >> > >> Ivan Rakov
>>>> >> > >>
>>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>>>> >> > akuznetsov@gridgain.com >
>>>> >> > >> wrote:
>>>> >> > >>
>>>> >> > >>> Alex,
>>>> >> > >>>
>>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>>>> >> > >>>
>>>> >> > >>> This issue is about BASELINE events and it is very useful for
>>>> >> > notification
>>>> >> > >>> external tools about changes in baseline.
>>>> >> > >>>
>>>> >> > >>> Thank you!
>>>> >> > >>>
>>>> >> > >>> ---
>>>> >> > >>> Alexey Kuznetsov
>>>> >> > >>>
>>>> >> > >>
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Sincerely yours,
>>>> >> Ivan Bessonov
>>>> >>
>>>> >
>>>>
>>>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Mekhanikov <dm...@gmail.com>.
Guys,
Is there a chance to squeeze the fix for the following issue in:
https://issues.apache.org/jira/browse/IGNITE-13306
<https://issues.apache.org/jira/browse/IGNITE-13306#>?
The issue makes the CPU load metric show -1 on Java 11. This is a quite
important metric, and this bug makes it harder to configure its monitoring.
Mirza, who's currently working on this issue says that he'll be able to
finish working on it today or tomorrow.
What do you think?
Denis
пт, 24 июл. 2020 г. в 11:04, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> I've cherry-picked IGNITE-12438 (One-way client-server connections) and
> IGNITE-13038 (Web console removing) to 2.9.
>
> Since there are no objections I will move IGNITE-13006 (Spring libs upgrade
> to 5.2 version), IGNITE-12489 (Error during purges by expiration) and
> IGNITE-12553 (public Java metrics API) to the next release.
>
> I will cherry-pick ticket IGNITE-11942 (IGFS and Hadoop removing) after it
> will be reviewed and merged to master.
>
> What about IGNITE-12911 [1] (B+Tree Corrupted exception when using a key
> extracted from a BinaryObject value object --- and SQL enabled)? Anton
> Kalashnikov, Ilya Kasnacheev, can you please clarify the ticket status?
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-12911
>
> чт, 23 июл. 2020 г. в 12:08, Alexey Kuznetsov <ak...@gridgain.com>:
>
> > Alex, Denis
> >
> > Issue with moving of Web Console to separate repository is merged to
> > master.
> > See: https://issues.apache.org/jira/browse/IGNITE-13038
> >
> > Please, consider to cherry-pick to ignite-2.9
> >
> > ---
> > Alexey Kuznetsov
> >
> >
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've cherry-picked IGNITE-12438 (One-way client-server connections) and
IGNITE-13038 (Web console removing) to 2.9.
Since there are no objections I will move IGNITE-13006 (Spring libs upgrade
to 5.2 version), IGNITE-12489 (Error during purges by expiration) and
IGNITE-12553 (public Java metrics API) to the next release.
I will cherry-pick ticket IGNITE-11942 (IGFS and Hadoop removing) after it
will be reviewed and merged to master.
What about IGNITE-12911 [1] (B+Tree Corrupted exception when using a key
extracted from a BinaryObject value object --- and SQL enabled)? Anton
Kalashnikov, Ilya Kasnacheev, can you please clarify the ticket status?
[1]: https://issues.apache.org/jira/browse/IGNITE-12911
чт, 23 июл. 2020 г. в 12:08, Alexey Kuznetsov <ak...@gridgain.com>:
> Alex, Denis
>
> Issue with moving of Web Console to separate repository is merged to
> master.
> See: https://issues.apache.org/jira/browse/IGNITE-13038
>
> Please, consider to cherry-pick to ignite-2.9
>
> ---
> Alexey Kuznetsov
>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Alex, Denis
Issue with moving of Web Console to separate repository is merged to master.
See: https://issues.apache.org/jira/browse/IGNITE-13038
Please, consider to cherry-pick to ignite-2.9
---
Alexey Kuznetsov
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Bessonov <be...@gmail.com>.
Hi guys,
considering Denis's question: [1] ("inverse TCP connection establishment")
is
already in master. I think we should port it in 2.9, would be a good thing.
"Serverless functions" [2] support is code complete in a private branch,
it's safe
to say that the issue will be completed next week (I need to run all tests
and
pass the review, it'll take some time). If we're not in a hurry then it
might be worth
waiting. Are you ok with this estimation?
[1] https://issues.apache.org/jira/browse/IGNITE-12438
[2] https://issues.apache.org/jira/browse/IGNITE-13013
ср, 22 июл. 2020 г. в 18:19, Denis Magda <dm...@apache.org>:
> Sharing a correct link for the Web Console task:
> https://issues.apache.org/jira/browse/IGNITE-13038
>
> -
> Denis
>
>
> On Wed, Jul 22, 2020 at 7:59 AM Denis Magda <dm...@apache.org> wrote:
>
> > Hi Alex,
> >
> > Thanks for wrapping this up and sharing the progress.
> >
> > I've continued the discussion in the Hadoop thread. Let's take a couple
> of
> > days to solve all open questions. Personally, I don't see any reason to
> put
> > the merge off to Ignite 3.0.
> >
> > Also, I would try to deliver the following two changes in Ignite 2.9:
> >
> > - Communication SPI changes [1] and serverless functions support.
> @Ivan
> > Bessonov <ib...@gridgain.com>, the first is completed but no
> > merged. The second should be already solved too. Could you please
> shed some
> > light on this?
> > - Phasing out Web Console [3]. It's ready for the review and I believe
> > that it can be merged quickly. @Alexey Kuznetsov
> > <ak...@apache.org>, could you please share your thoughts?
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-12438
> > [2] https://issues.apache.org/jira/browse/IGNITE-13013
> > [3] https://ggsystems.atlassian.net/browse/IGN-15304
> >
> > -
> > Denis
> >
> >
> > On Wed, Jul 22, 2020 at 12:22 AM Alex Plehanov <pl...@gmail.com>
> > wrote:
> >
> >> Guys,
> >>
> >> We are in code-freeze phase now. I've moved almost all non-blocker
> >> unresolved tickets from 2.9 to the next release. If you think that
> >> some ticket is a blocker and should be included into 2.9 release, please
> >> write a note in this thread.
> >>
> >> There are some tickets with "blocker" priority targeted to 2.9, some of
> >> them in "open" state and still unassigned, and I'm not sure we need all
> of
> >> these tickets in 2.9:
> >>
> >> IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
> >> spring 5.2 version or later) - Is it really a blocker for 2.9 release?
> If
> >> yes, can somebody help with resolving this ticket?
> >>
> >> IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket
> in
> >> "Patch available" state. There is a thread on dev-list related to this
> >> ticket ([6]), but as far as I understand we still don't have consensus
> >> about version for this patch (2.9, 2.10, 3.0).
> >>
> >> IGNITE-12489 [3] (Error during purges by expiration: Unknown page type)
> -
> >> perhaps issue is already resolved by some related tickets, there is
> still
> >> no reproducer, no additional details and no work in progress. I propose
> to
> >> move this ticket to the next release.
> >>
> >> IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
> >> from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
> >> available" state, but there is no activity since May 2020. Anton
> >> Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is
> it
> >> still in progress?
> >>
> >> IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new
> metrics
> >> framework is already released in 2.8 and it's still marked with
> >> @IgniteExperemental annotation, I think this ticket is not a blocker. I
> >> propose to change the ticket priority and move it to the next release.
> >>
> >>
> >> [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> >> [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> >> [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> >> [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> >> [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> >> [6]:
> >>
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
> >>
> >> пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
> >>
> >> > Ivan,
> >> >
> >> > Merged to 2.9.
> >> >
> >> > Thanks
> >> >
> >> > пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> >> >
> >> >> Alex,
> >> >>
> >> >> Tracing is merged to master:
> >> >> https://issues.apache.org/jira/browse/IGNITE-13060
> >> >>
> >> >> Can you please port it to 2.9?
> >> >> For you convenience, there's PR versus 2.9 with conflicts resolved:
> >> >> https://github.com/apache/ignite/pull/8046/files
> >> >>
> >> >> --
> >> >> Best Regards,
> >> >> Ivan Rakov
> >> >>
> >> >> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <
> plehanov.alex@gmail.com
> >> >
> >> >> wrote:
> >> >>
> >> >>> Ivan,
> >> >>>
> >> >>> Looks like master is broken after IGNITE-13246 (but everything is ok
> >> in
> >> >>> 2.9
> >> >>> branch)
> >> >>>
> >> >>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >> >>>
> >> >>> > Zhenya, Ivan,
> >> >>> >
> >> >>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
> >> branch.
> >> >>> > Thank you.
> >> >>> >
> >> >>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <bessonov.ip@gmail.com
> >:
> >> >>> >
> >> >>> >> Guys,
> >> >>> >>
> >> >>> >> can you please backport
> >> >>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> >> >>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
> >> events in
> >> >>> >> release.
> >> >>> >>
> >> >>> >> This time I prepared PR with resolved conflicts:
> >> >>> >> https://github.com/apache/ignite/pull/8042
> >> >>> >>
> >> >>> >> Thank you!
> >> >>> >>
> >> >>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> >> >>> >> <arzamas123@mail.ru.invalid
> >> >>> >> >:
> >> >>> >>
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> > Alex, i also suggest to merge this
> >> >>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> >> GridClient
> >> >>> >> > leakage and further TC OOM preventing.
> >> >>> >> >
> >> >>> >> > >Ivan,
> >> >>> >> > >
> >> >>> >> > >It was already in release scope as discussed in this thread.
> >> >>> >> > >
> >> >>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <
> ivan.glukos@gmail.com
> >> >:
> >> >>> >> > >
> >> >>> >> > >> Hi,
> >> >>> >> > >>
> >> >>> >> > >> We are still waiting for a final review of Tracing
> >> functionality
> >> >>> [1]
> >> >>> >> > until
> >> >>> >> > >> the end of tomorrow (July 15).
> >> >>> >> > >> We anticipate that it will be merged to Ignite master no
> later
> >> >>> than
> >> >>> >> July
> >> >>> >> > >> 16.
> >> >>> >> > >>
> >> >>> >> > >> Sorry for being a bit late here. Alex P., can you include
> [1]
> >> to
> >> >>> the
> >> >>> >> > >> release scope?
> >> >>> >> > >>
> >> >>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >> >>> >> > >>
> >> >>> >> > >> --
> >> >>> >> > >> Best Regards,
> >> >>> >> > >> Ivan Rakov
> >> >>> >> > >>
> >> >>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> >> >>> >> > akuznetsov@gridgain.com >
> >> >>> >> > >> wrote:
> >> >>> >> > >>
> >> >>> >> > >>> Alex,
> >> >>> >> > >>>
> >> >>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> >> >>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >> >>> >> > >>>
> >> >>> >> > >>> This issue is about BASELINE events and it is very useful
> for
> >> >>> >> > notification
> >> >>> >> > >>> external tools about changes in baseline.
> >> >>> >> > >>>
> >> >>> >> > >>> Thank you!
> >> >>> >> > >>>
> >> >>> >> > >>> ---
> >> >>> >> > >>> Alexey Kuznetsov
> >> >>> >> > >>>
> >> >>> >> > >>
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >> >
> >> >>> >>
> >> >>> >>
> >> >>> >>
> >> >>> >> --
> >> >>> >> Sincerely yours,
> >> >>> >> Ivan Bessonov
> >> >>> >>
> >> >>> >
> >> >>>
> >> >>
> >>
> >
>
--
Sincerely yours,
Ivan Bessonov
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Sharing a correct link for the Web Console task:
https://issues.apache.org/jira/browse/IGNITE-13038
-
Denis
On Wed, Jul 22, 2020 at 7:59 AM Denis Magda <dm...@apache.org> wrote:
> Hi Alex,
>
> Thanks for wrapping this up and sharing the progress.
>
> I've continued the discussion in the Hadoop thread. Let's take a couple of
> days to solve all open questions. Personally, I don't see any reason to put
> the merge off to Ignite 3.0.
>
> Also, I would try to deliver the following two changes in Ignite 2.9:
>
> - Communication SPI changes [1] and serverless functions support. @Ivan
> Bessonov <ib...@gridgain.com>, the first is completed but no
> merged. The second should be already solved too. Could you please shed some
> light on this?
> - Phasing out Web Console [3]. It's ready for the review and I believe
> that it can be merged quickly. @Alexey Kuznetsov
> <ak...@apache.org>, could you please share your thoughts?
>
> [1] https://issues.apache.org/jira/browse/IGNITE-12438
> [2] https://issues.apache.org/jira/browse/IGNITE-13013
> [3] https://ggsystems.atlassian.net/browse/IGN-15304
>
> -
> Denis
>
>
> On Wed, Jul 22, 2020 at 12:22 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Guys,
>>
>> We are in code-freeze phase now. I've moved almost all non-blocker
>> unresolved tickets from 2.9 to the next release. If you think that
>> some ticket is a blocker and should be included into 2.9 release, please
>> write a note in this thread.
>>
>> There are some tickets with "blocker" priority targeted to 2.9, some of
>> them in "open" state and still unassigned, and I'm not sure we need all of
>> these tickets in 2.9:
>>
>> IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
>> spring 5.2 version or later) - Is it really a blocker for 2.9 release? If
>> yes, can somebody help with resolving this ticket?
>>
>> IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket in
>> "Patch available" state. There is a thread on dev-list related to this
>> ticket ([6]), but as far as I understand we still don't have consensus
>> about version for this patch (2.9, 2.10, 3.0).
>>
>> IGNITE-12489 [3] (Error during purges by expiration: Unknown page type) -
>> perhaps issue is already resolved by some related tickets, there is still
>> no reproducer, no additional details and no work in progress. I propose to
>> move this ticket to the next release.
>>
>> IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
>> from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
>> available" state, but there is no activity since May 2020. Anton
>> Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is it
>> still in progress?
>>
>> IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new metrics
>> framework is already released in 2.8 and it's still marked with
>> @IgniteExperemental annotation, I think this ticket is not a blocker. I
>> propose to change the ticket priority and move it to the next release.
>>
>>
>> [1]: https://issues.apache.org/jira/browse/IGNITE-13006
>> [2]: https://issues.apache.org/jira/browse/IGNITE-11942
>> [3]: https://issues.apache.org/jira/browse/IGNITE-12489
>> [4]: https://issues.apache.org/jira/browse/IGNITE-12911
>> [5]: https://issues.apache.org/jira/browse/IGNITE-12553
>> [6]:
>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
>>
>> пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
>>
>> > Ivan,
>> >
>> > Merged to 2.9.
>> >
>> > Thanks
>> >
>> > пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
>> >
>> >> Alex,
>> >>
>> >> Tracing is merged to master:
>> >> https://issues.apache.org/jira/browse/IGNITE-13060
>> >>
>> >> Can you please port it to 2.9?
>> >> For you convenience, there's PR versus 2.9 with conflicts resolved:
>> >> https://github.com/apache/ignite/pull/8046/files
>> >>
>> >> --
>> >> Best Regards,
>> >> Ivan Rakov
>> >>
>> >> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <plehanov.alex@gmail.com
>> >
>> >> wrote:
>> >>
>> >>> Ivan,
>> >>>
>> >>> Looks like master is broken after IGNITE-13246 (but everything is ok
>> in
>> >>> 2.9
>> >>> branch)
>> >>>
>> >>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
>> >>>
>> >>> > Zhenya, Ivan,
>> >>> >
>> >>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
>> branch.
>> >>> > Thank you.
>> >>> >
>> >>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
>> >>> >
>> >>> >> Guys,
>> >>> >>
>> >>> >> can you please backport
>> >>> >> https://issues.apache.org/jira/browse/IGNITE-13246
>> >>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new
>> events in
>> >>> >> release.
>> >>> >>
>> >>> >> This time I prepared PR with resolved conflicts:
>> >>> >> https://github.com/apache/ignite/pull/8042
>> >>> >>
>> >>> >> Thank you!
>> >>> >>
>> >>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>> >>> >> <arzamas123@mail.ru.invalid
>> >>> >> >:
>> >>> >>
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> > Alex, i also suggest to merge this
>> >>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
>> GridClient
>> >>> >> > leakage and further TC OOM preventing.
>> >>> >> >
>> >>> >> > >Ivan,
>> >>> >> > >
>> >>> >> > >It was already in release scope as discussed in this thread.
>> >>> >> > >
>> >>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com
>> >:
>> >>> >> > >
>> >>> >> > >> Hi,
>> >>> >> > >>
>> >>> >> > >> We are still waiting for a final review of Tracing
>> functionality
>> >>> [1]
>> >>> >> > until
>> >>> >> > >> the end of tomorrow (July 15).
>> >>> >> > >> We anticipate that it will be merged to Ignite master no later
>> >>> than
>> >>> >> July
>> >>> >> > >> 16.
>> >>> >> > >>
>> >>> >> > >> Sorry for being a bit late here. Alex P., can you include [1]
>> to
>> >>> the
>> >>> >> > >> release scope?
>> >>> >> > >>
>> >>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>> >>> >> > >>
>> >>> >> > >> --
>> >>> >> > >> Best Regards,
>> >>> >> > >> Ivan Rakov
>> >>> >> > >>
>> >>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>> >>> >> > akuznetsov@gridgain.com >
>> >>> >> > >> wrote:
>> >>> >> > >>
>> >>> >> > >>> Alex,
>> >>> >> > >>>
>> >>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>> >>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>> >>> >> > >>>
>> >>> >> > >>> This issue is about BASELINE events and it is very useful for
>> >>> >> > notification
>> >>> >> > >>> external tools about changes in baseline.
>> >>> >> > >>>
>> >>> >> > >>> Thank you!
>> >>> >> > >>>
>> >>> >> > >>> ---
>> >>> >> > >>> Alexey Kuznetsov
>> >>> >> > >>>
>> >>> >> > >>
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >> >
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Sincerely yours,
>> >>> >> Ivan Bessonov
>> >>> >>
>> >>> >
>> >>>
>> >>
>>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Hi Alex,
Thanks for wrapping this up and sharing the progress.
I've continued the discussion in the Hadoop thread. Let's take a couple of
days to solve all open questions. Personally, I don't see any reason to put
the merge off to Ignite 3.0.
Also, I would try to deliver the following two changes in Ignite 2.9:
- Communication SPI changes [1] and serverless functions support. @Ivan
Bessonov <ib...@gridgain.com>, the first is completed but no merged.
The second should be already solved too. Could you please shed some light
on this?
- Phasing out Web Console [3]. It's ready for the review and I believe
that it can be merged quickly. @Alexey Kuznetsov <ak...@apache.org>,
could you please share your thoughts?
[1] https://issues.apache.org/jira/browse/IGNITE-12438
[2] https://issues.apache.org/jira/browse/IGNITE-13013
[3] https://ggsystems.atlassian.net/browse/IGN-15304
-
Denis
On Wed, Jul 22, 2020 at 12:22 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Guys,
>
> We are in code-freeze phase now. I've moved almost all non-blocker
> unresolved tickets from 2.9 to the next release. If you think that
> some ticket is a blocker and should be included into 2.9 release, please
> write a note in this thread.
>
> There are some tickets with "blocker" priority targeted to 2.9, some of
> them in "open" state and still unassigned, and I'm not sure we need all of
> these tickets in 2.9:
>
> IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
> spring 5.2 version or later) - Is it really a blocker for 2.9 release? If
> yes, can somebody help with resolving this ticket?
>
> IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket in
> "Patch available" state. There is a thread on dev-list related to this
> ticket ([6]), but as far as I understand we still don't have consensus
> about version for this patch (2.9, 2.10, 3.0).
>
> IGNITE-12489 [3] (Error during purges by expiration: Unknown page type) -
> perhaps issue is already resolved by some related tickets, there is still
> no reproducer, no additional details and no work in progress. I propose to
> move this ticket to the next release.
>
> IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
> from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
> available" state, but there is no activity since May 2020. Anton
> Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is it
> still in progress?
>
> IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new metrics
> framework is already released in 2.8 and it's still marked with
> @IgniteExperemental annotation, I think this ticket is not a blocker. I
> propose to change the ticket priority and move it to the next release.
>
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-13006
> [2]: https://issues.apache.org/jira/browse/IGNITE-11942
> [3]: https://issues.apache.org/jira/browse/IGNITE-12489
> [4]: https://issues.apache.org/jira/browse/IGNITE-12911
> [5]: https://issues.apache.org/jira/browse/IGNITE-12553
> [6]:
>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
>
> пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
>
> > Ivan,
> >
> > Merged to 2.9.
> >
> > Thanks
> >
> > пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> >
> >> Alex,
> >>
> >> Tracing is merged to master:
> >> https://issues.apache.org/jira/browse/IGNITE-13060
> >>
> >> Can you please port it to 2.9?
> >> For you convenience, there's PR versus 2.9 with conflicts resolved:
> >> https://github.com/apache/ignite/pull/8046/files
> >>
> >> --
> >> Best Regards,
> >> Ivan Rakov
> >>
> >> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <pl...@gmail.com>
> >> wrote:
> >>
> >>> Ivan,
> >>>
> >>> Looks like master is broken after IGNITE-13246 (but everything is ok in
> >>> 2.9
> >>> branch)
> >>>
> >>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
> >>>
> >>> > Zhenya, Ivan,
> >>> >
> >>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9
> branch.
> >>> > Thank you.
> >>> >
> >>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
> >>> >
> >>> >> Guys,
> >>> >>
> >>> >> can you please backport
> >>> >> https://issues.apache.org/jira/browse/IGNITE-13246
> >>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events
> in
> >>> >> release.
> >>> >>
> >>> >> This time I prepared PR with resolved conflicts:
> >>> >> https://github.com/apache/ignite/pull/8042
> >>> >>
> >>> >> Thank you!
> >>> >>
> >>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> >>> >> <arzamas123@mail.ru.invalid
> >>> >> >:
> >>> >>
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> > Alex, i also suggest to merge this
> >>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too,
> GridClient
> >>> >> > leakage and further TC OOM preventing.
> >>> >> >
> >>> >> > >Ivan,
> >>> >> > >
> >>> >> > >It was already in release scope as discussed in this thread.
> >>> >> > >
> >>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com
> >:
> >>> >> > >
> >>> >> > >> Hi,
> >>> >> > >>
> >>> >> > >> We are still waiting for a final review of Tracing
> functionality
> >>> [1]
> >>> >> > until
> >>> >> > >> the end of tomorrow (July 15).
> >>> >> > >> We anticipate that it will be merged to Ignite master no later
> >>> than
> >>> >> July
> >>> >> > >> 16.
> >>> >> > >>
> >>> >> > >> Sorry for being a bit late here. Alex P., can you include [1]
> to
> >>> the
> >>> >> > >> release scope?
> >>> >> > >>
> >>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >>> >> > >>
> >>> >> > >> --
> >>> >> > >> Best Regards,
> >>> >> > >> Ivan Rakov
> >>> >> > >>
> >>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> >>> >> > akuznetsov@gridgain.com >
> >>> >> > >> wrote:
> >>> >> > >>
> >>> >> > >>> Alex,
> >>> >> > >>>
> >>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> >>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >>> >> > >>>
> >>> >> > >>> This issue is about BASELINE events and it is very useful for
> >>> >> > notification
> >>> >> > >>> external tools about changes in baseline.
> >>> >> > >>>
> >>> >> > >>> Thank you!
> >>> >> > >>>
> >>> >> > >>> ---
> >>> >> > >>> Alexey Kuznetsov
> >>> >> > >>>
> >>> >> > >>
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >> >
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Sincerely yours,
> >>> >> Ivan Bessonov
> >>> >>
> >>> >
> >>>
> >>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
We are in code-freeze phase now. I've moved almost all non-blocker
unresolved tickets from 2.9 to the next release. If you think that
some ticket is a blocker and should be included into 2.9 release, please
write a note in this thread.
There are some tickets with "blocker" priority targeted to 2.9, some of
them in "open" state and still unassigned, and I'm not sure we need all of
these tickets in 2.9:
IGNITE-13006 [1] (Apache Ignite spring libs upgrade from version 4x to
spring 5.2 version or later) - Is it really a blocker for 2.9 release? If
yes, can somebody help with resolving this ticket?
IGNITE-11942 [2] (IGFS and Hadoop Accelerator Discontinuation) - ticket in
"Patch available" state. There is a thread on dev-list related to this
ticket ([6]), but as far as I understand we still don't have consensus
about version for this patch (2.9, 2.10, 3.0).
IGNITE-12489 [3] (Error during purges by expiration: Unknown page type) -
perhaps issue is already resolved by some related tickets, there is still
no reproducer, no additional details and no work in progress. I propose to
move this ticket to the next release.
IGNITE-12911 [4] (B+Tree Corrupted exception when using a key extracted
from a BinaryObject value object --- and SQL enabled) - ticket in "Patch
available" state, but there is no activity since May 2020. Anton
Kalashnikov, Ilya Kasnacheev, do we have any updates on this ticket? Is it
still in progress?
IGNITE-12553 [5] ([IEP-35] public Java metric API) - since the new metrics
framework is already released in 2.8 and it's still marked with
@IgniteExperemental annotation, I think this ticket is not a blocker. I
propose to change the ticket priority and move it to the next release.
[1]: https://issues.apache.org/jira/browse/IGNITE-13006
[2]: https://issues.apache.org/jira/browse/IGNITE-11942
[3]: https://issues.apache.org/jira/browse/IGNITE-12489
[4]: https://issues.apache.org/jira/browse/IGNITE-12911
[5]: https://issues.apache.org/jira/browse/IGNITE-12553
[6]:
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Complete-Discontinuation-of-IGFS-and-Hadoop-Accelerator-td42282.html
пт, 17 июл. 2020 г. в 11:50, Alex Plehanov <pl...@gmail.com>:
> Ivan,
>
> Merged to 2.9.
>
> Thanks
>
> пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
>
>> Alex,
>>
>> Tracing is merged to master:
>> https://issues.apache.org/jira/browse/IGNITE-13060
>>
>> Can you please port it to 2.9?
>> For you convenience, there's PR versus 2.9 with conflicts resolved:
>> https://github.com/apache/ignite/pull/8046/files
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <pl...@gmail.com>
>> wrote:
>>
>>> Ivan,
>>>
>>> Looks like master is broken after IGNITE-13246 (but everything is ok in
>>> 2.9
>>> branch)
>>>
>>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
>>>
>>> > Zhenya, Ivan,
>>> >
>>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
>>> > Thank you.
>>> >
>>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
>>> >
>>> >> Guys,
>>> >>
>>> >> can you please backport
>>> >> https://issues.apache.org/jira/browse/IGNITE-13246
>>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
>>> >> release.
>>> >>
>>> >> This time I prepared PR with resolved conflicts:
>>> >> https://github.com/apache/ignite/pull/8042
>>> >>
>>> >> Thank you!
>>> >>
>>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>>> >> <arzamas123@mail.ru.invalid
>>> >> >:
>>> >>
>>> >> >
>>> >> >
>>> >> >
>>> >> > Alex, i also suggest to merge this
>>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
>>> >> > leakage and further TC OOM preventing.
>>> >> >
>>> >> > >Ivan,
>>> >> > >
>>> >> > >It was already in release scope as discussed in this thread.
>>> >> > >
>>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
>>> >> > >
>>> >> > >> Hi,
>>> >> > >>
>>> >> > >> We are still waiting for a final review of Tracing functionality
>>> [1]
>>> >> > until
>>> >> > >> the end of tomorrow (July 15).
>>> >> > >> We anticipate that it will be merged to Ignite master no later
>>> than
>>> >> July
>>> >> > >> 16.
>>> >> > >>
>>> >> > >> Sorry for being a bit late here. Alex P., can you include [1] to
>>> the
>>> >> > >> release scope?
>>> >> > >>
>>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>>> >> > >>
>>> >> > >> --
>>> >> > >> Best Regards,
>>> >> > >> Ivan Rakov
>>> >> > >>
>>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>>> >> > akuznetsov@gridgain.com >
>>> >> > >> wrote:
>>> >> > >>
>>> >> > >>> Alex,
>>> >> > >>>
>>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>>> >> > >>>
>>> >> > >>> This issue is about BASELINE events and it is very useful for
>>> >> > notification
>>> >> > >>> external tools about changes in baseline.
>>> >> > >>>
>>> >> > >>> Thank you!
>>> >> > >>>
>>> >> > >>> ---
>>> >> > >>> Alexey Kuznetsov
>>> >> > >>>
>>> >> > >>
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> Sincerely yours,
>>> >> Ivan Bessonov
>>> >>
>>> >
>>>
>>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ivan,
Merged to 2.9.
Thanks
пт, 17 июл. 2020 г. в 01:35, Ivan Rakov <iv...@gmail.com>:
> Alex,
>
> Tracing is merged to master:
> https://issues.apache.org/jira/browse/IGNITE-13060
>
> Can you please port it to 2.9?
> For you convenience, there's PR versus 2.9 with conflicts resolved:
> https://github.com/apache/ignite/pull/8046/files
>
> --
> Best Regards,
> Ivan Rakov
>
> On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Ivan,
>>
>> Looks like master is broken after IGNITE-13246 (but everything is ok in
>> 2.9
>> branch)
>>
>> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
>>
>> > Zhenya, Ivan,
>> >
>> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
>> > Thank you.
>> >
>> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
>> >
>> >> Guys,
>> >>
>> >> can you please backport
>> >> https://issues.apache.org/jira/browse/IGNITE-13246
>> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
>> >> release.
>> >>
>> >> This time I prepared PR with resolved conflicts:
>> >> https://github.com/apache/ignite/pull/8042
>> >>
>> >> Thank you!
>> >>
>> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>> >> <arzamas123@mail.ru.invalid
>> >> >:
>> >>
>> >> >
>> >> >
>> >> >
>> >> > Alex, i also suggest to merge this
>> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
>> >> > leakage and further TC OOM preventing.
>> >> >
>> >> > >Ivan,
>> >> > >
>> >> > >It was already in release scope as discussed in this thread.
>> >> > >
>> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
>> >> > >
>> >> > >> Hi,
>> >> > >>
>> >> > >> We are still waiting for a final review of Tracing functionality
>> [1]
>> >> > until
>> >> > >> the end of tomorrow (July 15).
>> >> > >> We anticipate that it will be merged to Ignite master no later
>> than
>> >> July
>> >> > >> 16.
>> >> > >>
>> >> > >> Sorry for being a bit late here. Alex P., can you include [1] to
>> the
>> >> > >> release scope?
>> >> > >>
>> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>> >> > >>
>> >> > >> --
>> >> > >> Best Regards,
>> >> > >> Ivan Rakov
>> >> > >>
>> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>> >> > akuznetsov@gridgain.com >
>> >> > >> wrote:
>> >> > >>
>> >> > >>> Alex,
>> >> > >>>
>> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>> >> > >>>
>> >> > >>> This issue is about BASELINE events and it is very useful for
>> >> > notification
>> >> > >>> external tools about changes in baseline.
>> >> > >>>
>> >> > >>> Thank you!
>> >> > >>>
>> >> > >>> ---
>> >> > >>> Alexey Kuznetsov
>> >> > >>>
>> >> > >>
>> >> >
>> >> >
>> >> >
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Sincerely yours,
>> >> Ivan Bessonov
>> >>
>> >
>>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Rakov <iv...@gmail.com>.
Alex,
Tracing is merged to master:
https://issues.apache.org/jira/browse/IGNITE-13060
Can you please port it to 2.9?
For you convenience, there's PR versus 2.9 with conflicts resolved:
https://github.com/apache/ignite/pull/8046/files
--
Best Regards,
Ivan Rakov
On Wed, Jul 15, 2020 at 5:33 PM Alex Plehanov <pl...@gmail.com>
wrote:
> Ivan,
>
> Looks like master is broken after IGNITE-13246 (but everything is ok in 2.9
> branch)
>
> ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
>
> > Zhenya, Ivan,
> >
> > I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
> > Thank you.
> >
> > ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
> >
> >> Guys,
> >>
> >> can you please backport
> >> https://issues.apache.org/jira/browse/IGNITE-13246
> >> to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
> >> release.
> >>
> >> This time I prepared PR with resolved conflicts:
> >> https://github.com/apache/ignite/pull/8042
> >>
> >> Thank you!
> >>
> >> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
> >> <arzamas123@mail.ru.invalid
> >> >:
> >>
> >> >
> >> >
> >> >
> >> > Alex, i also suggest to merge this
> >> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
> >> > leakage and further TC OOM preventing.
> >> >
> >> > >Ivan,
> >> > >
> >> > >It was already in release scope as discussed in this thread.
> >> > >
> >> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
> >> > >
> >> > >> Hi,
> >> > >>
> >> > >> We are still waiting for a final review of Tracing functionality
> [1]
> >> > until
> >> > >> the end of tomorrow (July 15).
> >> > >> We anticipate that it will be merged to Ignite master no later than
> >> July
> >> > >> 16.
> >> > >>
> >> > >> Sorry for being a bit late here. Alex P., can you include [1] to
> the
> >> > >> release scope?
> >> > >>
> >> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >> > >>
> >> > >> --
> >> > >> Best Regards,
> >> > >> Ivan Rakov
> >> > >>
> >> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> >> > akuznetsov@gridgain.com >
> >> > >> wrote:
> >> > >>
> >> > >>> Alex,
> >> > >>>
> >> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> >> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >> > >>>
> >> > >>> This issue is about BASELINE events and it is very useful for
> >> > notification
> >> > >>> external tools about changes in baseline.
> >> > >>>
> >> > >>> Thank you!
> >> > >>>
> >> > >>> ---
> >> > >>> Alexey Kuznetsov
> >> > >>>
> >> > >>
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >> --
> >> Sincerely yours,
> >> Ivan Bessonov
> >>
> >
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ivan,
Looks like master is broken after IGNITE-13246 (but everything is ok in 2.9
branch)
ср, 15 июл. 2020 г. в 18:54, Alex Plehanov <pl...@gmail.com>:
> Zhenya, Ivan,
>
> I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
> Thank you.
>
> ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
>
>> Guys,
>>
>> can you please backport
>> https://issues.apache.org/jira/browse/IGNITE-13246
>> to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
>> release.
>>
>> This time I prepared PR with resolved conflicts:
>> https://github.com/apache/ignite/pull/8042
>>
>> Thank you!
>>
>> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky
>> <arzamas123@mail.ru.invalid
>> >:
>>
>> >
>> >
>> >
>> > Alex, i also suggest to merge this
>> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
>> > leakage and further TC OOM preventing.
>> >
>> > >Ivan,
>> > >
>> > >It was already in release scope as discussed in this thread.
>> > >
>> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
>> > >
>> > >> Hi,
>> > >>
>> > >> We are still waiting for a final review of Tracing functionality [1]
>> > until
>> > >> the end of tomorrow (July 15).
>> > >> We anticipate that it will be merged to Ignite master no later than
>> July
>> > >> 16.
>> > >>
>> > >> Sorry for being a bit late here. Alex P., can you include [1] to the
>> > >> release scope?
>> > >>
>> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>> > >>
>> > >> --
>> > >> Best Regards,
>> > >> Ivan Rakov
>> > >>
>> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
>> > akuznetsov@gridgain.com >
>> > >> wrote:
>> > >>
>> > >>> Alex,
>> > >>>
>> > >>> Can you cherry-pick to Ignite 2.9 this issue:
>> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>> > >>>
>> > >>> This issue is about BASELINE events and it is very useful for
>> > notification
>> > >>> external tools about changes in baseline.
>> > >>>
>> > >>> Thank you!
>> > >>>
>> > >>> ---
>> > >>> Alexey Kuznetsov
>> > >>>
>> > >>
>> >
>> >
>> >
>> >
>>
>>
>>
>> --
>> Sincerely yours,
>> Ivan Bessonov
>>
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Zhenya, Ivan,
I've cherry-picked IGNITE-13229 and IGNITE-13246 to ignite-2.9 branch.
Thank you.
ср, 15 июл. 2020 г. в 18:31, Ivan Bessonov <be...@gmail.com>:
> Guys,
>
> can you please backport https://issues.apache.org/jira/browse/IGNITE-13246
> to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
> release.
>
> This time I prepared PR with resolved conflicts:
> https://github.com/apache/ignite/pull/8042
>
> Thank you!
>
> вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky <arzamas123@mail.ru.invalid
> >:
>
> >
> >
> >
> > Alex, i also suggest to merge this
> > https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
> > leakage and further TC OOM preventing.
> >
> > >Ivan,
> > >
> > >It was already in release scope as discussed in this thread.
> > >
> > >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
> > >
> > >> Hi,
> > >>
> > >> We are still waiting for a final review of Tracing functionality [1]
> > until
> > >> the end of tomorrow (July 15).
> > >> We anticipate that it will be merged to Ignite master no later than
> July
> > >> 16.
> > >>
> > >> Sorry for being a bit late here. Alex P., can you include [1] to the
> > >> release scope?
> > >>
> > >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> > >>
> > >> --
> > >> Best Regards,
> > >> Ivan Rakov
> > >>
> > >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> > akuznetsov@gridgain.com >
> > >> wrote:
> > >>
> > >>> Alex,
> > >>>
> > >>> Can you cherry-pick to Ignite 2.9 this issue:
> > >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> > >>>
> > >>> This issue is about BASELINE events and it is very useful for
> > notification
> > >>> external tools about changes in baseline.
> > >>>
> > >>> Thank you!
> > >>>
> > >>> ---
> > >>> Alexey Kuznetsov
> > >>>
> > >>
> >
> >
> >
> >
>
>
>
> --
> Sincerely yours,
> Ivan Bessonov
>
Re: Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Bessonov <be...@gmail.com>.
Guys,
can you please backport https://issues.apache.org/jira/browse/IGNITE-13246
to ignite-2.9? Me and Alexey Kuznetsov really want these new events in
release.
This time I prepared PR with resolved conflicts:
https://github.com/apache/ignite/pull/8042
Thank you!
вт, 14 июл. 2020 г. в 19:39, Zhenya Stanilovsky <arzamas123@mail.ru.invalid
>:
>
>
>
> Alex, i also suggest to merge this
> https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient
> leakage and further TC OOM preventing.
>
> >Ivan,
> >
> >It was already in release scope as discussed in this thread.
> >
> >вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
> >
> >> Hi,
> >>
> >> We are still waiting for a final review of Tracing functionality [1]
> until
> >> the end of tomorrow (July 15).
> >> We anticipate that it will be merged to Ignite master no later than July
> >> 16.
> >>
> >> Sorry for being a bit late here. Alex P., can you include [1] to the
> >> release scope?
> >>
> >> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
> >>
> >> --
> >> Best Regards,
> >> Ivan Rakov
> >>
> >> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <
> akuznetsov@gridgain.com >
> >> wrote:
> >>
> >>> Alex,
> >>>
> >>> Can you cherry-pick to Ignite 2.9 this issue:
> >>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
> >>>
> >>> This issue is about BASELINE events and it is very useful for
> notification
> >>> external tools about changes in baseline.
> >>>
> >>> Thank you!
> >>>
> >>> ---
> >>> Alexey Kuznetsov
> >>>
> >>
>
>
>
>
--
Sincerely yours,
Ivan Bessonov
Re[2]: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Zhenya Stanilovsky <ar...@mail.ru.INVALID>.
Alex, i also suggest to merge this https://issues.apache.org/jira/browse/IGNITE-13229 too, GridClient leakage and further TC OOM preventing.
>Ivan,
>
>It was already in release scope as discussed in this thread.
>
>вт, 14 июл. 2020 г. в 14:31, Ivan Rakov < ivan.glukos@gmail.com >:
>
>> Hi,
>>
>> We are still waiting for a final review of Tracing functionality [1] until
>> the end of tomorrow (July 15).
>> We anticipate that it will be merged to Ignite master no later than July
>> 16.
>>
>> Sorry for being a bit late here. Alex P., can you include [1] to the
>> release scope?
>>
>> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov < akuznetsov@gridgain.com >
>> wrote:
>>
>>> Alex,
>>>
>>> Can you cherry-pick to Ignite 2.9 this issue:
>>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>>>
>>> This issue is about BASELINE events and it is very useful for notification
>>> external tools about changes in baseline.
>>>
>>> Thank you!
>>>
>>> ---
>>> Alexey Kuznetsov
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ivan,
It was already in release scope as discussed in this thread.
вт, 14 июл. 2020 г. в 14:31, Ivan Rakov <iv...@gmail.com>:
> Hi,
>
> We are still waiting for a final review of Tracing functionality [1] until
> the end of tomorrow (July 15).
> We anticipate that it will be merged to Ignite master no later than July
> 16.
>
> Sorry for being a bit late here. Alex P., can you include [1] to the
> release scope?
>
> [1]: https://issues.apache.org/jira/browse/IGNITE-13060
>
> --
> Best Regards,
> Ivan Rakov
>
> On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <ak...@gridgain.com>
> wrote:
>
>> Alex,
>>
>> Can you cherry-pick to Ignite 2.9 this issue:
>> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>>
>> This issue is about BASELINE events and it is very useful for notification
>> external tools about changes in baseline.
>>
>> Thank you!
>>
>> ---
>> Alexey Kuznetsov
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Rakov <iv...@gmail.com>.
Hi,
We are still waiting for a final review of Tracing functionality [1] until
the end of tomorrow (July 15).
We anticipate that it will be merged to Ignite master no later than July 16.
Sorry for being a bit late here. Alex P., can you include [1] to the
release scope?
[1]: https://issues.apache.org/jira/browse/IGNITE-13060
--
Best Regards,
Ivan Rakov
On Tue, Jul 14, 2020 at 6:16 AM Alexey Kuznetsov <ak...@gridgain.com>
wrote:
> Alex,
>
> Can you cherry-pick to Ignite 2.9 this issue:
> https://issues.apache.org/jira/browse/IGNITE-13246 ?
>
> This issue is about BASELINE events and it is very useful for notification
> external tools about changes in baseline.
>
> Thank you!
>
> ---
> Alexey Kuznetsov
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Kuznetsov <ak...@gridgain.com>.
Alex,
Can you cherry-pick to Ignite 2.9 this issue:
https://issues.apache.org/jira/browse/IGNITE-13246 ?
This issue is about BASELINE events and it is very useful for notification
external tools about changes in baseline.
Thank you!
---
Alexey Kuznetsov
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ilya,
Yes, it works, thank you.
I will update POM version today.
пн, 13 июл. 2020 г. в 16:01, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> It seems that it works and we now have 2.9 nightlies.
>
> Guys, can you please bump the master POM version to 2.10.0-SNAPSHOT?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 10 июл. 2020 г. в 17:43, Ilya Kasnacheev <il...@gmail.com>:
>
> > Hello!
> >
> > I have also set up nightly builds to (hopefully) build 2.9
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > вт, 7 июл. 2020 г. в 17:04, Ilya Kasnacheev <il...@gmail.com>:
> >
> >> Hello!
> >>
> >> I have changed the schedule condition so ignite-2.9 will be run instead.
> >> Please check if it works tomorrow.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> вт, 7 июл. 2020 г. в 16:42, Alex Plehanov <pl...@gmail.com>:
> >>
> >>> Guys,
> >>>
> >>> Can somebody help with creating scheduled build triggers for nightly
> "Run
> >>> All" on TC for "ignite-2.9" branch?
> >>>
> >>> Also, I found that we still have a nightly "Run All" for "ignite-2.8"
> >>> branch [1]. Do we still need it? Or just forgot to remove it after
> >>> release?
> >>>
> >>> [1] :
> >>>
> >>>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAllNightly&branch_IgniteTests24Java8=ignite-2.8&tab=buildTypeStatusDiv
> >>>
> >>> пт, 3 июл. 2020 г. в 23:18, Alex Plehanov <pl...@gmail.com>:
> >>>
> >>> > Anton, thank you.
> >>> >
> >>> > Guys,
> >>> >
> >>> > I've cut the release branch. From now on please pin resolved in the
> >>> master
> >>> > branch tickets to 2.10 release in Jira and cherry-pick targeted to
> 2.9
> >>> > tickets to ignite-2.9 branch.
> >>> >
> >>> > As far as the release branch cut was delayed, I propose to move code
> >>> > freeze dates by one week (to 17 July 2020). WDYT?
> >>> >
> >>> > I've moved long-time inactive non-blocker tickets (except
> documentation
> >>> > related, we will deal with them later) from 2.9 release, please check
> >>> your
> >>> > Jira issues to have the correct "Fix version" field if your ticket is
> >>> > planned to 2.9 release.
> >>> >
> >>> > All currently pinned to 2.9 issues can be found on the release page
> >>> [1].
> >>> >
> >>> > I didn't get about [2] ticket mentioned in this thread, are we going
> to
> >>> > include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please
> >>> clarify?
> >>> >
> >>> > [1]:
> >>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >>> > [2]: https://issues.apache.org/jira/browse/IGNITE-13156
> >>> >
> >>> > пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
> >>> >
> >>> >> You're now at the "Ignite Release Managers" group.
> >>> >> Please check you gain access.
> >>> >>
> >>> >> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <
> >>> plehanov.alex@gmail.com>
> >>> >> wrote:
> >>> >>
> >>> >> > Guys,
> >>> >> >
> >>> >> > I've created the 2.9 release confluence page [1].
> >>> >> > Also, I found that I don't have permission to Team City release
> >>> tasks.
> >>> >> Can
> >>> >> > anyone give me such permissions?
> >>> >> >
> >>> >> > [1]:
> >>> >>
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >>> >> >
> >>> >> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <
> >>> zaleslaw.sin@gmail.com>:
> >>> >> >
> >>> >> > > Igniters
> >>> >> > >
> >>> >> > > Unfortunately, the ML model export/import feature
> >>> >> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
> >>> >> development
> >>> >> > > yet.
> >>> >> > > I need to delay it before the 2.10 release.
> >>> >> > >
> >>> >> > >
> >>> >> > >
> >>> >> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <
> >>> plehanov.alex@gmail.com>:
> >>> >> > >
> >>> >> > > > Denis,
> >>> >> > > >
> >>> >> > > > Yes, I still ready to manage it.
> >>> >> > > > Today I will prepare a release page on wiki and will try to go
> >>> over
> >>> >> > > tickets
> >>> >> > > > list.
> >>> >> > > > Also, I have plans to cut the branch by the end of next week
> if
> >>> >> there
> >>> >> > are
> >>> >> > > > no objections.
> >>> >> > > >
> >>> >> > > >
> >>> >> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> >>> >> > > >
> >>> >> > > > > Igniters,
> >>> >> > > > >
> >>> >> > > > > Are we moving forward with this release? Alex Plehanov, are
> >>> you
> >>> >> still
> >>> >> > > > ready
> >>> >> > > > > to manage it? It seems like everyone agreed with the
> timeline
> >>> you
> >>> >> > > > proposed
> >>> >> > > > > in the very beginning.
> >>> >> > > > >
> >>> >> > > > > -
> >>> >> > > > > Denis
> >>> >> > > > >
> >>> >> > > > >
> >>> >> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <
> >>> dmagda@apache.org>
> >>> >> > wrote:
> >>> >> > > > >
> >>> >> > > > > > Sergey, Ivan,
> >>> >> > > > > >
> >>> >> > > > > > Could you please check the questions below? If it's
> >>> >> time-consuming
> >>> >> > to
> >>> >> > > > > > rework continuous queries, then the new mode can become
> >>> >> available
> >>> >> > in
> >>> >> > > > the
> >>> >> > > > > > experimental state and should not let register continuous
> >>> >> queries
> >>> >> > to
> >>> >> > > > > avoid
> >>> >> > > > > > potential deadlocks. Overall, this design gap in
> continuous
> >>> >> queries
> >>> >> > > was
> >>> >> > > > > > like a bomb that has just detonated [1]. Anyway, this new
> >>> >> > > connectivity
> >>> >> > > > > mode
> >>> >> > > > > > will be priceless even if you can't use continuous queries
> >>> with
> >>> >> > them
> >>> >> > > > > > because right now we cannot even start a thick client
> >>> inside of
> >>> >> a
> >>> >> > > > > > serverless function.
> >>> >> > > > > >
> >>> >> > > > > > Alexey Plehanov,
> >>> >> > > > > >
> >>> >> > > > > > It looks like we can proceed with the release taking your
> >>> >> > timelines.
> >>> >> > > > > >
> >>> >> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> >>> >> > > > > >
> >>> >> > > > > > -
> >>> >> > > > > > Denis
> >>> >> > > > > >
> >>> >> > > > > >
> >>> >> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <
> >>> dmagda@apache.org>
> >>> >> > > wrote:
> >>> >> > > > > >
> >>> >> > > > > >> Ivan, Sergey,
> >>> >> > > > > >>
> >>> >> > > > > >> How much effort should we put to resolve the issue with
> >>> >> > > > > >> continuous queries? Are you working on that issue
> actively?
> >>> >> Let's
> >>> >> > > try
> >>> >> > > > to
> >>> >> > > > > >> guess what would be the ETA.
> >>> >> > > > > >>
> >>> >> > > > > >> -
> >>> >> > > > > >> Denis
> >>> >> > > > > >>
> >>> >> > > > > >>
> >>> >> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
> >>> >> > > bessonov.ip@gmail.com>
> >>> >> > > > > >> wrote:
> >>> >> > > > > >>
> >>> >> > > > > >>> Hello,
> >>> >> > > > > >>>
> >>> >> > > > > >>> Sorry for the delay. Sergey Chugunov (
> >>> >> sergey.chugunov@gmail.com)
> >>> >> > > > just
> >>> >> > > > > >>> replied
> >>> >> > > > > >>> to the main conversation about "communication via
> >>> discovery"
> >>> >> [1].
> >>> >> > > We
> >>> >> > > > > >>> work on it
> >>> >> > > > > >>> together and recently have found one hard-to-fix
> scenario,
> >>> >> > detailed
> >>> >> > > > > >>> description is
> >>> >> > > > > >>> provided in Sergey's reply.
> >>> >> > > > > >>>
> >>> >> > > > > >>> In short, July 10th looks realistic only if we introduce
> >>> new
> >>> >> > > behavior
> >>> >> > > > > in
> >>> >> > > > > >>> its current
> >>> >> > > > > >>> implementation, with new setting and IgniteExperimental
> >>> >> status.
> >>> >> > > > Blocker
> >>> >> > > > > >>> here is
> >>> >> > > > > >>> current implementation of Continuos Query protocol that
> in
> >>> >> some
> >>> >> > > cases
> >>> >> > > > > >>> (described
> >>> >> > > > > >>> at the end) initiates TCP connection right from
> discovery
> >>> >> thread
> >>> >> > > > which
> >>> >> > > > > >>> obviously
> >>> >> > > > > >>> leads to deadlock. We haven't estimated efforts needed
> to
> >>> >> > redesign
> >>> >> > > of
> >>> >> > > > > CQ
> >>> >> > > > > >>> protocol
> >>> >> > > > > >>> but it is definitely a risk and fixing it isn't feasible
> >>> with
> >>> >> a
> >>> >> > > code
> >>> >> > > > > >>> freeze at 10th of July.
> >>> >> > > > > >>> So my verdict: we can include this new feature in 2.9
> >>> scope as
> >>> >> > > > > >>> experimental and with
> >>> >> > > > > >>> highlighted limitation on CQ usage. Is that OK?
> >>> >> > > > > >>>
> >>> >> > > > > >>> CQ limitation: server needs to open a communication
> >>> >> connection to
> >>> >> > > the
> >>> >> > > > > >>> client if during
> >>> >> > > > > >>> CQ registration client tries to p2p deploy new class not
> >>> >> > available
> >>> >> > > on
> >>> >> > > > > >>> server classpath.
> >>> >> > > > > >>> In other cases registration of CQ should be fine.
> >>> >> > > > > >>>
> >>> >> > > > > >>> [1]
> >>> >> > > > > >>>
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >>> >> > > > > >>>
> >>> >> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <
> >>> ivan.glukos@gmail.com
> >>> >> >:
> >>> >> > > > > >>>
> >>> >> > > > > >>>> Hi,
> >>> >> > > > > >>>>
> >>> >> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
> >>> >> > > > communication
> >>> >> > > > > >>>> and
> >>> >> > > > > >>>> transactions tracing will be introduced, as well as an
> >>> >> option to
> >>> >> > > > > >>>> configure
> >>> >> > > > > >>>> tracing in runtime. Right now we are working on final
> >>> >> > performance
> >>> >> > > > > >>>> optimizations, but it's very likely that we'll complete
> >>> this
> >>> >> > > > activity
> >>> >> > > > > >>>> before the code freeze date.
> >>> >> > > > > >>>> Let's include tracing to the 2.9 release scope.
> >>> >> > > > > >>>>
> >>> >> > > > > >>>> More info:
> >>> >> > > > > >>>>
> >>> >> > > >
> >>> >>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> >>> >> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> >>> >> > > > > >>>>
> >>> >> > > > > >>>> --
> >>> >> > > > > >>>> Best Regards,
> >>> >> > > > > >>>> Ivan Rakov
> >>> >> > > > > >>>>
> >>> >> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <
> >>> >> dmagda@apache.org>
> >>> >> > > > wrote:
> >>> >> > > > > >>>>
> >>> >> > > > > >>>> > Hi folks,
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds
> >>> reasonable
> >>> >> to
> >>> >> > > me.
> >>> >> > > > > I'd
> >>> >> > > > > >>>> like
> >>> >> > > > > >>>> > only to hear inputs of @Ivan Rakov <
> >>> irakov@gridgain.com>,
> >>> >> who
> >>> >> > > is
> >>> >> > > > > >>>> about to
> >>> >> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
> >>> >> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
> >>> >> limitation
> >>> >> > > for
> >>> >> > > > K8
> >>> >> > > > > >>>> > deployments [1]. Most likely, both features will be
> >>> ready
> >>> >> by
> >>> >> > the
> >>> >> > > > > code
> >>> >> > > > > >>>> > freeze date (July 10), but the guys should know it
> >>> better.
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> > [1]
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>>
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> > -
> >>> >> > > > > >>>> > Denis
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> >>> >> > > > > plehanov.alex@gmail.com
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> > wrote:
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>> >> Hello Igniters,
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed
> here
> >>> [1]
> >>> >> its
> >>> >> > > > time
> >>> >> > > > > to
> >>> >> > > > > >>>> >> start
> >>> >> > > > > >>>> >> the discussion about 2.9 release.
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> I want to propose myself to be the release manager
> of
> >>> the
> >>> >> 2.9
> >>> >> > > > > >>>> release.
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> What about release time, I agree with Maxim that we
> >>> should
> >>> >> > > > deliver
> >>> >> > > > > >>>> >> features
> >>> >> > > > > >>>> >> as frequently as possible. If some feature doesn't
> fit
> >>> >> into
> >>> >> > > > release
> >>> >> > > > > >>>> dates
> >>> >> > > > > >>>> >> we should better include it into the next release
> and
> >>> >> > schedule
> >>> >> > > > the
> >>> >> > > > > >>>> next
> >>> >> > > > > >>>> >> release earlier then postpone the current release.
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> I propose the following dates for 2.9 release:
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> Scope Freeze: June 26, 2020
> >>> >> > > > > >>>> >> Code Freeze: July 10, 2020
> >>> >> > > > > >>>> >> Voting Date: July 31, 2020
> >>> >> > > > > >>>> >> Release Date: August 7, 2019
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> WDYT?
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >> [1] :
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>>
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >>> >> > > > > >>>> >>
> >>> >> > > > > >>>> >
> >>> >> > > > > >>>>
> >>> >> > > > > >>>
> >>> >> > > > > >>>
> >>> >> > > > > >>> --
> >>> >> > > > > >>> Sincerely yours,
> >>> >> > > > > >>> Ivan Bessonov
> >>> >> > > > > >>>
> >>> >> > > > > >>
> >>> >> > > > >
> >>> >> > > >
> >>> >> > >
> >>> >> >
> >>> >>
> >>> >
> >>>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
It seems that it works and we now have 2.9 nightlies.
Guys, can you please bump the master POM version to 2.10.0-SNAPSHOT?
Regards,
--
Ilya Kasnacheev
пт, 10 июл. 2020 г. в 17:43, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> I have also set up nightly builds to (hopefully) build 2.9
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 7 июл. 2020 г. в 17:04, Ilya Kasnacheev <il...@gmail.com>:
>
>> Hello!
>>
>> I have changed the schedule condition so ignite-2.9 will be run instead.
>> Please check if it works tomorrow.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 7 июл. 2020 г. в 16:42, Alex Plehanov <pl...@gmail.com>:
>>
>>> Guys,
>>>
>>> Can somebody help with creating scheduled build triggers for nightly "Run
>>> All" on TC for "ignite-2.9" branch?
>>>
>>> Also, I found that we still have a nightly "Run All" for "ignite-2.8"
>>> branch [1]. Do we still need it? Or just forgot to remove it after
>>> release?
>>>
>>> [1] :
>>>
>>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAllNightly&branch_IgniteTests24Java8=ignite-2.8&tab=buildTypeStatusDiv
>>>
>>> пт, 3 июл. 2020 г. в 23:18, Alex Plehanov <pl...@gmail.com>:
>>>
>>> > Anton, thank you.
>>> >
>>> > Guys,
>>> >
>>> > I've cut the release branch. From now on please pin resolved in the
>>> master
>>> > branch tickets to 2.10 release in Jira and cherry-pick targeted to 2.9
>>> > tickets to ignite-2.9 branch.
>>> >
>>> > As far as the release branch cut was delayed, I propose to move code
>>> > freeze dates by one week (to 17 July 2020). WDYT?
>>> >
>>> > I've moved long-time inactive non-blocker tickets (except documentation
>>> > related, we will deal with them later) from 2.9 release, please check
>>> your
>>> > Jira issues to have the correct "Fix version" field if your ticket is
>>> > planned to 2.9 release.
>>> >
>>> > All currently pinned to 2.9 issues can be found on the release page
>>> [1].
>>> >
>>> > I didn't get about [2] ticket mentioned in this thread, are we going to
>>> > include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please
>>> clarify?
>>> >
>>> > [1]:
>>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>>> > [2]: https://issues.apache.org/jira/browse/IGNITE-13156
>>> >
>>> > пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
>>> >
>>> >> You're now at the "Ignite Release Managers" group.
>>> >> Please check you gain access.
>>> >>
>>> >> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <
>>> plehanov.alex@gmail.com>
>>> >> wrote:
>>> >>
>>> >> > Guys,
>>> >> >
>>> >> > I've created the 2.9 release confluence page [1].
>>> >> > Also, I found that I don't have permission to Team City release
>>> tasks.
>>> >> Can
>>> >> > anyone give me such permissions?
>>> >> >
>>> >> > [1]:
>>> >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>>> >> >
>>> >> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <
>>> zaleslaw.sin@gmail.com>:
>>> >> >
>>> >> > > Igniters
>>> >> > >
>>> >> > > Unfortunately, the ML model export/import feature
>>> >> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
>>> >> development
>>> >> > > yet.
>>> >> > > I need to delay it before the 2.10 release.
>>> >> > >
>>> >> > >
>>> >> > >
>>> >> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <
>>> plehanov.alex@gmail.com>:
>>> >> > >
>>> >> > > > Denis,
>>> >> > > >
>>> >> > > > Yes, I still ready to manage it.
>>> >> > > > Today I will prepare a release page on wiki and will try to go
>>> over
>>> >> > > tickets
>>> >> > > > list.
>>> >> > > > Also, I have plans to cut the branch by the end of next week if
>>> >> there
>>> >> > are
>>> >> > > > no objections.
>>> >> > > >
>>> >> > > >
>>> >> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
>>> >> > > >
>>> >> > > > > Igniters,
>>> >> > > > >
>>> >> > > > > Are we moving forward with this release? Alex Plehanov, are
>>> you
>>> >> still
>>> >> > > > ready
>>> >> > > > > to manage it? It seems like everyone agreed with the timeline
>>> you
>>> >> > > > proposed
>>> >> > > > > in the very beginning.
>>> >> > > > >
>>> >> > > > > -
>>> >> > > > > Denis
>>> >> > > > >
>>> >> > > > >
>>> >> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <
>>> dmagda@apache.org>
>>> >> > wrote:
>>> >> > > > >
>>> >> > > > > > Sergey, Ivan,
>>> >> > > > > >
>>> >> > > > > > Could you please check the questions below? If it's
>>> >> time-consuming
>>> >> > to
>>> >> > > > > > rework continuous queries, then the new mode can become
>>> >> available
>>> >> > in
>>> >> > > > the
>>> >> > > > > > experimental state and should not let register continuous
>>> >> queries
>>> >> > to
>>> >> > > > > avoid
>>> >> > > > > > potential deadlocks. Overall, this design gap in continuous
>>> >> queries
>>> >> > > was
>>> >> > > > > > like a bomb that has just detonated [1]. Anyway, this new
>>> >> > > connectivity
>>> >> > > > > mode
>>> >> > > > > > will be priceless even if you can't use continuous queries
>>> with
>>> >> > them
>>> >> > > > > > because right now we cannot even start a thick client
>>> inside of
>>> >> a
>>> >> > > > > > serverless function.
>>> >> > > > > >
>>> >> > > > > > Alexey Plehanov,
>>> >> > > > > >
>>> >> > > > > > It looks like we can proceed with the release taking your
>>> >> > timelines.
>>> >> > > > > >
>>> >> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
>>> >> > > > > >
>>> >> > > > > > -
>>> >> > > > > > Denis
>>> >> > > > > >
>>> >> > > > > >
>>> >> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <
>>> dmagda@apache.org>
>>> >> > > wrote:
>>> >> > > > > >
>>> >> > > > > >> Ivan, Sergey,
>>> >> > > > > >>
>>> >> > > > > >> How much effort should we put to resolve the issue with
>>> >> > > > > >> continuous queries? Are you working on that issue actively?
>>> >> Let's
>>> >> > > try
>>> >> > > > to
>>> >> > > > > >> guess what would be the ETA.
>>> >> > > > > >>
>>> >> > > > > >> -
>>> >> > > > > >> Denis
>>> >> > > > > >>
>>> >> > > > > >>
>>> >> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
>>> >> > > bessonov.ip@gmail.com>
>>> >> > > > > >> wrote:
>>> >> > > > > >>
>>> >> > > > > >>> Hello,
>>> >> > > > > >>>
>>> >> > > > > >>> Sorry for the delay. Sergey Chugunov (
>>> >> sergey.chugunov@gmail.com)
>>> >> > > > just
>>> >> > > > > >>> replied
>>> >> > > > > >>> to the main conversation about "communication via
>>> discovery"
>>> >> [1].
>>> >> > > We
>>> >> > > > > >>> work on it
>>> >> > > > > >>> together and recently have found one hard-to-fix scenario,
>>> >> > detailed
>>> >> > > > > >>> description is
>>> >> > > > > >>> provided in Sergey's reply.
>>> >> > > > > >>>
>>> >> > > > > >>> In short, July 10th looks realistic only if we introduce
>>> new
>>> >> > > behavior
>>> >> > > > > in
>>> >> > > > > >>> its current
>>> >> > > > > >>> implementation, with new setting and IgniteExperimental
>>> >> status.
>>> >> > > > Blocker
>>> >> > > > > >>> here is
>>> >> > > > > >>> current implementation of Continuos Query protocol that in
>>> >> some
>>> >> > > cases
>>> >> > > > > >>> (described
>>> >> > > > > >>> at the end) initiates TCP connection right from discovery
>>> >> thread
>>> >> > > > which
>>> >> > > > > >>> obviously
>>> >> > > > > >>> leads to deadlock. We haven't estimated efforts needed to
>>> >> > redesign
>>> >> > > of
>>> >> > > > > CQ
>>> >> > > > > >>> protocol
>>> >> > > > > >>> but it is definitely a risk and fixing it isn't feasible
>>> with
>>> >> a
>>> >> > > code
>>> >> > > > > >>> freeze at 10th of July.
>>> >> > > > > >>> So my verdict: we can include this new feature in 2.9
>>> scope as
>>> >> > > > > >>> experimental and with
>>> >> > > > > >>> highlighted limitation on CQ usage. Is that OK?
>>> >> > > > > >>>
>>> >> > > > > >>> CQ limitation: server needs to open a communication
>>> >> connection to
>>> >> > > the
>>> >> > > > > >>> client if during
>>> >> > > > > >>> CQ registration client tries to p2p deploy new class not
>>> >> > available
>>> >> > > on
>>> >> > > > > >>> server classpath.
>>> >> > > > > >>> In other cases registration of CQ should be fine.
>>> >> > > > > >>>
>>> >> > > > > >>> [1]
>>> >> > > > > >>>
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>> >> > > > > >>>
>>> >> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <
>>> ivan.glukos@gmail.com
>>> >> >:
>>> >> > > > > >>>
>>> >> > > > > >>>> Hi,
>>> >> > > > > >>>>
>>> >> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
>>> >> > > > communication
>>> >> > > > > >>>> and
>>> >> > > > > >>>> transactions tracing will be introduced, as well as an
>>> >> option to
>>> >> > > > > >>>> configure
>>> >> > > > > >>>> tracing in runtime. Right now we are working on final
>>> >> > performance
>>> >> > > > > >>>> optimizations, but it's very likely that we'll complete
>>> this
>>> >> > > > activity
>>> >> > > > > >>>> before the code freeze date.
>>> >> > > > > >>>> Let's include tracing to the 2.9 release scope.
>>> >> > > > > >>>>
>>> >> > > > > >>>> More info:
>>> >> > > > > >>>>
>>> >> > > >
>>> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>>> >> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
>>> >> > > > > >>>>
>>> >> > > > > >>>> --
>>> >> > > > > >>>> Best Regards,
>>> >> > > > > >>>> Ivan Rakov
>>> >> > > > > >>>>
>>> >> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <
>>> >> dmagda@apache.org>
>>> >> > > > wrote:
>>> >> > > > > >>>>
>>> >> > > > > >>>> > Hi folks,
>>> >> > > > > >>>> >
>>> >> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds
>>> reasonable
>>> >> to
>>> >> > > me.
>>> >> > > > > I'd
>>> >> > > > > >>>> like
>>> >> > > > > >>>> > only to hear inputs of @Ivan Rakov <
>>> irakov@gridgain.com>,
>>> >> who
>>> >> > > is
>>> >> > > > > >>>> about to
>>> >> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
>>> >> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
>>> >> limitation
>>> >> > > for
>>> >> > > > K8
>>> >> > > > > >>>> > deployments [1]. Most likely, both features will be
>>> ready
>>> >> by
>>> >> > the
>>> >> > > > > code
>>> >> > > > > >>>> > freeze date (July 10), but the guys should know it
>>> better.
>>> >> > > > > >>>> >
>>> >> > > > > >>>> > [1]
>>> >> > > > > >>>> >
>>> >> > > > > >>>>
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>> >> > > > > >>>> >
>>> >> > > > > >>>> > -
>>> >> > > > > >>>> > Denis
>>> >> > > > > >>>> >
>>> >> > > > > >>>> >
>>> >> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
>>> >> > > > > plehanov.alex@gmail.com
>>> >> > > > > >>>> >
>>> >> > > > > >>>> > wrote:
>>> >> > > > > >>>> >
>>> >> > > > > >>>> >> Hello Igniters,
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed here
>>> [1]
>>> >> its
>>> >> > > > time
>>> >> > > > > to
>>> >> > > > > >>>> >> start
>>> >> > > > > >>>> >> the discussion about 2.9 release.
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> I want to propose myself to be the release manager of
>>> the
>>> >> 2.9
>>> >> > > > > >>>> release.
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> What about release time, I agree with Maxim that we
>>> should
>>> >> > > > deliver
>>> >> > > > > >>>> >> features
>>> >> > > > > >>>> >> as frequently as possible. If some feature doesn't fit
>>> >> into
>>> >> > > > release
>>> >> > > > > >>>> dates
>>> >> > > > > >>>> >> we should better include it into the next release and
>>> >> > schedule
>>> >> > > > the
>>> >> > > > > >>>> next
>>> >> > > > > >>>> >> release earlier then postpone the current release.
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> I propose the following dates for 2.9 release:
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> Scope Freeze: June 26, 2020
>>> >> > > > > >>>> >> Code Freeze: July 10, 2020
>>> >> > > > > >>>> >> Voting Date: July 31, 2020
>>> >> > > > > >>>> >> Release Date: August 7, 2019
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> WDYT?
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >> [1] :
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >>
>>> >> > > > > >>>>
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>> >> > > > > >>>> >>
>>> >> > > > > >>>> >
>>> >> > > > > >>>>
>>> >> > > > > >>>
>>> >> > > > > >>>
>>> >> > > > > >>> --
>>> >> > > > > >>> Sincerely yours,
>>> >> > > > > >>> Ivan Bessonov
>>> >> > > > > >>>
>>> >> > > > > >>
>>> >> > > > >
>>> >> > > >
>>> >> > >
>>> >> >
>>> >>
>>> >
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I have also set up nightly builds to (hopefully) build 2.9
Regards,
--
Ilya Kasnacheev
вт, 7 июл. 2020 г. в 17:04, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> I have changed the schedule condition so ignite-2.9 will be run instead.
> Please check if it works tomorrow.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 7 июл. 2020 г. в 16:42, Alex Plehanov <pl...@gmail.com>:
>
>> Guys,
>>
>> Can somebody help with creating scheduled build triggers for nightly "Run
>> All" on TC for "ignite-2.9" branch?
>>
>> Also, I found that we still have a nightly "Run All" for "ignite-2.8"
>> branch [1]. Do we still need it? Or just forgot to remove it after
>> release?
>>
>> [1] :
>>
>> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAllNightly&branch_IgniteTests24Java8=ignite-2.8&tab=buildTypeStatusDiv
>>
>> пт, 3 июл. 2020 г. в 23:18, Alex Plehanov <pl...@gmail.com>:
>>
>> > Anton, thank you.
>> >
>> > Guys,
>> >
>> > I've cut the release branch. From now on please pin resolved in the
>> master
>> > branch tickets to 2.10 release in Jira and cherry-pick targeted to 2.9
>> > tickets to ignite-2.9 branch.
>> >
>> > As far as the release branch cut was delayed, I propose to move code
>> > freeze dates by one week (to 17 July 2020). WDYT?
>> >
>> > I've moved long-time inactive non-blocker tickets (except documentation
>> > related, we will deal with them later) from 2.9 release, please check
>> your
>> > Jira issues to have the correct "Fix version" field if your ticket is
>> > planned to 2.9 release.
>> >
>> > All currently pinned to 2.9 issues can be found on the release page [1].
>> >
>> > I didn't get about [2] ticket mentioned in this thread, are we going to
>> > include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please
>> clarify?
>> >
>> > [1]:
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>> > [2]: https://issues.apache.org/jira/browse/IGNITE-13156
>> >
>> > пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
>> >
>> >> You're now at the "Ignite Release Managers" group.
>> >> Please check you gain access.
>> >>
>> >> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <plehanov.alex@gmail.com
>> >
>> >> wrote:
>> >>
>> >> > Guys,
>> >> >
>> >> > I've created the 2.9 release confluence page [1].
>> >> > Also, I found that I don't have permission to Team City release
>> tasks.
>> >> Can
>> >> > anyone give me such permissions?
>> >> >
>> >> > [1]:
>> >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>> >> >
>> >> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <zaleslaw.sin@gmail.com
>> >:
>> >> >
>> >> > > Igniters
>> >> > >
>> >> > > Unfortunately, the ML model export/import feature
>> >> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
>> >> development
>> >> > > yet.
>> >> > > I need to delay it before the 2.10 release.
>> >> > >
>> >> > >
>> >> > >
>> >> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <
>> plehanov.alex@gmail.com>:
>> >> > >
>> >> > > > Denis,
>> >> > > >
>> >> > > > Yes, I still ready to manage it.
>> >> > > > Today I will prepare a release page on wiki and will try to go
>> over
>> >> > > tickets
>> >> > > > list.
>> >> > > > Also, I have plans to cut the branch by the end of next week if
>> >> there
>> >> > are
>> >> > > > no objections.
>> >> > > >
>> >> > > >
>> >> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
>> >> > > >
>> >> > > > > Igniters,
>> >> > > > >
>> >> > > > > Are we moving forward with this release? Alex Plehanov, are you
>> >> still
>> >> > > > ready
>> >> > > > > to manage it? It seems like everyone agreed with the timeline
>> you
>> >> > > > proposed
>> >> > > > > in the very beginning.
>> >> > > > >
>> >> > > > > -
>> >> > > > > Denis
>> >> > > > >
>> >> > > > >
>> >> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dmagda@apache.org
>> >
>> >> > wrote:
>> >> > > > >
>> >> > > > > > Sergey, Ivan,
>> >> > > > > >
>> >> > > > > > Could you please check the questions below? If it's
>> >> time-consuming
>> >> > to
>> >> > > > > > rework continuous queries, then the new mode can become
>> >> available
>> >> > in
>> >> > > > the
>> >> > > > > > experimental state and should not let register continuous
>> >> queries
>> >> > to
>> >> > > > > avoid
>> >> > > > > > potential deadlocks. Overall, this design gap in continuous
>> >> queries
>> >> > > was
>> >> > > > > > like a bomb that has just detonated [1]. Anyway, this new
>> >> > > connectivity
>> >> > > > > mode
>> >> > > > > > will be priceless even if you can't use continuous queries
>> with
>> >> > them
>> >> > > > > > because right now we cannot even start a thick client inside
>> of
>> >> a
>> >> > > > > > serverless function.
>> >> > > > > >
>> >> > > > > > Alexey Plehanov,
>> >> > > > > >
>> >> > > > > > It looks like we can proceed with the release taking your
>> >> > timelines.
>> >> > > > > >
>> >> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
>> >> > > > > >
>> >> > > > > > -
>> >> > > > > > Denis
>> >> > > > > >
>> >> > > > > >
>> >> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <
>> dmagda@apache.org>
>> >> > > wrote:
>> >> > > > > >
>> >> > > > > >> Ivan, Sergey,
>> >> > > > > >>
>> >> > > > > >> How much effort should we put to resolve the issue with
>> >> > > > > >> continuous queries? Are you working on that issue actively?
>> >> Let's
>> >> > > try
>> >> > > > to
>> >> > > > > >> guess what would be the ETA.
>> >> > > > > >>
>> >> > > > > >> -
>> >> > > > > >> Denis
>> >> > > > > >>
>> >> > > > > >>
>> >> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
>> >> > > bessonov.ip@gmail.com>
>> >> > > > > >> wrote:
>> >> > > > > >>
>> >> > > > > >>> Hello,
>> >> > > > > >>>
>> >> > > > > >>> Sorry for the delay. Sergey Chugunov (
>> >> sergey.chugunov@gmail.com)
>> >> > > > just
>> >> > > > > >>> replied
>> >> > > > > >>> to the main conversation about "communication via
>> discovery"
>> >> [1].
>> >> > > We
>> >> > > > > >>> work on it
>> >> > > > > >>> together and recently have found one hard-to-fix scenario,
>> >> > detailed
>> >> > > > > >>> description is
>> >> > > > > >>> provided in Sergey's reply.
>> >> > > > > >>>
>> >> > > > > >>> In short, July 10th looks realistic only if we introduce
>> new
>> >> > > behavior
>> >> > > > > in
>> >> > > > > >>> its current
>> >> > > > > >>> implementation, with new setting and IgniteExperimental
>> >> status.
>> >> > > > Blocker
>> >> > > > > >>> here is
>> >> > > > > >>> current implementation of Continuos Query protocol that in
>> >> some
>> >> > > cases
>> >> > > > > >>> (described
>> >> > > > > >>> at the end) initiates TCP connection right from discovery
>> >> thread
>> >> > > > which
>> >> > > > > >>> obviously
>> >> > > > > >>> leads to deadlock. We haven't estimated efforts needed to
>> >> > redesign
>> >> > > of
>> >> > > > > CQ
>> >> > > > > >>> protocol
>> >> > > > > >>> but it is definitely a risk and fixing it isn't feasible
>> with
>> >> a
>> >> > > code
>> >> > > > > >>> freeze at 10th of July.
>> >> > > > > >>> So my verdict: we can include this new feature in 2.9
>> scope as
>> >> > > > > >>> experimental and with
>> >> > > > > >>> highlighted limitation on CQ usage. Is that OK?
>> >> > > > > >>>
>> >> > > > > >>> CQ limitation: server needs to open a communication
>> >> connection to
>> >> > > the
>> >> > > > > >>> client if during
>> >> > > > > >>> CQ registration client tries to p2p deploy new class not
>> >> > available
>> >> > > on
>> >> > > > > >>> server classpath.
>> >> > > > > >>> In other cases registration of CQ should be fine.
>> >> > > > > >>>
>> >> > > > > >>> [1]
>> >> > > > > >>>
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>> >> > > > > >>>
>> >> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <
>> ivan.glukos@gmail.com
>> >> >:
>> >> > > > > >>>
>> >> > > > > >>>> Hi,
>> >> > > > > >>>>
>> >> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
>> >> > > > communication
>> >> > > > > >>>> and
>> >> > > > > >>>> transactions tracing will be introduced, as well as an
>> >> option to
>> >> > > > > >>>> configure
>> >> > > > > >>>> tracing in runtime. Right now we are working on final
>> >> > performance
>> >> > > > > >>>> optimizations, but it's very likely that we'll complete
>> this
>> >> > > > activity
>> >> > > > > >>>> before the code freeze date.
>> >> > > > > >>>> Let's include tracing to the 2.9 release scope.
>> >> > > > > >>>>
>> >> > > > > >>>> More info:
>> >> > > > > >>>>
>> >> > > >
>> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>> >> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
>> >> > > > > >>>>
>> >> > > > > >>>> --
>> >> > > > > >>>> Best Regards,
>> >> > > > > >>>> Ivan Rakov
>> >> > > > > >>>>
>> >> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <
>> >> dmagda@apache.org>
>> >> > > > wrote:
>> >> > > > > >>>>
>> >> > > > > >>>> > Hi folks,
>> >> > > > > >>>> >
>> >> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds
>> reasonable
>> >> to
>> >> > > me.
>> >> > > > > I'd
>> >> > > > > >>>> like
>> >> > > > > >>>> > only to hear inputs of @Ivan Rakov <irakov@gridgain.com
>> >,
>> >> who
>> >> > > is
>> >> > > > > >>>> about to
>> >> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
>> >> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
>> >> limitation
>> >> > > for
>> >> > > > K8
>> >> > > > > >>>> > deployments [1]. Most likely, both features will be
>> ready
>> >> by
>> >> > the
>> >> > > > > code
>> >> > > > > >>>> > freeze date (July 10), but the guys should know it
>> better.
>> >> > > > > >>>> >
>> >> > > > > >>>> > [1]
>> >> > > > > >>>> >
>> >> > > > > >>>>
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>> >> > > > > >>>> >
>> >> > > > > >>>> > -
>> >> > > > > >>>> > Denis
>> >> > > > > >>>> >
>> >> > > > > >>>> >
>> >> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
>> >> > > > > plehanov.alex@gmail.com
>> >> > > > > >>>> >
>> >> > > > > >>>> > wrote:
>> >> > > > > >>>> >
>> >> > > > > >>>> >> Hello Igniters,
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed here
>> [1]
>> >> its
>> >> > > > time
>> >> > > > > to
>> >> > > > > >>>> >> start
>> >> > > > > >>>> >> the discussion about 2.9 release.
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> I want to propose myself to be the release manager of
>> the
>> >> 2.9
>> >> > > > > >>>> release.
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> What about release time, I agree with Maxim that we
>> should
>> >> > > > deliver
>> >> > > > > >>>> >> features
>> >> > > > > >>>> >> as frequently as possible. If some feature doesn't fit
>> >> into
>> >> > > > release
>> >> > > > > >>>> dates
>> >> > > > > >>>> >> we should better include it into the next release and
>> >> > schedule
>> >> > > > the
>> >> > > > > >>>> next
>> >> > > > > >>>> >> release earlier then postpone the current release.
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> I propose the following dates for 2.9 release:
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> Scope Freeze: June 26, 2020
>> >> > > > > >>>> >> Code Freeze: July 10, 2020
>> >> > > > > >>>> >> Voting Date: July 31, 2020
>> >> > > > > >>>> >> Release Date: August 7, 2019
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> WDYT?
>> >> > > > > >>>> >>
>> >> > > > > >>>> >> [1] :
>> >> > > > > >>>> >>
>> >> > > > > >>>> >>
>> >> > > > > >>>>
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>> >> > > > > >>>> >>
>> >> > > > > >>>> >
>> >> > > > > >>>>
>> >> > > > > >>>
>> >> > > > > >>>
>> >> > > > > >>> --
>> >> > > > > >>> Sincerely yours,
>> >> > > > > >>> Ivan Bessonov
>> >> > > > > >>>
>> >> > > > > >>
>> >> > > > >
>> >> > > >
>> >> > >
>> >> >
>> >>
>> >
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I have changed the schedule condition so ignite-2.9 will be run instead.
Please check if it works tomorrow.
Regards,
--
Ilya Kasnacheev
вт, 7 июл. 2020 г. в 16:42, Alex Plehanov <pl...@gmail.com>:
> Guys,
>
> Can somebody help with creating scheduled build triggers for nightly "Run
> All" on TC for "ignite-2.9" branch?
>
> Also, I found that we still have a nightly "Run All" for "ignite-2.8"
> branch [1]. Do we still need it? Or just forgot to remove it after release?
>
> [1] :
>
> https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAllNightly&branch_IgniteTests24Java8=ignite-2.8&tab=buildTypeStatusDiv
>
> пт, 3 июл. 2020 г. в 23:18, Alex Plehanov <pl...@gmail.com>:
>
> > Anton, thank you.
> >
> > Guys,
> >
> > I've cut the release branch. From now on please pin resolved in the
> master
> > branch tickets to 2.10 release in Jira and cherry-pick targeted to 2.9
> > tickets to ignite-2.9 branch.
> >
> > As far as the release branch cut was delayed, I propose to move code
> > freeze dates by one week (to 17 July 2020). WDYT?
> >
> > I've moved long-time inactive non-blocker tickets (except documentation
> > related, we will deal with them later) from 2.9 release, please check
> your
> > Jira issues to have the correct "Fix version" field if your ticket is
> > planned to 2.9 release.
> >
> > All currently pinned to 2.9 issues can be found on the release page [1].
> >
> > I didn't get about [2] ticket mentioned in this thread, are we going to
> > include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please
> clarify?
> >
> > [1]:
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> > [2]: https://issues.apache.org/jira/browse/IGNITE-13156
> >
> > пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
> >
> >> You're now at the "Ignite Release Managers" group.
> >> Please check you gain access.
> >>
> >> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <pl...@gmail.com>
> >> wrote:
> >>
> >> > Guys,
> >> >
> >> > I've created the 2.9 release confluence page [1].
> >> > Also, I found that I don't have permission to Team City release tasks.
> >> Can
> >> > anyone give me such permissions?
> >> >
> >> > [1]:
> >> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >> >
> >> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <zaleslaw.sin@gmail.com
> >:
> >> >
> >> > > Igniters
> >> > >
> >> > > Unfortunately, the ML model export/import feature
> >> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
> >> development
> >> > > yet.
> >> > > I need to delay it before the 2.10 release.
> >> > >
> >> > >
> >> > >
> >> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <plehanov.alex@gmail.com
> >:
> >> > >
> >> > > > Denis,
> >> > > >
> >> > > > Yes, I still ready to manage it.
> >> > > > Today I will prepare a release page on wiki and will try to go
> over
> >> > > tickets
> >> > > > list.
> >> > > > Also, I have plans to cut the branch by the end of next week if
> >> there
> >> > are
> >> > > > no objections.
> >> > > >
> >> > > >
> >> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> >> > > >
> >> > > > > Igniters,
> >> > > > >
> >> > > > > Are we moving forward with this release? Alex Plehanov, are you
> >> still
> >> > > > ready
> >> > > > > to manage it? It seems like everyone agreed with the timeline
> you
> >> > > > proposed
> >> > > > > in the very beginning.
> >> > > > >
> >> > > > > -
> >> > > > > Denis
> >> > > > >
> >> > > > >
> >> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org>
> >> > wrote:
> >> > > > >
> >> > > > > > Sergey, Ivan,
> >> > > > > >
> >> > > > > > Could you please check the questions below? If it's
> >> time-consuming
> >> > to
> >> > > > > > rework continuous queries, then the new mode can become
> >> available
> >> > in
> >> > > > the
> >> > > > > > experimental state and should not let register continuous
> >> queries
> >> > to
> >> > > > > avoid
> >> > > > > > potential deadlocks. Overall, this design gap in continuous
> >> queries
> >> > > was
> >> > > > > > like a bomb that has just detonated [1]. Anyway, this new
> >> > > connectivity
> >> > > > > mode
> >> > > > > > will be priceless even if you can't use continuous queries
> with
> >> > them
> >> > > > > > because right now we cannot even start a thick client inside
> of
> >> a
> >> > > > > > serverless function.
> >> > > > > >
> >> > > > > > Alexey Plehanov,
> >> > > > > >
> >> > > > > > It looks like we can proceed with the release taking your
> >> > timelines.
> >> > > > > >
> >> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> >> > > > > >
> >> > > > > > -
> >> > > > > > Denis
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <
> dmagda@apache.org>
> >> > > wrote:
> >> > > > > >
> >> > > > > >> Ivan, Sergey,
> >> > > > > >>
> >> > > > > >> How much effort should we put to resolve the issue with
> >> > > > > >> continuous queries? Are you working on that issue actively?
> >> Let's
> >> > > try
> >> > > > to
> >> > > > > >> guess what would be the ETA.
> >> > > > > >>
> >> > > > > >> -
> >> > > > > >> Denis
> >> > > > > >>
> >> > > > > >>
> >> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
> >> > > bessonov.ip@gmail.com>
> >> > > > > >> wrote:
> >> > > > > >>
> >> > > > > >>> Hello,
> >> > > > > >>>
> >> > > > > >>> Sorry for the delay. Sergey Chugunov (
> >> sergey.chugunov@gmail.com)
> >> > > > just
> >> > > > > >>> replied
> >> > > > > >>> to the main conversation about "communication via discovery"
> >> [1].
> >> > > We
> >> > > > > >>> work on it
> >> > > > > >>> together and recently have found one hard-to-fix scenario,
> >> > detailed
> >> > > > > >>> description is
> >> > > > > >>> provided in Sergey's reply.
> >> > > > > >>>
> >> > > > > >>> In short, July 10th looks realistic only if we introduce new
> >> > > behavior
> >> > > > > in
> >> > > > > >>> its current
> >> > > > > >>> implementation, with new setting and IgniteExperimental
> >> status.
> >> > > > Blocker
> >> > > > > >>> here is
> >> > > > > >>> current implementation of Continuos Query protocol that in
> >> some
> >> > > cases
> >> > > > > >>> (described
> >> > > > > >>> at the end) initiates TCP connection right from discovery
> >> thread
> >> > > > which
> >> > > > > >>> obviously
> >> > > > > >>> leads to deadlock. We haven't estimated efforts needed to
> >> > redesign
> >> > > of
> >> > > > > CQ
> >> > > > > >>> protocol
> >> > > > > >>> but it is definitely a risk and fixing it isn't feasible
> with
> >> a
> >> > > code
> >> > > > > >>> freeze at 10th of July.
> >> > > > > >>> So my verdict: we can include this new feature in 2.9 scope
> as
> >> > > > > >>> experimental and with
> >> > > > > >>> highlighted limitation on CQ usage. Is that OK?
> >> > > > > >>>
> >> > > > > >>> CQ limitation: server needs to open a communication
> >> connection to
> >> > > the
> >> > > > > >>> client if during
> >> > > > > >>> CQ registration client tries to p2p deploy new class not
> >> > available
> >> > > on
> >> > > > > >>> server classpath.
> >> > > > > >>> In other cases registration of CQ should be fine.
> >> > > > > >>>
> >> > > > > >>> [1]
> >> > > > > >>>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >> > > > > >>>
> >> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <
> ivan.glukos@gmail.com
> >> >:
> >> > > > > >>>
> >> > > > > >>>> Hi,
> >> > > > > >>>>
> >> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
> >> > > > communication
> >> > > > > >>>> and
> >> > > > > >>>> transactions tracing will be introduced, as well as an
> >> option to
> >> > > > > >>>> configure
> >> > > > > >>>> tracing in runtime. Right now we are working on final
> >> > performance
> >> > > > > >>>> optimizations, but it's very likely that we'll complete
> this
> >> > > > activity
> >> > > > > >>>> before the code freeze date.
> >> > > > > >>>> Let's include tracing to the 2.9 release scope.
> >> > > > > >>>>
> >> > > > > >>>> More info:
> >> > > > > >>>>
> >> > > >
> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> >> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> >> > > > > >>>>
> >> > > > > >>>> --
> >> > > > > >>>> Best Regards,
> >> > > > > >>>> Ivan Rakov
> >> > > > > >>>>
> >> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <
> >> dmagda@apache.org>
> >> > > > wrote:
> >> > > > > >>>>
> >> > > > > >>>> > Hi folks,
> >> > > > > >>>> >
> >> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds
> reasonable
> >> to
> >> > > me.
> >> > > > > I'd
> >> > > > > >>>> like
> >> > > > > >>>> > only to hear inputs of @Ivan Rakov <irakov@gridgain.com
> >,
> >> who
> >> > > is
> >> > > > > >>>> about to
> >> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
> >> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
> >> limitation
> >> > > for
> >> > > > K8
> >> > > > > >>>> > deployments [1]. Most likely, both features will be ready
> >> by
> >> > the
> >> > > > > code
> >> > > > > >>>> > freeze date (July 10), but the guys should know it
> better.
> >> > > > > >>>> >
> >> > > > > >>>> > [1]
> >> > > > > >>>> >
> >> > > > > >>>>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >> > > > > >>>> >
> >> > > > > >>>> > -
> >> > > > > >>>> > Denis
> >> > > > > >>>> >
> >> > > > > >>>> >
> >> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> >> > > > > plehanov.alex@gmail.com
> >> > > > > >>>> >
> >> > > > > >>>> > wrote:
> >> > > > > >>>> >
> >> > > > > >>>> >> Hello Igniters,
> >> > > > > >>>> >>
> >> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed here
> [1]
> >> its
> >> > > > time
> >> > > > > to
> >> > > > > >>>> >> start
> >> > > > > >>>> >> the discussion about 2.9 release.
> >> > > > > >>>> >>
> >> > > > > >>>> >> I want to propose myself to be the release manager of
> the
> >> 2.9
> >> > > > > >>>> release.
> >> > > > > >>>> >>
> >> > > > > >>>> >> What about release time, I agree with Maxim that we
> should
> >> > > > deliver
> >> > > > > >>>> >> features
> >> > > > > >>>> >> as frequently as possible. If some feature doesn't fit
> >> into
> >> > > > release
> >> > > > > >>>> dates
> >> > > > > >>>> >> we should better include it into the next release and
> >> > schedule
> >> > > > the
> >> > > > > >>>> next
> >> > > > > >>>> >> release earlier then postpone the current release.
> >> > > > > >>>> >>
> >> > > > > >>>> >> I propose the following dates for 2.9 release:
> >> > > > > >>>> >>
> >> > > > > >>>> >> Scope Freeze: June 26, 2020
> >> > > > > >>>> >> Code Freeze: July 10, 2020
> >> > > > > >>>> >> Voting Date: July 31, 2020
> >> > > > > >>>> >> Release Date: August 7, 2019
> >> > > > > >>>> >>
> >> > > > > >>>> >> WDYT?
> >> > > > > >>>> >>
> >> > > > > >>>> >> [1] :
> >> > > > > >>>> >>
> >> > > > > >>>> >>
> >> > > > > >>>>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >> > > > > >>>> >>
> >> > > > > >>>> >
> >> > > > > >>>>
> >> > > > > >>>
> >> > > > > >>>
> >> > > > > >>> --
> >> > > > > >>> Sincerely yours,
> >> > > > > >>> Ivan Bessonov
> >> > > > > >>>
> >> > > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
Can somebody help with creating scheduled build triggers for nightly "Run
All" on TC for "ignite-2.9" branch?
Also, I found that we still have a nightly "Run All" for "ignite-2.8"
branch [1]. Do we still need it? Or just forgot to remove it after release?
[1] :
https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_RunAllNightly&branch_IgniteTests24Java8=ignite-2.8&tab=buildTypeStatusDiv
пт, 3 июл. 2020 г. в 23:18, Alex Plehanov <pl...@gmail.com>:
> Anton, thank you.
>
> Guys,
>
> I've cut the release branch. From now on please pin resolved in the master
> branch tickets to 2.10 release in Jira and cherry-pick targeted to 2.9
> tickets to ignite-2.9 branch.
>
> As far as the release branch cut was delayed, I propose to move code
> freeze dates by one week (to 17 July 2020). WDYT?
>
> I've moved long-time inactive non-blocker tickets (except documentation
> related, we will deal with them later) from 2.9 release, please check your
> Jira issues to have the correct "Fix version" field if your ticket is
> planned to 2.9 release.
>
> All currently pinned to 2.9 issues can be found on the release page [1].
>
> I didn't get about [2] ticket mentioned in this thread, are we going to
> include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please clarify?
>
> [1]: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> [2]: https://issues.apache.org/jira/browse/IGNITE-13156
>
> пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
>
>> You're now at the "Ignite Release Managers" group.
>> Please check you gain access.
>>
>> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <pl...@gmail.com>
>> wrote:
>>
>> > Guys,
>> >
>> > I've created the 2.9 release confluence page [1].
>> > Also, I found that I don't have permission to Team City release tasks.
>> Can
>> > anyone give me such permissions?
>> >
>> > [1]:
>> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>> >
>> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <za...@gmail.com>:
>> >
>> > > Igniters
>> > >
>> > > Unfortunately, the ML model export/import feature
>> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
>> development
>> > > yet.
>> > > I need to delay it before the 2.10 release.
>> > >
>> > >
>> > >
>> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <pl...@gmail.com>:
>> > >
>> > > > Denis,
>> > > >
>> > > > Yes, I still ready to manage it.
>> > > > Today I will prepare a release page on wiki and will try to go over
>> > > tickets
>> > > > list.
>> > > > Also, I have plans to cut the branch by the end of next week if
>> there
>> > are
>> > > > no objections.
>> > > >
>> > > >
>> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
>> > > >
>> > > > > Igniters,
>> > > > >
>> > > > > Are we moving forward with this release? Alex Plehanov, are you
>> still
>> > > > ready
>> > > > > to manage it? It seems like everyone agreed with the timeline you
>> > > > proposed
>> > > > > in the very beginning.
>> > > > >
>> > > > > -
>> > > > > Denis
>> > > > >
>> > > > >
>> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org>
>> > wrote:
>> > > > >
>> > > > > > Sergey, Ivan,
>> > > > > >
>> > > > > > Could you please check the questions below? If it's
>> time-consuming
>> > to
>> > > > > > rework continuous queries, then the new mode can become
>> available
>> > in
>> > > > the
>> > > > > > experimental state and should not let register continuous
>> queries
>> > to
>> > > > > avoid
>> > > > > > potential deadlocks. Overall, this design gap in continuous
>> queries
>> > > was
>> > > > > > like a bomb that has just detonated [1]. Anyway, this new
>> > > connectivity
>> > > > > mode
>> > > > > > will be priceless even if you can't use continuous queries with
>> > them
>> > > > > > because right now we cannot even start a thick client inside of
>> a
>> > > > > > serverless function.
>> > > > > >
>> > > > > > Alexey Plehanov,
>> > > > > >
>> > > > > > It looks like we can proceed with the release taking your
>> > timelines.
>> > > > > >
>> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
>> > > > > >
>> > > > > > -
>> > > > > > Denis
>> > > > > >
>> > > > > >
>> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org>
>> > > wrote:
>> > > > > >
>> > > > > >> Ivan, Sergey,
>> > > > > >>
>> > > > > >> How much effort should we put to resolve the issue with
>> > > > > >> continuous queries? Are you working on that issue actively?
>> Let's
>> > > try
>> > > > to
>> > > > > >> guess what would be the ETA.
>> > > > > >>
>> > > > > >> -
>> > > > > >> Denis
>> > > > > >>
>> > > > > >>
>> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
>> > > bessonov.ip@gmail.com>
>> > > > > >> wrote:
>> > > > > >>
>> > > > > >>> Hello,
>> > > > > >>>
>> > > > > >>> Sorry for the delay. Sergey Chugunov (
>> sergey.chugunov@gmail.com)
>> > > > just
>> > > > > >>> replied
>> > > > > >>> to the main conversation about "communication via discovery"
>> [1].
>> > > We
>> > > > > >>> work on it
>> > > > > >>> together and recently have found one hard-to-fix scenario,
>> > detailed
>> > > > > >>> description is
>> > > > > >>> provided in Sergey's reply.
>> > > > > >>>
>> > > > > >>> In short, July 10th looks realistic only if we introduce new
>> > > behavior
>> > > > > in
>> > > > > >>> its current
>> > > > > >>> implementation, with new setting and IgniteExperimental
>> status.
>> > > > Blocker
>> > > > > >>> here is
>> > > > > >>> current implementation of Continuos Query protocol that in
>> some
>> > > cases
>> > > > > >>> (described
>> > > > > >>> at the end) initiates TCP connection right from discovery
>> thread
>> > > > which
>> > > > > >>> obviously
>> > > > > >>> leads to deadlock. We haven't estimated efforts needed to
>> > redesign
>> > > of
>> > > > > CQ
>> > > > > >>> protocol
>> > > > > >>> but it is definitely a risk and fixing it isn't feasible with
>> a
>> > > code
>> > > > > >>> freeze at 10th of July.
>> > > > > >>> So my verdict: we can include this new feature in 2.9 scope as
>> > > > > >>> experimental and with
>> > > > > >>> highlighted limitation on CQ usage. Is that OK?
>> > > > > >>>
>> > > > > >>> CQ limitation: server needs to open a communication
>> connection to
>> > > the
>> > > > > >>> client if during
>> > > > > >>> CQ registration client tries to p2p deploy new class not
>> > available
>> > > on
>> > > > > >>> server classpath.
>> > > > > >>> In other cases registration of CQ should be fine.
>> > > > > >>>
>> > > > > >>> [1]
>> > > > > >>>
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>> > > > > >>>
>> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <ivan.glukos@gmail.com
>> >:
>> > > > > >>>
>> > > > > >>>> Hi,
>> > > > > >>>>
>> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
>> > > > communication
>> > > > > >>>> and
>> > > > > >>>> transactions tracing will be introduced, as well as an
>> option to
>> > > > > >>>> configure
>> > > > > >>>> tracing in runtime. Right now we are working on final
>> > performance
>> > > > > >>>> optimizations, but it's very likely that we'll complete this
>> > > > activity
>> > > > > >>>> before the code freeze date.
>> > > > > >>>> Let's include tracing to the 2.9 release scope.
>> > > > > >>>>
>> > > > > >>>> More info:
>> > > > > >>>>
>> > > >
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
>> > > > > >>>>
>> > > > > >>>> --
>> > > > > >>>> Best Regards,
>> > > > > >>>> Ivan Rakov
>> > > > > >>>>
>> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <
>> dmagda@apache.org>
>> > > > wrote:
>> > > > > >>>>
>> > > > > >>>> > Hi folks,
>> > > > > >>>> >
>> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds reasonable
>> to
>> > > me.
>> > > > > I'd
>> > > > > >>>> like
>> > > > > >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>,
>> who
>> > > is
>> > > > > >>>> about to
>> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
>> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
>> limitation
>> > > for
>> > > > K8
>> > > > > >>>> > deployments [1]. Most likely, both features will be ready
>> by
>> > the
>> > > > > code
>> > > > > >>>> > freeze date (July 10), but the guys should know it better.
>> > > > > >>>> >
>> > > > > >>>> > [1]
>> > > > > >>>> >
>> > > > > >>>>
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>> > > > > >>>> >
>> > > > > >>>> > -
>> > > > > >>>> > Denis
>> > > > > >>>> >
>> > > > > >>>> >
>> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
>> > > > > plehanov.alex@gmail.com
>> > > > > >>>> >
>> > > > > >>>> > wrote:
>> > > > > >>>> >
>> > > > > >>>> >> Hello Igniters,
>> > > > > >>>> >>
>> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed here [1]
>> its
>> > > > time
>> > > > > to
>> > > > > >>>> >> start
>> > > > > >>>> >> the discussion about 2.9 release.
>> > > > > >>>> >>
>> > > > > >>>> >> I want to propose myself to be the release manager of the
>> 2.9
>> > > > > >>>> release.
>> > > > > >>>> >>
>> > > > > >>>> >> What about release time, I agree with Maxim that we should
>> > > > deliver
>> > > > > >>>> >> features
>> > > > > >>>> >> as frequently as possible. If some feature doesn't fit
>> into
>> > > > release
>> > > > > >>>> dates
>> > > > > >>>> >> we should better include it into the next release and
>> > schedule
>> > > > the
>> > > > > >>>> next
>> > > > > >>>> >> release earlier then postpone the current release.
>> > > > > >>>> >>
>> > > > > >>>> >> I propose the following dates for 2.9 release:
>> > > > > >>>> >>
>> > > > > >>>> >> Scope Freeze: June 26, 2020
>> > > > > >>>> >> Code Freeze: July 10, 2020
>> > > > > >>>> >> Voting Date: July 31, 2020
>> > > > > >>>> >> Release Date: August 7, 2019
>> > > > > >>>> >>
>> > > > > >>>> >> WDYT?
>> > > > > >>>> >>
>> > > > > >>>> >> [1] :
>> > > > > >>>> >>
>> > > > > >>>> >>
>> > > > > >>>>
>> > > > >
>> > > >
>> > >
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>> > > > > >>>> >>
>> > > > > >>>> >
>> > > > > >>>>
>> > > > > >>>
>> > > > > >>>
>> > > > > >>> --
>> > > > > >>> Sincerely yours,
>> > > > > >>> Ivan Bessonov
>> > > > > >>>
>> > > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Anton, thank you.
Guys,
I've cut the release branch. From now on please pin resolved in the master
branch tickets to 2.10 release in Jira and cherry-pick targeted to 2.9
tickets to ignite-2.9 branch.
As far as the release branch cut was delayed, I propose to move code freeze
dates by one week (to 17 July 2020). WDYT?
I've moved long-time inactive non-blocker tickets (except documentation
related, we will deal with them later) from 2.9 release, please check your
Jira issues to have the correct "Fix version" field if your ticket is
planned to 2.9 release.
All currently pinned to 2.9 issues can be found on the release page [1].
I didn't get about [2] ticket mentioned in this thread, are we going to
include it into 2.9? Ivan Bessonov, Sergey Chugunov, can you please clarify?
[1]: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
[2]: https://issues.apache.org/jira/browse/IGNITE-13156
пн, 29 июн. 2020 г. в 12:38, Anton Vinogradov <av...@apache.org>:
> You're now at the "Ignite Release Managers" group.
> Please check you gain access.
>
> On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <pl...@gmail.com>
> wrote:
>
> > Guys,
> >
> > I've created the 2.9 release confluence page [1].
> > Also, I found that I don't have permission to Team City release tasks.
> Can
> > anyone give me such permissions?
> >
> > [1]:
> https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
> >
> > пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <za...@gmail.com>:
> >
> > > Igniters
> > >
> > > Unfortunately, the ML model export/import feature
> > > <https://issues.apache.org/jira/browse/IGNITE-6642> is under
> development
> > > yet.
> > > I need to delay it before the 2.10 release.
> > >
> > >
> > >
> > > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <pl...@gmail.com>:
> > >
> > > > Denis,
> > > >
> > > > Yes, I still ready to manage it.
> > > > Today I will prepare a release page on wiki and will try to go over
> > > tickets
> > > > list.
> > > > Also, I have plans to cut the branch by the end of next week if there
> > are
> > > > no objections.
> > > >
> > > >
> > > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> > > >
> > > > > Igniters,
> > > > >
> > > > > Are we moving forward with this release? Alex Plehanov, are you
> still
> > > > ready
> > > > > to manage it? It seems like everyone agreed with the timeline you
> > > > proposed
> > > > > in the very beginning.
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org>
> > wrote:
> > > > >
> > > > > > Sergey, Ivan,
> > > > > >
> > > > > > Could you please check the questions below? If it's
> time-consuming
> > to
> > > > > > rework continuous queries, then the new mode can become available
> > in
> > > > the
> > > > > > experimental state and should not let register continuous queries
> > to
> > > > > avoid
> > > > > > potential deadlocks. Overall, this design gap in continuous
> queries
> > > was
> > > > > > like a bomb that has just detonated [1]. Anyway, this new
> > > connectivity
> > > > > mode
> > > > > > will be priceless even if you can't use continuous queries with
> > them
> > > > > > because right now we cannot even start a thick client inside of a
> > > > > > serverless function.
> > > > > >
> > > > > > Alexey Plehanov,
> > > > > >
> > > > > > It looks like we can proceed with the release taking your
> > timelines.
> > > > > >
> > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> > > > > >
> > > > > > -
> > > > > > Denis
> > > > > >
> > > > > >
> > > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org>
> > > wrote:
> > > > > >
> > > > > >> Ivan, Sergey,
> > > > > >>
> > > > > >> How much effort should we put to resolve the issue with
> > > > > >> continuous queries? Are you working on that issue actively?
> Let's
> > > try
> > > > to
> > > > > >> guess what would be the ETA.
> > > > > >>
> > > > > >> -
> > > > > >> Denis
> > > > > >>
> > > > > >>
> > > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
> > > bessonov.ip@gmail.com>
> > > > > >> wrote:
> > > > > >>
> > > > > >>> Hello,
> > > > > >>>
> > > > > >>> Sorry for the delay. Sergey Chugunov (
> sergey.chugunov@gmail.com)
> > > > just
> > > > > >>> replied
> > > > > >>> to the main conversation about "communication via discovery"
> [1].
> > > We
> > > > > >>> work on it
> > > > > >>> together and recently have found one hard-to-fix scenario,
> > detailed
> > > > > >>> description is
> > > > > >>> provided in Sergey's reply.
> > > > > >>>
> > > > > >>> In short, July 10th looks realistic only if we introduce new
> > > behavior
> > > > > in
> > > > > >>> its current
> > > > > >>> implementation, with new setting and IgniteExperimental status.
> > > > Blocker
> > > > > >>> here is
> > > > > >>> current implementation of Continuos Query protocol that in some
> > > cases
> > > > > >>> (described
> > > > > >>> at the end) initiates TCP connection right from discovery
> thread
> > > > which
> > > > > >>> obviously
> > > > > >>> leads to deadlock. We haven't estimated efforts needed to
> > redesign
> > > of
> > > > > CQ
> > > > > >>> protocol
> > > > > >>> but it is definitely a risk and fixing it isn't feasible with a
> > > code
> > > > > >>> freeze at 10th of July.
> > > > > >>> So my verdict: we can include this new feature in 2.9 scope as
> > > > > >>> experimental and with
> > > > > >>> highlighted limitation on CQ usage. Is that OK?
> > > > > >>>
> > > > > >>> CQ limitation: server needs to open a communication connection
> to
> > > the
> > > > > >>> client if during
> > > > > >>> CQ registration client tries to p2p deploy new class not
> > available
> > > on
> > > > > >>> server classpath.
> > > > > >>> In other cases registration of CQ should be fine.
> > > > > >>>
> > > > > >>> [1]
> > > > > >>>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > > > >>>
> > > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <ivan.glukos@gmail.com
> >:
> > > > > >>>
> > > > > >>>> Hi,
> > > > > >>>>
> > > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
> > > > communication
> > > > > >>>> and
> > > > > >>>> transactions tracing will be introduced, as well as an option
> to
> > > > > >>>> configure
> > > > > >>>> tracing in runtime. Right now we are working on final
> > performance
> > > > > >>>> optimizations, but it's very likely that we'll complete this
> > > > activity
> > > > > >>>> before the code freeze date.
> > > > > >>>> Let's include tracing to the 2.9 release scope.
> > > > > >>>>
> > > > > >>>> More info:
> > > > > >>>>
> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> > > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> > > > > >>>>
> > > > > >>>> --
> > > > > >>>> Best Regards,
> > > > > >>>> Ivan Rakov
> > > > > >>>>
> > > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dmagda@apache.org
> >
> > > > wrote:
> > > > > >>>>
> > > > > >>>> > Hi folks,
> > > > > >>>> >
> > > > > >>>> > The timelines proposed by Alex Plekhanov sounds reasonable
> to
> > > me.
> > > > > I'd
> > > > > >>>> like
> > > > > >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>,
> who
> > > is
> > > > > >>>> about to
> > > > > >>>> > finish with the tracing support, and @Ivan Bessonov
> > > > > >>>> > <ib...@gridgain.com>, who is fixing a serious
> limitation
> > > for
> > > > K8
> > > > > >>>> > deployments [1]. Most likely, both features will be ready by
> > the
> > > > > code
> > > > > >>>> > freeze date (July 10), but the guys should know it better.
> > > > > >>>> >
> > > > > >>>> > [1]
> > > > > >>>> >
> > > > > >>>>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > > > >>>> >
> > > > > >>>> > -
> > > > > >>>> > Denis
> > > > > >>>> >
> > > > > >>>> >
> > > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> > > > > plehanov.alex@gmail.com
> > > > > >>>> >
> > > > > >>>> > wrote:
> > > > > >>>> >
> > > > > >>>> >> Hello Igniters,
> > > > > >>>> >>
> > > > > >>>> >> AI 2.8.1 is finally released and as we discussed here [1]
> its
> > > > time
> > > > > to
> > > > > >>>> >> start
> > > > > >>>> >> the discussion about 2.9 release.
> > > > > >>>> >>
> > > > > >>>> >> I want to propose myself to be the release manager of the
> 2.9
> > > > > >>>> release.
> > > > > >>>> >>
> > > > > >>>> >> What about release time, I agree with Maxim that we should
> > > > deliver
> > > > > >>>> >> features
> > > > > >>>> >> as frequently as possible. If some feature doesn't fit into
> > > > release
> > > > > >>>> dates
> > > > > >>>> >> we should better include it into the next release and
> > schedule
> > > > the
> > > > > >>>> next
> > > > > >>>> >> release earlier then postpone the current release.
> > > > > >>>> >>
> > > > > >>>> >> I propose the following dates for 2.9 release:
> > > > > >>>> >>
> > > > > >>>> >> Scope Freeze: June 26, 2020
> > > > > >>>> >> Code Freeze: July 10, 2020
> > > > > >>>> >> Voting Date: July 31, 2020
> > > > > >>>> >> Release Date: August 7, 2019
> > > > > >>>> >>
> > > > > >>>> >> WDYT?
> > > > > >>>> >>
> > > > > >>>> >> [1] :
> > > > > >>>> >>
> > > > > >>>> >>
> > > > > >>>>
> > > > >
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > > > > >>>> >>
> > > > > >>>> >
> > > > > >>>>
> > > > > >>>
> > > > > >>>
> > > > > >>> --
> > > > > >>> Sincerely yours,
> > > > > >>> Ivan Bessonov
> > > > > >>>
> > > > > >>
> > > > >
> > > >
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Anton Vinogradov <av...@apache.org>.
You're now at the "Ignite Release Managers" group.
Please check you gain access.
On Fri, Jun 26, 2020 at 9:38 PM Alex Plehanov <pl...@gmail.com>
wrote:
> Guys,
>
> I've created the 2.9 release confluence page [1].
> Also, I found that I don't have permission to Team City release tasks. Can
> anyone give me such permissions?
>
> [1]: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
>
> пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <za...@gmail.com>:
>
> > Igniters
> >
> > Unfortunately, the ML model export/import feature
> > <https://issues.apache.org/jira/browse/IGNITE-6642> is under development
> > yet.
> > I need to delay it before the 2.10 release.
> >
> >
> >
> > пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <pl...@gmail.com>:
> >
> > > Denis,
> > >
> > > Yes, I still ready to manage it.
> > > Today I will prepare a release page on wiki and will try to go over
> > tickets
> > > list.
> > > Also, I have plans to cut the branch by the end of next week if there
> are
> > > no objections.
> > >
> > >
> > > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> > >
> > > > Igniters,
> > > >
> > > > Are we moving forward with this release? Alex Plehanov, are you still
> > > ready
> > > > to manage it? It seems like everyone agreed with the timeline you
> > > proposed
> > > > in the very beginning.
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org>
> wrote:
> > > >
> > > > > Sergey, Ivan,
> > > > >
> > > > > Could you please check the questions below? If it's time-consuming
> to
> > > > > rework continuous queries, then the new mode can become available
> in
> > > the
> > > > > experimental state and should not let register continuous queries
> to
> > > > avoid
> > > > > potential deadlocks. Overall, this design gap in continuous queries
> > was
> > > > > like a bomb that has just detonated [1]. Anyway, this new
> > connectivity
> > > > mode
> > > > > will be priceless even if you can't use continuous queries with
> them
> > > > > because right now we cannot even start a thick client inside of a
> > > > > serverless function.
> > > > >
> > > > > Alexey Plehanov,
> > > > >
> > > > > It looks like we can proceed with the release taking your
> timelines.
> > > > >
> > > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> > > > >
> > > > > -
> > > > > Denis
> > > > >
> > > > >
> > > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org>
> > wrote:
> > > > >
> > > > >> Ivan, Sergey,
> > > > >>
> > > > >> How much effort should we put to resolve the issue with
> > > > >> continuous queries? Are you working on that issue actively? Let's
> > try
> > > to
> > > > >> guess what would be the ETA.
> > > > >>
> > > > >> -
> > > > >> Denis
> > > > >>
> > > > >>
> > > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
> > bessonov.ip@gmail.com>
> > > > >> wrote:
> > > > >>
> > > > >>> Hello,
> > > > >>>
> > > > >>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com)
> > > just
> > > > >>> replied
> > > > >>> to the main conversation about "communication via discovery" [1].
> > We
> > > > >>> work on it
> > > > >>> together and recently have found one hard-to-fix scenario,
> detailed
> > > > >>> description is
> > > > >>> provided in Sergey's reply.
> > > > >>>
> > > > >>> In short, July 10th looks realistic only if we introduce new
> > behavior
> > > > in
> > > > >>> its current
> > > > >>> implementation, with new setting and IgniteExperimental status.
> > > Blocker
> > > > >>> here is
> > > > >>> current implementation of Continuos Query protocol that in some
> > cases
> > > > >>> (described
> > > > >>> at the end) initiates TCP connection right from discovery thread
> > > which
> > > > >>> obviously
> > > > >>> leads to deadlock. We haven't estimated efforts needed to
> redesign
> > of
> > > > CQ
> > > > >>> protocol
> > > > >>> but it is definitely a risk and fixing it isn't feasible with a
> > code
> > > > >>> freeze at 10th of July.
> > > > >>> So my verdict: we can include this new feature in 2.9 scope as
> > > > >>> experimental and with
> > > > >>> highlighted limitation on CQ usage. Is that OK?
> > > > >>>
> > > > >>> CQ limitation: server needs to open a communication connection to
> > the
> > > > >>> client if during
> > > > >>> CQ registration client tries to p2p deploy new class not
> available
> > on
> > > > >>> server classpath.
> > > > >>> In other cases registration of CQ should be fine.
> > > > >>>
> > > > >>> [1]
> > > > >>>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > > >>>
> > > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
> > > > >>>
> > > > >>>> Hi,
> > > > >>>>
> > > > >>>> Indeed, the tracing feature is almost ready. Discovery,
> > > communication
> > > > >>>> and
> > > > >>>> transactions tracing will be introduced, as well as an option to
> > > > >>>> configure
> > > > >>>> tracing in runtime. Right now we are working on final
> performance
> > > > >>>> optimizations, but it's very likely that we'll complete this
> > > activity
> > > > >>>> before the code freeze date.
> > > > >>>> Let's include tracing to the 2.9 release scope.
> > > > >>>>
> > > > >>>> More info:
> > > > >>>>
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> > > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> > > > >>>>
> > > > >>>> --
> > > > >>>> Best Regards,
> > > > >>>> Ivan Rakov
> > > > >>>>
> > > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org>
> > > wrote:
> > > > >>>>
> > > > >>>> > Hi folks,
> > > > >>>> >
> > > > >>>> > The timelines proposed by Alex Plekhanov sounds reasonable to
> > me.
> > > > I'd
> > > > >>>> like
> > > > >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who
> > is
> > > > >>>> about to
> > > > >>>> > finish with the tracing support, and @Ivan Bessonov
> > > > >>>> > <ib...@gridgain.com>, who is fixing a serious limitation
> > for
> > > K8
> > > > >>>> > deployments [1]. Most likely, both features will be ready by
> the
> > > > code
> > > > >>>> > freeze date (July 10), but the guys should know it better.
> > > > >>>> >
> > > > >>>> > [1]
> > > > >>>> >
> > > > >>>>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > > >>>> >
> > > > >>>> > -
> > > > >>>> > Denis
> > > > >>>> >
> > > > >>>> >
> > > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> > > > plehanov.alex@gmail.com
> > > > >>>> >
> > > > >>>> > wrote:
> > > > >>>> >
> > > > >>>> >> Hello Igniters,
> > > > >>>> >>
> > > > >>>> >> AI 2.8.1 is finally released and as we discussed here [1] its
> > > time
> > > > to
> > > > >>>> >> start
> > > > >>>> >> the discussion about 2.9 release.
> > > > >>>> >>
> > > > >>>> >> I want to propose myself to be the release manager of the 2.9
> > > > >>>> release.
> > > > >>>> >>
> > > > >>>> >> What about release time, I agree with Maxim that we should
> > > deliver
> > > > >>>> >> features
> > > > >>>> >> as frequently as possible. If some feature doesn't fit into
> > > release
> > > > >>>> dates
> > > > >>>> >> we should better include it into the next release and
> schedule
> > > the
> > > > >>>> next
> > > > >>>> >> release earlier then postpone the current release.
> > > > >>>> >>
> > > > >>>> >> I propose the following dates for 2.9 release:
> > > > >>>> >>
> > > > >>>> >> Scope Freeze: June 26, 2020
> > > > >>>> >> Code Freeze: July 10, 2020
> > > > >>>> >> Voting Date: July 31, 2020
> > > > >>>> >> Release Date: August 7, 2019
> > > > >>>> >>
> > > > >>>> >> WDYT?
> > > > >>>> >>
> > > > >>>> >> [1] :
> > > > >>>> >>
> > > > >>>> >>
> > > > >>>>
> > > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > > > >>>> >>
> > > > >>>> >
> > > > >>>>
> > > > >>>
> > > > >>>
> > > > >>> --
> > > > >>> Sincerely yours,
> > > > >>> Ivan Bessonov
> > > > >>>
> > > > >>
> > > >
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Guys,
I've created the 2.9 release confluence page [1].
Also, I found that I don't have permission to Team City release tasks. Can
anyone give me such permissions?
[1]: https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.9
пт, 26 июн. 2020 г. в 13:38, Alexey Zinoviev <za...@gmail.com>:
> Igniters
>
> Unfortunately, the ML model export/import feature
> <https://issues.apache.org/jira/browse/IGNITE-6642> is under development
> yet.
> I need to delay it before the 2.10 release.
>
>
>
> пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <pl...@gmail.com>:
>
> > Denis,
> >
> > Yes, I still ready to manage it.
> > Today I will prepare a release page on wiki and will try to go over
> tickets
> > list.
> > Also, I have plans to cut the branch by the end of next week if there are
> > no objections.
> >
> >
> > пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> >
> > > Igniters,
> > >
> > > Are we moving forward with this release? Alex Plehanov, are you still
> > ready
> > > to manage it? It seems like everyone agreed with the timeline you
> > proposed
> > > in the very beginning.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org> wrote:
> > >
> > > > Sergey, Ivan,
> > > >
> > > > Could you please check the questions below? If it's time-consuming to
> > > > rework continuous queries, then the new mode can become available in
> > the
> > > > experimental state and should not let register continuous queries to
> > > avoid
> > > > potential deadlocks. Overall, this design gap in continuous queries
> was
> > > > like a bomb that has just detonated [1]. Anyway, this new
> connectivity
> > > mode
> > > > will be priceless even if you can't use continuous queries with them
> > > > because right now we cannot even start a thick client inside of a
> > > > serverless function.
> > > >
> > > > Alexey Plehanov,
> > > >
> > > > It looks like we can proceed with the release taking your timelines.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> > > >
> > > > -
> > > > Denis
> > > >
> > > >
> > > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org>
> wrote:
> > > >
> > > >> Ivan, Sergey,
> > > >>
> > > >> How much effort should we put to resolve the issue with
> > > >> continuous queries? Are you working on that issue actively? Let's
> try
> > to
> > > >> guess what would be the ETA.
> > > >>
> > > >> -
> > > >> Denis
> > > >>
> > > >>
> > > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <
> bessonov.ip@gmail.com>
> > > >> wrote:
> > > >>
> > > >>> Hello,
> > > >>>
> > > >>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com)
> > just
> > > >>> replied
> > > >>> to the main conversation about "communication via discovery" [1].
> We
> > > >>> work on it
> > > >>> together and recently have found one hard-to-fix scenario, detailed
> > > >>> description is
> > > >>> provided in Sergey's reply.
> > > >>>
> > > >>> In short, July 10th looks realistic only if we introduce new
> behavior
> > > in
> > > >>> its current
> > > >>> implementation, with new setting and IgniteExperimental status.
> > Blocker
> > > >>> here is
> > > >>> current implementation of Continuos Query protocol that in some
> cases
> > > >>> (described
> > > >>> at the end) initiates TCP connection right from discovery thread
> > which
> > > >>> obviously
> > > >>> leads to deadlock. We haven't estimated efforts needed to redesign
> of
> > > CQ
> > > >>> protocol
> > > >>> but it is definitely a risk and fixing it isn't feasible with a
> code
> > > >>> freeze at 10th of July.
> > > >>> So my verdict: we can include this new feature in 2.9 scope as
> > > >>> experimental and with
> > > >>> highlighted limitation on CQ usage. Is that OK?
> > > >>>
> > > >>> CQ limitation: server needs to open a communication connection to
> the
> > > >>> client if during
> > > >>> CQ registration client tries to p2p deploy new class not available
> on
> > > >>> server classpath.
> > > >>> In other cases registration of CQ should be fine.
> > > >>>
> > > >>> [1]
> > > >>>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > >>>
> > > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
> > > >>>
> > > >>>> Hi,
> > > >>>>
> > > >>>> Indeed, the tracing feature is almost ready. Discovery,
> > communication
> > > >>>> and
> > > >>>> transactions tracing will be introduced, as well as an option to
> > > >>>> configure
> > > >>>> tracing in runtime. Right now we are working on final performance
> > > >>>> optimizations, but it's very likely that we'll complete this
> > activity
> > > >>>> before the code freeze date.
> > > >>>> Let's include tracing to the 2.9 release scope.
> > > >>>>
> > > >>>> More info:
> > > >>>>
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> > > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> > > >>>>
> > > >>>> --
> > > >>>> Best Regards,
> > > >>>> Ivan Rakov
> > > >>>>
> > > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org>
> > wrote:
> > > >>>>
> > > >>>> > Hi folks,
> > > >>>> >
> > > >>>> > The timelines proposed by Alex Plekhanov sounds reasonable to
> me.
> > > I'd
> > > >>>> like
> > > >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who
> is
> > > >>>> about to
> > > >>>> > finish with the tracing support, and @Ivan Bessonov
> > > >>>> > <ib...@gridgain.com>, who is fixing a serious limitation
> for
> > K8
> > > >>>> > deployments [1]. Most likely, both features will be ready by the
> > > code
> > > >>>> > freeze date (July 10), but the guys should know it better.
> > > >>>> >
> > > >>>> > [1]
> > > >>>> >
> > > >>>>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > > >>>> >
> > > >>>> > -
> > > >>>> > Denis
> > > >>>> >
> > > >>>> >
> > > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> > > plehanov.alex@gmail.com
> > > >>>> >
> > > >>>> > wrote:
> > > >>>> >
> > > >>>> >> Hello Igniters,
> > > >>>> >>
> > > >>>> >> AI 2.8.1 is finally released and as we discussed here [1] its
> > time
> > > to
> > > >>>> >> start
> > > >>>> >> the discussion about 2.9 release.
> > > >>>> >>
> > > >>>> >> I want to propose myself to be the release manager of the 2.9
> > > >>>> release.
> > > >>>> >>
> > > >>>> >> What about release time, I agree with Maxim that we should
> > deliver
> > > >>>> >> features
> > > >>>> >> as frequently as possible. If some feature doesn't fit into
> > release
> > > >>>> dates
> > > >>>> >> we should better include it into the next release and schedule
> > the
> > > >>>> next
> > > >>>> >> release earlier then postpone the current release.
> > > >>>> >>
> > > >>>> >> I propose the following dates for 2.9 release:
> > > >>>> >>
> > > >>>> >> Scope Freeze: June 26, 2020
> > > >>>> >> Code Freeze: July 10, 2020
> > > >>>> >> Voting Date: July 31, 2020
> > > >>>> >> Release Date: August 7, 2019
> > > >>>> >>
> > > >>>> >> WDYT?
> > > >>>> >>
> > > >>>> >> [1] :
> > > >>>> >>
> > > >>>> >>
> > > >>>>
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > > >>>> >>
> > > >>>> >
> > > >>>>
> > > >>>
> > > >>>
> > > >>> --
> > > >>> Sincerely yours,
> > > >>> Ivan Bessonov
> > > >>>
> > > >>
> > >
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Zinoviev <za...@gmail.com>.
Igniters
Unfortunately, the ML model export/import feature
<https://issues.apache.org/jira/browse/IGNITE-6642> is under development
yet.
I need to delay it before the 2.10 release.
пт, 26 июн. 2020 г. в 06:50, Alex Plehanov <pl...@gmail.com>:
> Denis,
>
> Yes, I still ready to manage it.
> Today I will prepare a release page on wiki and will try to go over tickets
> list.
> Also, I have plans to cut the branch by the end of next week if there are
> no objections.
>
>
> пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
>
> > Igniters,
> >
> > Are we moving forward with this release? Alex Plehanov, are you still
> ready
> > to manage it? It seems like everyone agreed with the timeline you
> proposed
> > in the very beginning.
> >
> > -
> > Denis
> >
> >
> > On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org> wrote:
> >
> > > Sergey, Ivan,
> > >
> > > Could you please check the questions below? If it's time-consuming to
> > > rework continuous queries, then the new mode can become available in
> the
> > > experimental state and should not let register continuous queries to
> > avoid
> > > potential deadlocks. Overall, this design gap in continuous queries was
> > > like a bomb that has just detonated [1]. Anyway, this new connectivity
> > mode
> > > will be priceless even if you can't use continuous queries with them
> > > because right now we cannot even start a thick client inside of a
> > > serverless function.
> > >
> > > Alexey Plehanov,
> > >
> > > It looks like we can proceed with the release taking your timelines.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org> wrote:
> > >
> > >> Ivan, Sergey,
> > >>
> > >> How much effort should we put to resolve the issue with
> > >> continuous queries? Are you working on that issue actively? Let's try
> to
> > >> guess what would be the ETA.
> > >>
> > >> -
> > >> Denis
> > >>
> > >>
> > >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <be...@gmail.com>
> > >> wrote:
> > >>
> > >>> Hello,
> > >>>
> > >>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com)
> just
> > >>> replied
> > >>> to the main conversation about "communication via discovery" [1]. We
> > >>> work on it
> > >>> together and recently have found one hard-to-fix scenario, detailed
> > >>> description is
> > >>> provided in Sergey's reply.
> > >>>
> > >>> In short, July 10th looks realistic only if we introduce new behavior
> > in
> > >>> its current
> > >>> implementation, with new setting and IgniteExperimental status.
> Blocker
> > >>> here is
> > >>> current implementation of Continuos Query protocol that in some cases
> > >>> (described
> > >>> at the end) initiates TCP connection right from discovery thread
> which
> > >>> obviously
> > >>> leads to deadlock. We haven't estimated efforts needed to redesign of
> > CQ
> > >>> protocol
> > >>> but it is definitely a risk and fixing it isn't feasible with a code
> > >>> freeze at 10th of July.
> > >>> So my verdict: we can include this new feature in 2.9 scope as
> > >>> experimental and with
> > >>> highlighted limitation on CQ usage. Is that OK?
> > >>>
> > >>> CQ limitation: server needs to open a communication connection to the
> > >>> client if during
> > >>> CQ registration client tries to p2p deploy new class not available on
> > >>> server classpath.
> > >>> In other cases registration of CQ should be fine.
> > >>>
> > >>> [1]
> > >>>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > >>>
> > >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
> > >>>
> > >>>> Hi,
> > >>>>
> > >>>> Indeed, the tracing feature is almost ready. Discovery,
> communication
> > >>>> and
> > >>>> transactions tracing will be introduced, as well as an option to
> > >>>> configure
> > >>>> tracing in runtime. Right now we are working on final performance
> > >>>> optimizations, but it's very likely that we'll complete this
> activity
> > >>>> before the code freeze date.
> > >>>> Let's include tracing to the 2.9 release scope.
> > >>>>
> > >>>> More info:
> > >>>>
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> > >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> > >>>>
> > >>>> --
> > >>>> Best Regards,
> > >>>> Ivan Rakov
> > >>>>
> > >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org>
> wrote:
> > >>>>
> > >>>> > Hi folks,
> > >>>> >
> > >>>> > The timelines proposed by Alex Plekhanov sounds reasonable to me.
> > I'd
> > >>>> like
> > >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is
> > >>>> about to
> > >>>> > finish with the tracing support, and @Ivan Bessonov
> > >>>> > <ib...@gridgain.com>, who is fixing a serious limitation for
> K8
> > >>>> > deployments [1]. Most likely, both features will be ready by the
> > code
> > >>>> > freeze date (July 10), but the guys should know it better.
> > >>>> >
> > >>>> > [1]
> > >>>> >
> > >>>>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> > >>>> >
> > >>>> > -
> > >>>> > Denis
> > >>>> >
> > >>>> >
> > >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> > plehanov.alex@gmail.com
> > >>>> >
> > >>>> > wrote:
> > >>>> >
> > >>>> >> Hello Igniters,
> > >>>> >>
> > >>>> >> AI 2.8.1 is finally released and as we discussed here [1] its
> time
> > to
> > >>>> >> start
> > >>>> >> the discussion about 2.9 release.
> > >>>> >>
> > >>>> >> I want to propose myself to be the release manager of the 2.9
> > >>>> release.
> > >>>> >>
> > >>>> >> What about release time, I agree with Maxim that we should
> deliver
> > >>>> >> features
> > >>>> >> as frequently as possible. If some feature doesn't fit into
> release
> > >>>> dates
> > >>>> >> we should better include it into the next release and schedule
> the
> > >>>> next
> > >>>> >> release earlier then postpone the current release.
> > >>>> >>
> > >>>> >> I propose the following dates for 2.9 release:
> > >>>> >>
> > >>>> >> Scope Freeze: June 26, 2020
> > >>>> >> Code Freeze: July 10, 2020
> > >>>> >> Voting Date: July 31, 2020
> > >>>> >> Release Date: August 7, 2019
> > >>>> >>
> > >>>> >> WDYT?
> > >>>> >>
> > >>>> >> [1] :
> > >>>> >>
> > >>>> >>
> > >>>>
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > >>>> >>
> > >>>> >
> > >>>>
> > >>>
> > >>>
> > >>> --
> > >>> Sincerely yours,
> > >>> Ivan Bessonov
> > >>>
> > >>
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Denis,
Yes, I still ready to manage it.
Today I will prepare a release page on wiki and will try to go over tickets
list.
Also, I have plans to cut the branch by the end of next week if there are
no objections.
пт, 26 июн. 2020 г. в 03:48, Denis Magda <dm...@apache.org>:
> Igniters,
>
> Are we moving forward with this release? Alex Plehanov, are you still ready
> to manage it? It seems like everyone agreed with the timeline you proposed
> in the very beginning.
>
> -
> Denis
>
>
> On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org> wrote:
>
> > Sergey, Ivan,
> >
> > Could you please check the questions below? If it's time-consuming to
> > rework continuous queries, then the new mode can become available in the
> > experimental state and should not let register continuous queries to
> avoid
> > potential deadlocks. Overall, this design gap in continuous queries was
> > like a bomb that has just detonated [1]. Anyway, this new connectivity
> mode
> > will be priceless even if you can't use continuous queries with them
> > because right now we cannot even start a thick client inside of a
> > serverless function.
> >
> > Alexey Plehanov,
> >
> > It looks like we can proceed with the release taking your timelines.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-13156
> >
> > -
> > Denis
> >
> >
> > On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org> wrote:
> >
> >> Ivan, Sergey,
> >>
> >> How much effort should we put to resolve the issue with
> >> continuous queries? Are you working on that issue actively? Let's try to
> >> guess what would be the ETA.
> >>
> >> -
> >> Denis
> >>
> >>
> >> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <be...@gmail.com>
> >> wrote:
> >>
> >>> Hello,
> >>>
> >>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
> >>> replied
> >>> to the main conversation about "communication via discovery" [1]. We
> >>> work on it
> >>> together and recently have found one hard-to-fix scenario, detailed
> >>> description is
> >>> provided in Sergey's reply.
> >>>
> >>> In short, July 10th looks realistic only if we introduce new behavior
> in
> >>> its current
> >>> implementation, with new setting and IgniteExperimental status. Blocker
> >>> here is
> >>> current implementation of Continuos Query protocol that in some cases
> >>> (described
> >>> at the end) initiates TCP connection right from discovery thread which
> >>> obviously
> >>> leads to deadlock. We haven't estimated efforts needed to redesign of
> CQ
> >>> protocol
> >>> but it is definitely a risk and fixing it isn't feasible with a code
> >>> freeze at 10th of July.
> >>> So my verdict: we can include this new feature in 2.9 scope as
> >>> experimental and with
> >>> highlighted limitation on CQ usage. Is that OK?
> >>>
> >>> CQ limitation: server needs to open a communication connection to the
> >>> client if during
> >>> CQ registration client tries to p2p deploy new class not available on
> >>> server classpath.
> >>> In other cases registration of CQ should be fine.
> >>>
> >>> [1]
> >>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >>>
> >>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
> >>>
> >>>> Hi,
> >>>>
> >>>> Indeed, the tracing feature is almost ready. Discovery, communication
> >>>> and
> >>>> transactions tracing will be introduced, as well as an option to
> >>>> configure
> >>>> tracing in runtime. Right now we are working on final performance
> >>>> optimizations, but it's very likely that we'll complete this activity
> >>>> before the code freeze date.
> >>>> Let's include tracing to the 2.9 release scope.
> >>>>
> >>>> More info:
> >>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> >>>> https://issues.apache.org/jira/browse/IGNITE-13060
> >>>>
> >>>> --
> >>>> Best Regards,
> >>>> Ivan Rakov
> >>>>
> >>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
> >>>>
> >>>> > Hi folks,
> >>>> >
> >>>> > The timelines proposed by Alex Plekhanov sounds reasonable to me.
> I'd
> >>>> like
> >>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is
> >>>> about to
> >>>> > finish with the tracing support, and @Ivan Bessonov
> >>>> > <ib...@gridgain.com>, who is fixing a serious limitation for K8
> >>>> > deployments [1]. Most likely, both features will be ready by the
> code
> >>>> > freeze date (July 10), but the guys should know it better.
> >>>> >
> >>>> > [1]
> >>>> >
> >>>>
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >>>> >
> >>>> > -
> >>>> > Denis
> >>>> >
> >>>> >
> >>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <
> plehanov.alex@gmail.com
> >>>> >
> >>>> > wrote:
> >>>> >
> >>>> >> Hello Igniters,
> >>>> >>
> >>>> >> AI 2.8.1 is finally released and as we discussed here [1] its time
> to
> >>>> >> start
> >>>> >> the discussion about 2.9 release.
> >>>> >>
> >>>> >> I want to propose myself to be the release manager of the 2.9
> >>>> release.
> >>>> >>
> >>>> >> What about release time, I agree with Maxim that we should deliver
> >>>> >> features
> >>>> >> as frequently as possible. If some feature doesn't fit into release
> >>>> dates
> >>>> >> we should better include it into the next release and schedule the
> >>>> next
> >>>> >> release earlier then postpone the current release.
> >>>> >>
> >>>> >> I propose the following dates for 2.9 release:
> >>>> >>
> >>>> >> Scope Freeze: June 26, 2020
> >>>> >> Code Freeze: July 10, 2020
> >>>> >> Voting Date: July 31, 2020
> >>>> >> Release Date: August 7, 2019
> >>>> >>
> >>>> >> WDYT?
> >>>> >>
> >>>> >> [1] :
> >>>> >>
> >>>> >>
> >>>>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >>>> >>
> >>>> >
> >>>>
> >>>
> >>>
> >>> --
> >>> Sincerely yours,
> >>> Ivan Bessonov
> >>>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Igniters,
Are we moving forward with this release? Alex Plehanov, are you still ready
to manage it? It seems like everyone agreed with the timeline you proposed
in the very beginning.
-
Denis
On Tue, Jun 16, 2020 at 8:52 AM Denis Magda <dm...@apache.org> wrote:
> Sergey, Ivan,
>
> Could you please check the questions below? If it's time-consuming to
> rework continuous queries, then the new mode can become available in the
> experimental state and should not let register continuous queries to avoid
> potential deadlocks. Overall, this design gap in continuous queries was
> like a bomb that has just detonated [1]. Anyway, this new connectivity mode
> will be priceless even if you can't use continuous queries with them
> because right now we cannot even start a thick client inside of a
> serverless function.
>
> Alexey Plehanov,
>
> It looks like we can proceed with the release taking your timelines.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-13156
>
> -
> Denis
>
>
> On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org> wrote:
>
>> Ivan, Sergey,
>>
>> How much effort should we put to resolve the issue with
>> continuous queries? Are you working on that issue actively? Let's try to
>> guess what would be the ETA.
>>
>> -
>> Denis
>>
>>
>> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <be...@gmail.com>
>> wrote:
>>
>>> Hello,
>>>
>>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
>>> replied
>>> to the main conversation about "communication via discovery" [1]. We
>>> work on it
>>> together and recently have found one hard-to-fix scenario, detailed
>>> description is
>>> provided in Sergey's reply.
>>>
>>> In short, July 10th looks realistic only if we introduce new behavior in
>>> its current
>>> implementation, with new setting and IgniteExperimental status. Blocker
>>> here is
>>> current implementation of Continuos Query protocol that in some cases
>>> (described
>>> at the end) initiates TCP connection right from discovery thread which
>>> obviously
>>> leads to deadlock. We haven't estimated efforts needed to redesign of CQ
>>> protocol
>>> but it is definitely a risk and fixing it isn't feasible with a code
>>> freeze at 10th of July.
>>> So my verdict: we can include this new feature in 2.9 scope as
>>> experimental and with
>>> highlighted limitation on CQ usage. Is that OK?
>>>
>>> CQ limitation: server needs to open a communication connection to the
>>> client if during
>>> CQ registration client tries to p2p deploy new class not available on
>>> server classpath.
>>> In other cases registration of CQ should be fine.
>>>
>>> [1]
>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>>
>>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
>>>
>>>> Hi,
>>>>
>>>> Indeed, the tracing feature is almost ready. Discovery, communication
>>>> and
>>>> transactions tracing will be introduced, as well as an option to
>>>> configure
>>>> tracing in runtime. Right now we are working on final performance
>>>> optimizations, but it's very likely that we'll complete this activity
>>>> before the code freeze date.
>>>> Let's include tracing to the 2.9 release scope.
>>>>
>>>> More info:
>>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>>>> https://issues.apache.org/jira/browse/IGNITE-13060
>>>>
>>>> --
>>>> Best Regards,
>>>> Ivan Rakov
>>>>
>>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
>>>>
>>>> > Hi folks,
>>>> >
>>>> > The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd
>>>> like
>>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is
>>>> about to
>>>> > finish with the tracing support, and @Ivan Bessonov
>>>> > <ib...@gridgain.com>, who is fixing a serious limitation for K8
>>>> > deployments [1]. Most likely, both features will be ready by the code
>>>> > freeze date (July 10), but the guys should know it better.
>>>> >
>>>> > [1]
>>>> >
>>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>>> >
>>>> > -
>>>> > Denis
>>>> >
>>>> >
>>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <plehanov.alex@gmail.com
>>>> >
>>>> > wrote:
>>>> >
>>>> >> Hello Igniters,
>>>> >>
>>>> >> AI 2.8.1 is finally released and as we discussed here [1] its time to
>>>> >> start
>>>> >> the discussion about 2.9 release.
>>>> >>
>>>> >> I want to propose myself to be the release manager of the 2.9
>>>> release.
>>>> >>
>>>> >> What about release time, I agree with Maxim that we should deliver
>>>> >> features
>>>> >> as frequently as possible. If some feature doesn't fit into release
>>>> dates
>>>> >> we should better include it into the next release and schedule the
>>>> next
>>>> >> release earlier then postpone the current release.
>>>> >>
>>>> >> I propose the following dates for 2.9 release:
>>>> >>
>>>> >> Scope Freeze: June 26, 2020
>>>> >> Code Freeze: July 10, 2020
>>>> >> Voting Date: July 31, 2020
>>>> >> Release Date: August 7, 2019
>>>> >>
>>>> >> WDYT?
>>>> >>
>>>> >> [1] :
>>>> >>
>>>> >>
>>>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>>> >>
>>>> >
>>>>
>>>
>>>
>>> --
>>> Sincerely yours,
>>> Ivan Bessonov
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Sergey, Ivan,
Could you please check the questions below? If it's time-consuming to
rework continuous queries, then the new mode can become available in the
experimental state and should not let register continuous queries to avoid
potential deadlocks. Overall, this design gap in continuous queries was
like a bomb that has just detonated [1]. Anyway, this new connectivity mode
will be priceless even if you can't use continuous queries with them
because right now we cannot even start a thick client inside of a
serverless function.
Alexey Plehanov,
It looks like we can proceed with the release taking your timelines.
[1] https://issues.apache.org/jira/browse/IGNITE-13156
-
Denis
On Wed, Jun 10, 2020 at 4:16 PM Denis Magda <dm...@apache.org> wrote:
> Ivan, Sergey,
>
> How much effort should we put to resolve the issue with
> continuous queries? Are you working on that issue actively? Let's try to
> guess what would be the ETA.
>
> -
> Denis
>
>
> On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <be...@gmail.com>
> wrote:
>
>> Hello,
>>
>> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
>> replied
>> to the main conversation about "communication via discovery" [1]. We work
>> on it
>> together and recently have found one hard-to-fix scenario, detailed
>> description is
>> provided in Sergey's reply.
>>
>> In short, July 10th looks realistic only if we introduce new behavior in
>> its current
>> implementation, with new setting and IgniteExperimental status. Blocker
>> here is
>> current implementation of Continuos Query protocol that in some cases
>> (described
>> at the end) initiates TCP connection right from discovery thread which
>> obviously
>> leads to deadlock. We haven't estimated efforts needed to redesign of CQ
>> protocol
>> but it is definitely a risk and fixing it isn't feasible with a code
>> freeze at 10th of July.
>> So my verdict: we can include this new feature in 2.9 scope as
>> experimental and with
>> highlighted limitation on CQ usage. Is that OK?
>>
>> CQ limitation: server needs to open a communication connection to the
>> client if during
>> CQ registration client tries to p2p deploy new class not available on
>> server classpath.
>> In other cases registration of CQ should be fine.
>>
>> [1]
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>
>> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
>>
>>> Hi,
>>>
>>> Indeed, the tracing feature is almost ready. Discovery, communication and
>>> transactions tracing will be introduced, as well as an option to
>>> configure
>>> tracing in runtime. Right now we are working on final performance
>>> optimizations, but it's very likely that we'll complete this activity
>>> before the code freeze date.
>>> Let's include tracing to the 2.9 release scope.
>>>
>>> More info:
>>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>>> https://issues.apache.org/jira/browse/IGNITE-13060
>>>
>>> --
>>> Best Regards,
>>> Ivan Rakov
>>>
>>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
>>>
>>> > Hi folks,
>>> >
>>> > The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd
>>> like
>>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is
>>> about to
>>> > finish with the tracing support, and @Ivan Bessonov
>>> > <ib...@gridgain.com>, who is fixing a serious limitation for K8
>>> > deployments [1]. Most likely, both features will be ready by the code
>>> > freeze date (July 10), but the guys should know it better.
>>> >
>>> > [1]
>>> >
>>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>>> >
>>> > -
>>> > Denis
>>> >
>>> >
>>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
>>> > wrote:
>>> >
>>> >> Hello Igniters,
>>> >>
>>> >> AI 2.8.1 is finally released and as we discussed here [1] its time to
>>> >> start
>>> >> the discussion about 2.9 release.
>>> >>
>>> >> I want to propose myself to be the release manager of the 2.9 release.
>>> >>
>>> >> What about release time, I agree with Maxim that we should deliver
>>> >> features
>>> >> as frequently as possible. If some feature doesn't fit into release
>>> dates
>>> >> we should better include it into the next release and schedule the
>>> next
>>> >> release earlier then postpone the current release.
>>> >>
>>> >> I propose the following dates for 2.9 release:
>>> >>
>>> >> Scope Freeze: June 26, 2020
>>> >> Code Freeze: July 10, 2020
>>> >> Voting Date: July 31, 2020
>>> >> Release Date: August 7, 2019
>>> >>
>>> >> WDYT?
>>> >>
>>> >> [1] :
>>> >>
>>> >>
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>> >>
>>> >
>>>
>>
>>
>> --
>> Sincerely yours,
>> Ivan Bessonov
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Ivan, Sergey,
How much effort should we put to resolve the issue with continuous queries?
Are you working on that issue actively? Let's try to guess what would be
the ETA.
-
Denis
On Wed, Jun 10, 2020 at 3:55 AM Ivan Bessonov <be...@gmail.com> wrote:
> Hello,
>
> Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
> replied
> to the main conversation about "communication via discovery" [1]. We work
> on it
> together and recently have found one hard-to-fix scenario, detailed
> description is
> provided in Sergey's reply.
>
> In short, July 10th looks realistic only if we introduce new behavior in
> its current
> implementation, with new setting and IgniteExperimental status. Blocker
> here is
> current implementation of Continuos Query protocol that in some cases
> (described
> at the end) initiates TCP connection right from discovery thread which
> obviously
> leads to deadlock. We haven't estimated efforts needed to redesign of CQ
> protocol
> but it is definitely a risk and fixing it isn't feasible with a code
> freeze at 10th of July.
> So my verdict: we can include this new feature in 2.9 scope as
> experimental and with
> highlighted limitation on CQ usage. Is that OK?
>
> CQ limitation: server needs to open a communication connection to the
> client if during
> CQ registration client tries to p2p deploy new class not available on
> server classpath.
> In other cases registration of CQ should be fine.
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>
> вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
>
>> Hi,
>>
>> Indeed, the tracing feature is almost ready. Discovery, communication and
>> transactions tracing will be introduced, as well as an option to configure
>> tracing in runtime. Right now we are working on final performance
>> optimizations, but it's very likely that we'll complete this activity
>> before the code freeze date.
>> Let's include tracing to the 2.9 release scope.
>>
>> More info:
>> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
>> https://issues.apache.org/jira/browse/IGNITE-13060
>>
>> --
>> Best Regards,
>> Ivan Rakov
>>
>> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
>>
>> > Hi folks,
>> >
>> > The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd
>> like
>> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is about
>> to
>> > finish with the tracing support, and @Ivan Bessonov
>> > <ib...@gridgain.com>, who is fixing a serious limitation for K8
>> > deployments [1]. Most likely, both features will be ready by the code
>> > freeze date (July 10), but the guys should know it better.
>> >
>> > [1]
>> >
>> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>> >
>> > -
>> > Denis
>> >
>> >
>> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
>> > wrote:
>> >
>> >> Hello Igniters,
>> >>
>> >> AI 2.8.1 is finally released and as we discussed here [1] its time to
>> >> start
>> >> the discussion about 2.9 release.
>> >>
>> >> I want to propose myself to be the release manager of the 2.9 release.
>> >>
>> >> What about release time, I agree with Maxim that we should deliver
>> >> features
>> >> as frequently as possible. If some feature doesn't fit into release
>> dates
>> >> we should better include it into the next release and schedule the next
>> >> release earlier then postpone the current release.
>> >>
>> >> I propose the following dates for 2.9 release:
>> >>
>> >> Scope Freeze: June 26, 2020
>> >> Code Freeze: July 10, 2020
>> >> Voting Date: July 31, 2020
>> >> Release Date: August 7, 2019
>> >>
>> >> WDYT?
>> >>
>> >> [1] :
>> >>
>> >>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>> >>
>> >
>>
>
>
> --
> Sincerely yours,
> Ivan Bessonov
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Bessonov <be...@gmail.com>.
Hello,
Sorry for the delay. Sergey Chugunov (sergey.chugunov@gmail.com) just
replied
to the main conversation about "communication via discovery" [1]. We work
on it
together and recently have found one hard-to-fix scenario, detailed
description is
provided in Sergey's reply.
In short, July 10th looks realistic only if we introduce new behavior in
its current
implementation, with new setting and IgniteExperimental status. Blocker
here is
current implementation of Continuos Query protocol that in some cases
(described
at the end) initiates TCP connection right from discovery thread which
obviously
leads to deadlock. We haven't estimated efforts needed to redesign of CQ
protocol
but it is definitely a risk and fixing it isn't feasible with a code freeze
at 10th of July.
So my verdict: we can include this new feature in 2.9 scope as experimental
and with
highlighted limitation on CQ usage. Is that OK?
CQ limitation: server needs to open a communication connection to the
client if during
CQ registration client tries to p2p deploy new class not available on
server classpath.
In other cases registration of CQ should be fine.
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
вт, 9 июн. 2020 г. в 19:36, Ivan Rakov <iv...@gmail.com>:
> Hi,
>
> Indeed, the tracing feature is almost ready. Discovery, communication and
> transactions tracing will be introduced, as well as an option to configure
> tracing in runtime. Right now we are working on final performance
> optimizations, but it's very likely that we'll complete this activity
> before the code freeze date.
> Let's include tracing to the 2.9 release scope.
>
> More info:
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
> https://issues.apache.org/jira/browse/IGNITE-13060
>
> --
> Best Regards,
> Ivan Rakov
>
> On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
>
> > Hi folks,
> >
> > The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd
> like
> > only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is about
> to
> > finish with the tracing support, and @Ivan Bessonov
> > <ib...@gridgain.com>, who is fixing a serious limitation for K8
> > deployments [1]. Most likely, both features will be ready by the code
> > freeze date (July 10), but the guys should know it better.
> >
> > [1]
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
> >
> > -
> > Denis
> >
> >
> > On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
> > wrote:
> >
> >> Hello Igniters,
> >>
> >> AI 2.8.1 is finally released and as we discussed here [1] its time to
> >> start
> >> the discussion about 2.9 release.
> >>
> >> I want to propose myself to be the release manager of the 2.9 release.
> >>
> >> What about release time, I agree with Maxim that we should deliver
> >> features
> >> as frequently as possible. If some feature doesn't fit into release
> dates
> >> we should better include it into the next release and schedule the next
> >> release earlier then postpone the current release.
> >>
> >> I propose the following dates for 2.9 release:
> >>
> >> Scope Freeze: June 26, 2020
> >> Code Freeze: July 10, 2020
> >> Voting Date: July 31, 2020
> >> Release Date: August 7, 2019
> >>
> >> WDYT?
> >>
> >> [1] :
> >>
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >>
> >
>
--
Sincerely yours,
Ivan Bessonov
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ivan Rakov <iv...@gmail.com>.
Hi,
Indeed, the tracing feature is almost ready. Discovery, communication and
transactions tracing will be introduced, as well as an option to configure
tracing in runtime. Right now we are working on final performance
optimizations, but it's very likely that we'll complete this activity
before the code freeze date.
Let's include tracing to the 2.9 release scope.
More info:
https://cwiki.apache.org/confluence/display/IGNITE/IEP-48%3A+Tracing
https://issues.apache.org/jira/browse/IGNITE-13060
--
Best Regards,
Ivan Rakov
On Sat, Jun 6, 2020 at 4:30 PM Denis Magda <dm...@apache.org> wrote:
> Hi folks,
>
> The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd like
> only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is about to
> finish with the tracing support, and @Ivan Bessonov
> <ib...@gridgain.com>, who is fixing a serious limitation for K8
> deployments [1]. Most likely, both features will be ready by the code
> freeze date (July 10), but the guys should know it better.
>
> [1]
> http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
>
> -
> Denis
>
>
> On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
> wrote:
>
>> Hello Igniters,
>>
>> AI 2.8.1 is finally released and as we discussed here [1] its time to
>> start
>> the discussion about 2.9 release.
>>
>> I want to propose myself to be the release manager of the 2.9 release.
>>
>> What about release time, I agree with Maxim that we should deliver
>> features
>> as frequently as possible. If some feature doesn't fit into release dates
>> we should better include it into the next release and schedule the next
>> release earlier then postpone the current release.
>>
>> I propose the following dates for 2.9 release:
>>
>> Scope Freeze: June 26, 2020
>> Code Freeze: July 10, 2020
>> Voting Date: July 31, 2020
>> Release Date: August 7, 2019
>>
>> WDYT?
>>
>> [1] :
>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Denis Magda <dm...@apache.org>.
Hi folks,
The timelines proposed by Alex Plekhanov sounds reasonable to me. I'd like
only to hear inputs of @Ivan Rakov <ir...@gridgain.com>, who is about to
finish with the tracing support, and @Ivan Bessonov
<ib...@gridgain.com>, who
is fixing a serious limitation for K8 deployments [1]. Most likely, both
features will be ready by the code freeze date (July 10), but the guys
should know it better.
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-New-Ignite-settings-for-IGNITE-12438-and-IGNITE-13013-td47586.html
-
Denis
On Wed, Jun 3, 2020 at 4:45 AM Alex Plehanov <pl...@gmail.com>
wrote:
> Hello Igniters,
>
> AI 2.8.1 is finally released and as we discussed here [1] its time to start
> the discussion about 2.9 release.
>
> I want to propose myself to be the release manager of the 2.9 release.
>
> What about release time, I agree with Maxim that we should deliver features
> as frequently as possible. If some feature doesn't fit into release dates
> we should better include it into the next release and schedule the next
> release earlier then postpone the current release.
>
> I propose the following dates for 2.9 release:
>
> Scope Freeze: June 26, 2020
> Code Freeze: July 10, 2020
> Voting Date: July 31, 2020
> Release Date: August 7, 2019
>
> WDYT?
>
> [1] :
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Maxim Muzafarov <mm...@apache.org>.
Ilya,
I've assumed that the next 2.10 release will start in August, so folks
can commit their changes without a rush. Another option, everyone can
ask in the current thread to wait for their changes :-)
On Wed, 3 Jun 2020 at 17:29, Ilya Kasnacheev <il...@gmail.com> wrote:
>
> Hello!
>
> I think we should have at least a month before code freeze. People may have
> code planned for commit and we don't want to rush them and commit
> sub-optimal code.
>
> Otherwise, +1.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 июн. 2020 г. в 17:24, Maxim Muzafarov <mm...@apache.org>:
>
> > +1 to start release and Alexey Plekhanov as a release manager.
> >
> >
> > Alexey, can you clarify why we need a month for scope and code freeze
> > phases? What should we wait for? Maybe I'm missing something, but the
> > scope for the release already exists.
> >
> > So, I'd like to suggest the following dates:
> >
> > Code Freeze: June 10, 2020
> > Voting Date: July 1, 2020
> > Release Date: July 7, 2019
> >
> > On Wed, 3 Jun 2020 at 16:38, Nikolay Izhikov <ni...@apache.org> wrote:
> > >
> > > +1 To start release process for 2.9
> > > +1 For Alexey Plekhanov as a release manager.
> > >
> > > > 3 июня 2020 г., в 16:34, Alex Plehanov <pl...@gmail.com>
> > написал(а):
> > > >
> > > > Ilya,
> > > >
> > > > We already have a lot of features implemented in the master branch,
> > but not
> > > > released (perhaps not so big as Calcite, but still useful), some of
> > them
> > > > already mentioned in the previous thread:
> > > > - Sandbox for user-defined code
> > > > - .NET: Native Near Cache
> > > > - TDE - Phase-2. Master key rotation
> > > > - Thin client: compute support
> > > > - Snapshots
> > > > - Tasks/services cancellation commands
> > > > etc.
> > > >
> > > > ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> > >:
> > > >
> > > >> Hello!
> > > >>
> > > >> Can you please clarify what is the scope of 2.9 release?
> > > >>
> > > >> I have a feeling that we don't really have any big features in the
> > current
> > > >> 2.9 code base. No Calcite, etc.
> > > >>
> > > >> So I'm asking whether it is worth it.
> > > >>
> > > >> Regards,
> > > >> --
> > > >> Ilya Kasnacheev
> > > >>
> > > >>
> > > >> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> > > >>
> > > >>> Hello Igniters,
> > > >>>
> > > >>> AI 2.8.1 is finally released and as we discussed here [1] its time to
> > > >> start
> > > >>> the discussion about 2.9 release.
> > > >>>
> > > >>> I want to propose myself to be the release manager of the 2.9
> > release.
> > > >>>
> > > >>> What about release time, I agree with Maxim that we should deliver
> > > >> features
> > > >>> as frequently as possible. If some feature doesn't fit into release
> > dates
> > > >>> we should better include it into the next release and schedule the
> > next
> > > >>> release earlier then postpone the current release.
> > > >>>
> > > >>> I propose the following dates for 2.9 release:
> > > >>>
> > > >>> Scope Freeze: June 26, 2020
> > > >>> Code Freeze: July 10, 2020
> > > >>> Voting Date: July 31, 2020
> > > >>> Release Date: August 7, 2019
> > > >>>
> > > >>> WDYT?
> > > >>>
> > > >>> [1] :
> > > >>>
> > > >>>
> > > >>
> > http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > > >>>
> > > >>
> > >
> >
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
I think we should have at least a month before code freeze. People may have
code planned for commit and we don't want to rush them and commit
sub-optimal code.
Otherwise, +1.
Regards,
--
Ilya Kasnacheev
ср, 3 июн. 2020 г. в 17:24, Maxim Muzafarov <mm...@apache.org>:
> +1 to start release and Alexey Plekhanov as a release manager.
>
>
> Alexey, can you clarify why we need a month for scope and code freeze
> phases? What should we wait for? Maybe I'm missing something, but the
> scope for the release already exists.
>
> So, I'd like to suggest the following dates:
>
> Code Freeze: June 10, 2020
> Voting Date: July 1, 2020
> Release Date: July 7, 2019
>
> On Wed, 3 Jun 2020 at 16:38, Nikolay Izhikov <ni...@apache.org> wrote:
> >
> > +1 To start release process for 2.9
> > +1 For Alexey Plekhanov as a release manager.
> >
> > > 3 июня 2020 г., в 16:34, Alex Plehanov <pl...@gmail.com>
> написал(а):
> > >
> > > Ilya,
> > >
> > > We already have a lot of features implemented in the master branch,
> but not
> > > released (perhaps not so big as Calcite, but still useful), some of
> them
> > > already mentioned in the previous thread:
> > > - Sandbox for user-defined code
> > > - .NET: Native Near Cache
> > > - TDE - Phase-2. Master key rotation
> > > - Thin client: compute support
> > > - Snapshots
> > > - Tasks/services cancellation commands
> > > etc.
> > >
> > > ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <ilya.kasnacheev@gmail.com
> >:
> > >
> > >> Hello!
> > >>
> > >> Can you please clarify what is the scope of 2.9 release?
> > >>
> > >> I have a feeling that we don't really have any big features in the
> current
> > >> 2.9 code base. No Calcite, etc.
> > >>
> > >> So I'm asking whether it is worth it.
> > >>
> > >> Regards,
> > >> --
> > >> Ilya Kasnacheev
> > >>
> > >>
> > >> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> > >>
> > >>> Hello Igniters,
> > >>>
> > >>> AI 2.8.1 is finally released and as we discussed here [1] its time to
> > >> start
> > >>> the discussion about 2.9 release.
> > >>>
> > >>> I want to propose myself to be the release manager of the 2.9
> release.
> > >>>
> > >>> What about release time, I agree with Maxim that we should deliver
> > >> features
> > >>> as frequently as possible. If some feature doesn't fit into release
> dates
> > >>> we should better include it into the next release and schedule the
> next
> > >>> release earlier then postpone the current release.
> > >>>
> > >>> I propose the following dates for 2.9 release:
> > >>>
> > >>> Scope Freeze: June 26, 2020
> > >>> Code Freeze: July 10, 2020
> > >>> Voting Date: July 31, 2020
> > >>> Release Date: August 7, 2019
> > >>>
> > >>> WDYT?
> > >>>
> > >>> [1] :
> > >>>
> > >>>
> > >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > >>>
> > >>
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Maxim Muzafarov <mm...@apache.org>.
+1 to start release and Alexey Plekhanov as a release manager.
Alexey, can you clarify why we need a month for scope and code freeze
phases? What should we wait for? Maybe I'm missing something, but the
scope for the release already exists.
So, I'd like to suggest the following dates:
Code Freeze: June 10, 2020
Voting Date: July 1, 2020
Release Date: July 7, 2019
On Wed, 3 Jun 2020 at 16:38, Nikolay Izhikov <ni...@apache.org> wrote:
>
> +1 To start release process for 2.9
> +1 For Alexey Plekhanov as a release manager.
>
> > 3 июня 2020 г., в 16:34, Alex Plehanov <pl...@gmail.com> написал(а):
> >
> > Ilya,
> >
> > We already have a lot of features implemented in the master branch, but not
> > released (perhaps not so big as Calcite, but still useful), some of them
> > already mentioned in the previous thread:
> > - Sandbox for user-defined code
> > - .NET: Native Near Cache
> > - TDE - Phase-2. Master key rotation
> > - Thin client: compute support
> > - Snapshots
> > - Tasks/services cancellation commands
> > etc.
> >
> > ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <il...@gmail.com>:
> >
> >> Hello!
> >>
> >> Can you please clarify what is the scope of 2.9 release?
> >>
> >> I have a feeling that we don't really have any big features in the current
> >> 2.9 code base. No Calcite, etc.
> >>
> >> So I'm asking whether it is worth it.
> >>
> >> Regards,
> >> --
> >> Ilya Kasnacheev
> >>
> >>
> >> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> >>
> >>> Hello Igniters,
> >>>
> >>> AI 2.8.1 is finally released and as we discussed here [1] its time to
> >> start
> >>> the discussion about 2.9 release.
> >>>
> >>> I want to propose myself to be the release manager of the 2.9 release.
> >>>
> >>> What about release time, I agree with Maxim that we should deliver
> >> features
> >>> as frequently as possible. If some feature doesn't fit into release dates
> >>> we should better include it into the next release and schedule the next
> >>> release earlier then postpone the current release.
> >>>
> >>> I propose the following dates for 2.9 release:
> >>>
> >>> Scope Freeze: June 26, 2020
> >>> Code Freeze: July 10, 2020
> >>> Voting Date: July 31, 2020
> >>> Release Date: August 7, 2019
> >>>
> >>> WDYT?
> >>>
> >>> [1] :
> >>>
> >>>
> >> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >>>
> >>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
+1 To start release process for 2.9
+1 For Alexey Plekhanov as a release manager.
> 3 июня 2020 г., в 16:34, Alex Plehanov <pl...@gmail.com> написал(а):
>
> Ilya,
>
> We already have a lot of features implemented in the master branch, but not
> released (perhaps not so big as Calcite, but still useful), some of them
> already mentioned in the previous thread:
> - Sandbox for user-defined code
> - .NET: Native Near Cache
> - TDE - Phase-2. Master key rotation
> - Thin client: compute support
> - Snapshots
> - Tasks/services cancellation commands
> etc.
>
> ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <il...@gmail.com>:
>
>> Hello!
>>
>> Can you please clarify what is the scope of 2.9 release?
>>
>> I have a feeling that we don't really have any big features in the current
>> 2.9 code base. No Calcite, etc.
>>
>> So I'm asking whether it is worth it.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
>>
>>> Hello Igniters,
>>>
>>> AI 2.8.1 is finally released and as we discussed here [1] its time to
>> start
>>> the discussion about 2.9 release.
>>>
>>> I want to propose myself to be the release manager of the 2.9 release.
>>>
>>> What about release time, I agree with Maxim that we should deliver
>> features
>>> as frequently as possible. If some feature doesn't fit into release dates
>>> we should better include it into the next release and schedule the next
>>> release earlier then postpone the current release.
>>>
>>> I propose the following dates for 2.9 release:
>>>
>>> Scope Freeze: June 26, 2020
>>> Code Freeze: July 10, 2020
>>> Voting Date: July 31, 2020
>>> Release Date: August 7, 2019
>>>
>>> WDYT?
>>>
>>> [1] :
>>>
>>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>>
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alex Plehanov <pl...@gmail.com>.
Ilya,
We already have a lot of features implemented in the master branch, but not
released (perhaps not so big as Calcite, but still useful), some of them
already mentioned in the previous thread:
- Sandbox for user-defined code
- .NET: Native Near Cache
- TDE - Phase-2. Master key rotation
- Thin client: compute support
- Snapshots
- Tasks/services cancellation commands
etc.
ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> Can you please clarify what is the scope of 2.9 release?
>
> I have a feeling that we don't really have any big features in the current
> 2.9 code base. No Calcite, etc.
>
> So I'm asking whether it is worth it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
>
> > Hello Igniters,
> >
> > AI 2.8.1 is finally released and as we discussed here [1] its time to
> start
> > the discussion about 2.9 release.
> >
> > I want to propose myself to be the release manager of the 2.9 release.
> >
> > What about release time, I agree with Maxim that we should deliver
> features
> > as frequently as possible. If some feature doesn't fit into release dates
> > we should better include it into the next release and schedule the next
> > release earlier then postpone the current release.
> >
> > I propose the following dates for 2.9 release:
> >
> > Scope Freeze: June 26, 2020
> > Code Freeze: July 10, 2020
> > Voting Date: July 31, 2020
> > Release Date: August 7, 2019
> >
> > WDYT?
> >
> > [1] :
> >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Anton Vinogradov <av...@apache.org>.
Cellular switch
On Wed, Jun 3, 2020 at 4:10 PM Nikolay Izhikov <ni...@apache.org> wrote:
> Snapshots.
>
> > 3 июня 2020 г., в 16:10, Ilya Kasnacheev <il...@gmail.com>
> написал(а):
> >
> > Hello!
> >
> > Can you please clarify what is the scope of 2.9 release?
> >
> > I have a feeling that we don't really have any big features in the
> current
> > 2.9 code base. No Calcite, etc.
> >
> > So I'm asking whether it is worth it.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> >
> >> Hello Igniters,
> >>
> >> AI 2.8.1 is finally released and as we discussed here [1] its time to
> start
> >> the discussion about 2.9 release.
> >>
> >> I want to propose myself to be the release manager of the 2.9 release.
> >>
> >> What about release time, I agree with Maxim that we should deliver
> features
> >> as frequently as possible. If some feature doesn't fit into release
> dates
> >> we should better include it into the next release and schedule the next
> >> release earlier then postpone the current release.
> >>
> >> I propose the following dates for 2.9 release:
> >>
> >> Scope Freeze: June 26, 2020
> >> Code Freeze: July 10, 2020
> >> Voting Date: July 31, 2020
> >> Release Date: August 7, 2019
> >>
> >> WDYT?
> >>
> >> [1] :
> >>
> >>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >>
>
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Nikolay Izhikov <ni...@apache.org>.
Snapshots.
> 3 июня 2020 г., в 16:10, Ilya Kasnacheev <il...@gmail.com> написал(а):
>
> Hello!
>
> Can you please clarify what is the scope of 2.9 release?
>
> I have a feeling that we don't really have any big features in the current
> 2.9 code base. No Calcite, etc.
>
> So I'm asking whether it is worth it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
>
>> Hello Igniters,
>>
>> AI 2.8.1 is finally released and as we discussed here [1] its time to start
>> the discussion about 2.9 release.
>>
>> I want to propose myself to be the release manager of the 2.9 release.
>>
>> What about release time, I agree with Maxim that we should deliver features
>> as frequently as possible. If some feature doesn't fit into release dates
>> we should better include it into the next release and schedule the next
>> release earlier then postpone the current release.
>>
>> I propose the following dates for 2.9 release:
>>
>> Scope Freeze: June 26, 2020
>> Code Freeze: July 10, 2020
>> Voting Date: July 31, 2020
>> Release Date: August 7, 2019
>>
>> WDYT?
>>
>> [1] :
>>
>> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Alexey Zinoviev <za...@gmail.com>.
+1 for initial dates, RM candidate, need June to finish and test features
Model export/import for ML and a fewminor fixes
ср, 3 июн. 2020 г., 18:05 Sergey Antonov <an...@gmail.com>:
> Cluster read-only mode and new cluster state change API.
>
> ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <il...@gmail.com>:
>
> > Hello!
> >
> > Can you please clarify what is the scope of 2.9 release?
> >
> > I have a feeling that we don't really have any big features in the
> current
> > 2.9 code base. No Calcite, etc.
> >
> > So I'm asking whether it is worth it.
> >
> > Regards,
> > --
> > Ilya Kasnacheev
> >
> >
> > ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> >
> > > Hello Igniters,
> > >
> > > AI 2.8.1 is finally released and as we discussed here [1] its time to
> > start
> > > the discussion about 2.9 release.
> > >
> > > I want to propose myself to be the release manager of the 2.9 release.
> > >
> > > What about release time, I agree with Maxim that we should deliver
> > features
> > > as frequently as possible. If some feature doesn't fit into release
> dates
> > > we should better include it into the next release and schedule the next
> > > release earlier then postpone the current release.
> > >
> > > I propose the following dates for 2.9 release:
> > >
> > > Scope Freeze: June 26, 2020
> > > Code Freeze: July 10, 2020
> > > Voting Date: July 31, 2020
> > > Release Date: August 7, 2019
> > >
> > > WDYT?
> > >
> > > [1] :
> > >
> > >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> > >
> >
>
>
> --
> BR, Sergey Antonov
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Sergey Antonov <an...@gmail.com>.
Cluster read-only mode and new cluster state change API.
ср, 3 июн. 2020 г. в 16:10, Ilya Kasnacheev <il...@gmail.com>:
> Hello!
>
> Can you please clarify what is the scope of 2.9 release?
>
> I have a feeling that we don't really have any big features in the current
> 2.9 code base. No Calcite, etc.
>
> So I'm asking whether it is worth it.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
>
> > Hello Igniters,
> >
> > AI 2.8.1 is finally released and as we discussed here [1] its time to
> start
> > the discussion about 2.9 release.
> >
> > I want to propose myself to be the release manager of the 2.9 release.
> >
> > What about release time, I agree with Maxim that we should deliver
> features
> > as frequently as possible. If some feature doesn't fit into release dates
> > we should better include it into the next release and schedule the next
> > release earlier then postpone the current release.
> >
> > I propose the following dates for 2.9 release:
> >
> > Scope Freeze: June 26, 2020
> > Code Freeze: July 10, 2020
> > Voting Date: July 31, 2020
> > Release Date: August 7, 2019
> >
> > WDYT?
> >
> > [1] :
> >
> >
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
> >
>
--
BR, Sergey Antonov
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Ilya Kasnacheev <il...@gmail.com>.
Hello!
Can you please clarify what is the scope of 2.9 release?
I have a feeling that we don't really have any big features in the current
2.9 code base. No Calcite, etc.
So I'm asking whether it is worth it.
Regards,
--
Ilya Kasnacheev
ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> Hello Igniters,
>
> AI 2.8.1 is finally released and as we discussed here [1] its time to start
> the discussion about 2.9 release.
>
> I want to propose myself to be the release manager of the 2.9 release.
>
> What about release time, I agree with Maxim that we should deliver features
> as frequently as possible. If some feature doesn't fit into release dates
> we should better include it into the next release and schedule the next
> release earlier then postpone the current release.
>
> I propose the following dates for 2.9 release:
>
> Scope Freeze: June 26, 2020
> Code Freeze: July 10, 2020
> Voting Date: July 31, 2020
> Release Date: August 7, 2019
>
> WDYT?
>
> [1] :
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>
Re: Apache Ignite 2.9.0 RELEASE [Time, Scope, Manager]
Posted by Dmitriy Pavlov <dp...@apache.org>.
++1 for Alexey as a release manager, that's good we are keeping a rotation
of RMs so every committer/PMC knows how to release the code.
0 for dates, actually, it is up to contributors to propose alternatives.
All in all, it is up to the discussion's course.
Sincerely,
Dmitriy Pavlov
ср, 3 июн. 2020 г. в 14:45, Alex Plehanov <pl...@gmail.com>:
> Hello Igniters,
>
> AI 2.8.1 is finally released and as we discussed here [1] its time to start
> the discussion about 2.9 release.
>
> I want to propose myself to be the release manager of the 2.9 release.
>
> What about release time, I agree with Maxim that we should deliver features
> as frequently as possible. If some feature doesn't fit into release dates
> we should better include it into the next release and schedule the next
> release earlier then postpone the current release.
>
> I propose the following dates for 2.9 release:
>
> Scope Freeze: June 26, 2020
> Code Freeze: July 10, 2020
> Voting Date: July 31, 2020
> Release Date: August 7, 2019
>
> WDYT?
>
> [1] :
>
> http://apache-ignite-developers.2346864.n4.nabble.com/Ignite-Releases-Plan-td47360.html#a47575
>