[jira] (IGNITE-17274) Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor

["Pavel Tupitsyn (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/IGNITE-17274 ]


    Pavel Tupitsyn deleted comment on IGNITE-17274:
    -----------------------------------------

was (Author: ignitetcbot):
{panel:title=Branch: [pull/10123/head] Base: [master] : Possible Blockers (10)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS (Compatibility){color} [[tests 0 TIMEOUT , Exit Code |https://ci.ignite.apache.org/viewLog.html?buildId=6660612]]

{color:#d04437}PDS (Indexing){color} [[tests 0 TIMEOUT , Exit Code |https://ci.ignite.apache.org/viewLog.html?buildId=6660613]]

{color:#d04437}Snapshots{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=6660632]]
* IgniteSnapshotTestSuite: EncryptedSnapshotTest.testSnapshotRestoringAfterMultipleReencryption[Encryption is enabled.] - Test has low fail rate in base branch 0,0% and is not flaky

{color:#d04437}JDBC Driver{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=6660596]]
* IgniteJdbcDriverTestSuite: JdbcThinJdbcToCacheDataTypesCoverageTest.testIntDataType[atomicityMode=ATOMIC, cacheMode=PARTITIONED, ttlFactory=null, backups=2, evictionFactory=null, onheapCacheEnabled=false, writeSyncMode=FULL_SYNC, persistenceEnabled=false, useBinaryArrays=false] - Test has low fail rate in base branch 0,0% and is not flaky

{color:#d04437}Cache 1{color} [[tests 0 TIMEOUT , Out Of Memory Error , Exit Code |https://ci.ignite.apache.org/viewLog.html?buildId=6660546]]

{color:#d04437}Queries 2{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=6660622]]
* org.apache.ignite.testsuites.IgniteBinaryCacheQueryTestSuite2: org.apache.ignite.internal.processors.cache.distributed.near.IgniteCacheQueryNodeRestartDistributedJoinSelfTest. - History for base branch is absent.

{color:#d04437}Queries 1 (lazy=true){color} [[tests 1 TIMEOUT , Exit Code |https://ci.ignite.apache.org/viewLog.html?buildId=6660621]]
* IgniteBinaryCacheQueryLazyTestSuite: BasicIndexMultinodeTest.testEqualFieldsDynamicIndexesWithPersistence - Test has low fail rate in base branch 0,0% and is not flaky

{color:#d04437}Platform C++ CMake (Win x64 / Release){color} [[tests 0 BuildFailureOnMessage |https://ci.ignite.apache.org/viewLog.html?buildId=6660615]]

{color:#d04437}Continuous Query 1{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=6660578]]
* IgniteCacheQuerySelfTestSuite3: CacheContinuousQueryCounterReplicatedTxTest.testTwoQueryListener - Test has low fail rate in base branch 0,0% and is not flaky

{panel}
{panel:title=Branch: [pull/10123/head] Base: [master] : New Tests (1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#00008b}Queries 2{color} [[tests 1|https://ci.ignite.apache.org/viewLog.html?buildId=6660622]]
* {color:#8b0000}org.apache.ignite.testsuites.IgniteBinaryCacheQueryTestSuite2: org.apache.ignite.internal.processors.cache.distributed.near.IgniteCacheQueryNodeRestartDistributedJoinSelfTest. - FAILED{color}

{panel}
[TeamCity *--> Run :: All* Results|https://ci.ignite.apache.org/viewLog.html?buildId=6660650&buildTypeId=IgniteTests24Java8_RunAll]

> Startup of large numbers of servers slowed by linear lookup in IgniteServiceProcessor
> -------------------------------------------------------------------------------------
>
>                 Key: IGNITE-17274
>                 URL: https://issues.apache.org/jira/browse/IGNITE-17274
>             Project: Ignite
>          Issue Type: Improvement
>          Components: managed services
>    Affects Versions: 2.13
>            Reporter: Arthur Naseef
>            Assignee: Arthur Naseef
>            Priority: Major
>             Fix For: 2.14
>
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Using a small POC, spinning up many servers is slow.  In addition, the startup time appears to be exponential.
> Using timing measurements, found a linear lookup inside the IgniteServiceProcessor that is taking most of the time.
> Replacing that linear lookup with a Map lookup, and maintaining the map, significantly speeds up the process, and startup time is now linear with the number of services started.
> Note this was tested with 20K and 50K services on a 1-node ignite cluster.  Timings against the stock 2.13.0 code come in at 30s for 20K and 250s for 50K services.  Modifying the linear lookup to use a Map, the timing come in at 8s for 20K and 14s for 50K services.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)