[jira] [Commented] (UNOMI-725) Session/Event index rollover: measure performance of session being loaded by query

["Kevan Jahanshahi (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/UNOMI-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17686603#comment-17686603 ] 

Kevan Jahanshahi commented on UNOMI-725:
----------------------------------------

h2. Unomi 2.1.0

150 users, 2 loops, 10sec ramp-up, 10sec ES Ilm polling
180 users, 2 loops, 10sec ramp-up, 10sec ES Ilm polling
200 users, 2 loops, 10sec ramp-up, 10sec ES Ilm polling
220 users, 2 loops, 10sec ramp-up, 10sec ES Ilm polling
{code:java}
Test type, Run Name, sampleCount, errorPct, avgResTime, 95%ile, throughput 
150 users-10sec, Unomi-test-plan, 5100, 0, 488.02823529411825, 1285.5499999999984, 119.0198366394399 
180 users-10sec, Unomi-test-plan, 6120, 0, 482.8083333333331, 1300.8999999999996, 147.2959638017762 
200 users-10sec, Unomi-test-plan, 6800, 0, 560.5782352941164, 1319.8499999999995, 153.24288998061928 
220 users-10sec, Unomi-test-plan, 7480, 0, 761.4437165775416, 1752.9499999999998, 145.4969850223692 {code}
h2. Unomi 2.2.0-SNAPSHOT

150 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
180 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
200 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
200 users, 2 loops, 10sec ramp-up, rollover: 180 maxDocs, 10sec ES Ilm polling
{code:java}
Test type, Run Name, sampleCount, errorPct, avgResTime, 95%ile, throughput
150 users-10sec, Unomi-test-plan, 5100, 0, 505.70843137255065, 1374.9499999999998, 116.22077389362381 
180 users-10sec, Unomi-test-plan, 6120, 0, 655.0895424836592, 1595.9499999999998, 126.76056338028168 
200 users-10sec, Unomi-test-plan, 6800, 0, 779.6564705882371, 1811.8999999999996, 129.15970217292204 
220 users-10sec, Unomi-test-plan, 7480, 0, 1456.5633689839567, 3418.95, 96.36194991239823 {code}
h2. Unomi 2.2.0-SNAPSHOT (with cache affinity POC)

150 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
180 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
200 users, 2 loops, 10sec ramp-up, rollover: 150 maxDocs, 10sec ES Ilm polling
200 users, 2 loops, 10sec ramp-up, rollover: 180 maxDocs, 10sec ES Ilm polling
{code:java}
Test type, Run Name, sampleCount, errorPct, avgResTime, 95%ile, throughput
150 users-10sec, Unomi-test-plan, 5100, 0, 487.8654901960784, 1321.9499999999998, 117.41682974559686
180 users-10sec, Unomi-test-plan, 6120, 0, 601.6326797385593, 1498.9499999999998, 131.20096042533123  
200 users-10sec, Unomi-test-plan, 6800, 0, 747.7475000000022, 1960, 132.65186687995006
220 users-10sec, Unomi-test-plan, 7480, 0, 1005.6133689839573, 2685, 122.47237003683995{code}
h1. Perf results:

It's obvious that we are beakring performance by using a query to retrieve the current user session and the results is confirming it.

An attempt have been made to provide a cache strategy to cache the session affinity using strategy:
 * if session not in affinity cache > do a query on {*}context-session{*}*
 * if session not found by query -> session not exists

As soon as a session is saved/retrieved we will cache it in the affinity cache:

using: *sessionId* and {*}indexName{*}.
h1. Query issue that is also painful:

the indexing time done by ES is painful again, if 2 requests come really fast, the first one will create the session, but the second one will not be able to see the newly created session.

test in a .sh file:
{code:java}
curl "http://localhost:8181/context.json?sessionId=test1"
curl "http://localhost:8181/context.json?sessionId=test1"
curl "http://localhost:8181/context.json?sessionId=test1" {code}
and we can see that the issue is happening.
h2. Solutions:
 * Always load on last session index ?
 ** Not perfect in case a rollover happen we will not find all current live sessions.
 ** Also we would need to maintain in-memory the variable of the last available index and update as soon as a rollover is triggered. could create race conditions on sessions retrieval.
 * Have a in-memory cache/map that is storing sessionId/indexName affinity
 ** in case of server restart query will be done first time session will come back
 ** easy to maintain with a time-to-live time-to-idle to have an auto clear map/cache of this data. So that the cache/map only contains currently used sessions.

None of this solutions are perfect, but it's the only possible solutions we found to address the issue of sessions being potentially split in multiple indices and the need to being able to retrieve them by their ids.

> Session/Event index rollover: measure performance of session being loaded by query
> ----------------------------------------------------------------------------------
>
>                 Key: UNOMI-725
>                 URL: https://issues.apache.org/jira/browse/UNOMI-725
>             Project: Apache Unomi
>          Issue Type: Task
>    Affects Versions: unomi-2.2.0
>            Reporter: Kevan Jahanshahi
>            Assignee: Kevan Jahanshahi
>            Priority: Major
>             Fix For: unomi-2.2.0
>
>
> Since: UNOMI-724 the sessions are now loaded using query.
> This ticket need to:
>  * measure performance difference with previous implem that was doing a direct load instead of query
>  * Take action in case of performance drawback
>  ** Consider in memory cache system to cache the sessionID/index affinity
>  ** Consider doing multiple get on all indices until session is found if it's more efficient than doing a query.
> Personally I do prefer a caching here if it's require and performance show a real difference.
> h2. To be done:
>  * new perfomance test simple using jmeter to create lot of sessions and measure differences on contextrequest system stability thresholds.
>  * Depeding on results: create dedicated ticket for next steps:
>  ** cache implem for affinity
>  ** or other logic to load sessions if better idea is found.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)