You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by "Kevan Jahanshahi (Jira)" <ji...@apache.org> on 2021/07/23 10:06:00 UTC
[jira] [Updated] (UNOMI-471) sessionReassigned event causes mapping
explosion in elastic
[ https://issues.apache.org/jira/browse/UNOMI-471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevan Jahanshahi updated UNOMI-471:
-----------------------------------
Fix Version/s: (was: 1.5.7)
1.6.0
> sessionReassigned event causes mapping explosion in elastic
> -----------------------------------------------------------
>
> Key: UNOMI-471
> URL: https://issues.apache.org/jira/browse/UNOMI-471
> Project: Apache Unomi
> Issue Type: Bug
> Components: core
> Affects Versions: 1.5.4
> Reporter: Ben Schiefer
> Priority: Critical
> Fix For: 1.6.0
>
>
> In the MergeProfilesOnPropertyAction.java ([https://github.com/apache/unomi/blob/2baf16da141679b3f4fd12307b840982a4740592/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java#L122)]
>
> There is a sessionReassigned event raised with the current event as the "source" and the currentSession as the "target" arguments. This is problematic due to the fact that these entities (as well as their nested child profile/session/properties) are then ALL mapped as new fields in elastic.
>
> If you can imagine a profile with 50 text attributes (with 2 mappings per attribute, text and keyword) and 5 different consents (with 8 different mappings per consent) - this would add an additional 140 mappings in the current event index.
>
> Now add the fact that the current event is sent as the source argument in the sessionReassigned event, you will have every source/target/property of the event that triggered the MergeProfilesOnProperty action additionally indexed as part of the "source" property of the sessionReassigned event, easily leading to hundreds more mappings in elasticsearch, and quickly hitting the default mapping fields limit of 1000 set by elasticsearch.
>
> Recommendation:
> a: Only send the event.itemId and event.itemType as the "source", and only send the currentSession.itemId and currentSession.itemType as the "target" when creating the sessionReassigned event.
> OR
> b: specify the default mapping on the event index ahead of time, including only properties that are necessary in source and target, and making those entities dynamic: false (this solution seems less optimal than solution a)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)