You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by "Kevan Jahanshahi (Jira)" <ji...@apache.org> on 2023/02/01 13:48:00 UTC
[jira] [Updated] (UNOMI-736) Metadata Items: indices reduction PoC cleanup
[ https://issues.apache.org/jira/browse/UNOMI-736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kevan Jahanshahi updated UNOMI-736:
-----------------------------------
Description:
Recently we worked on reducing the number of ElasticSearch indices used in order to reduce ElasticSearch cost and optimize memory cost mostly due to a lot of small indices.
So the idea is to store all the Unomi items in the same index as much as possible.
Here is the PoC PR: [https://github.com/apache/unomi/pull/571]
What remains to do in the PoC:
* make the itemType/index name map configurable instead of currently hardcoded in the Java Persistence Service ({*}to be discussed. could be dangerous to change the conf during server runtime{*})
* Fix groovy action id conflict:
** currently the rule resolution system is trying to load a *ActionType* item with the id of a groovy action, since they are stored in the same index now, it found a groovy Action and the deserialzation is failing, we should find a way to avoid such resolution.
* Fix tests if necessary (current test scope should be green as everything should continue work like before.)
Not in current ticket:
* handle migration
was:
Recently we worked on reducing the number of ElasticSearch indices used in order to reduce ElasticSearch cost and optimize memory cost mostly due to a lot of small indices.
So the idea is to store all the Unomi items in the same index as much as possible.
Here is the PoC PR: [https://github.com/apache/unomi/pull/571]
What remains to do in the PoC:
* make the itemType/index name map configurable instead of currently hardcoded in the Java Persistence Service
* Fix groovy action id conflict:
** currently the rule resolution system is trying to load a *ActionType* item with the id of a groovy action, since they are stored in the same index now, it found a groovy Action and the deserialzation is failing, we should find a way to avoid such resolution.
* Fix tests if necessary (current test scope should be green as everything should continue work like before.)
Not in current ticket:
* handle migration
> Metadata Items: indices reduction PoC cleanup
> ---------------------------------------------
>
> Key: UNOMI-736
> URL: https://issues.apache.org/jira/browse/UNOMI-736
> Project: Apache Unomi
> Issue Type: Task
> Affects Versions: unomi-2.1.0
> Reporter: Kevan Jahanshahi
> Priority: Major
> Fix For: unomi-2.2.0
>
>
> Recently we worked on reducing the number of ElasticSearch indices used in order to reduce ElasticSearch cost and optimize memory cost mostly due to a lot of small indices.
> So the idea is to store all the Unomi items in the same index as much as possible.
> Here is the PoC PR: [https://github.com/apache/unomi/pull/571]
> What remains to do in the PoC:
> * make the itemType/index name map configurable instead of currently hardcoded in the Java Persistence Service ({*}to be discussed. could be dangerous to change the conf during server runtime{*})
> * Fix groovy action id conflict:
> ** currently the rule resolution system is trying to load a *ActionType* item with the id of a groovy action, since they are stored in the same index now, it found a groovy Action and the deserialzation is failing, we should find a way to avoid such resolution.
> * Fix tests if necessary (current test scope should be green as everything should continue work like before.)
> Not in current ticket:
> * handle migration
--
This message was sent by Atlassian Jira
(v8.20.10#820010)