[jira] [Commented] (UNOMI-759) Perfomance issue on export profile

["Kevan Jahanshahi (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/UNOMI-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17706300#comment-17706300 ] 

Kevan Jahanshahi commented on UNOMI-759:
----------------------------------------

h1. Other performances concerned:

1 - All profiles are loaded in RAM during export: 
{code:java}
return persistenceService.query("segments", segment,null, Profile.class); {code}
2 - To much transformation:
 * ES contains JSON.
 * Unomi persistence service transform to Java Profile during query.
 * Export reader router will transform Java Profile to JSON Profile -> +Seem's to be a useless transformation+
 * Export producer route will transform JSON Profile to Java Profile -> +Seem's to be a useless transformation+
 * Export producer route will transform Java Profile to CSV line

> Perfomance issue on export profile
> ----------------------------------
>
>                 Key: UNOMI-759
>                 URL: https://issues.apache.org/jira/browse/UNOMI-759
>             Project: Apache Unomi
>          Issue Type: Bug
>            Reporter: David Griffon
>            Priority: Major
>
> Currently the profile export do a query for each profile exported to get the property types:
> https://github.com/apache/unomi/blob/9fbdf43abcdcb3aa5ebae4612f96d698d3e0a28f/extensions/router/router-service/src/main/java/org/apache/unomi/router/services/ProfileExportServiceImpl.java#L62
> {code}
> public String convertProfileToCSVLine(Profile profile, ExportConfiguration exportConfiguration) {
>         Collection<PropertyType> propertiesDef = persistenceService.query("target", "profiles", null, PropertyType.class);
>         Map<String, String> mapping = (Map<String, String>) exportConfiguration.getProperty("mapping");
>         String lineToWrite = "";
>    ...
> {code}
> This is used to check if the property is multivalued or not. 
> 2 ways of improvement are possible:
> 1 / Store the information in the export configuration, it currently contains a mapping (the list of properties to export), we could add details of it at the same level. To keep the compatibility with previous configuration, we can fallback on checking on the property type if the information is missing. 
> 1 / Use a cache for property types (as we do for rules or other unomi Items) 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)