You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@unomi.apache.org by sh...@apache.org on 2020/06/05 15:41:03 UTC
svn commit: r1878523 [2/3] - in /unomi/website/manual: 1_5_x/index.html
1_5_x/migrations/ latest/index.html latest/migrations/
Modified: unomi/website/manual/1_5_x/index.html
URL: http://svn.apache.org/viewvc/unomi/website/manual/1_5_x/index.html?rev=1878523&r1=1878522&r2=1878523&view=diff
==============================================================================
--- unomi/website/manual/1_5_x/index.html (original)
+++ unomi/website/manual/1_5_x/index.html Fri Jun 5 15:41:03 2020
@@ -61,328 +61,338 @@
<li><a href="#_tracking_form_submissions">2.4.4. Tracking form submissions</a></li>
</ul>
</li>
-<li><a href="#_configuration">2.5. Configuration</a>
+</ul>
+</li>
+<li><a href="#_configuration">3. Configuration</a>
+<ul class="sectlevel2">
+<li><a href="#_centralized_configuration">3.1. Centralized configuration</a></li>
+<li><a href="#_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">3.2. Changing the default configuration using environment variables (i.e. Docker configuration)</a></li>
+<li><a href="#_changing_the_default_configuration_using_property_files">3.3. Changing the default configuration using property files</a></li>
+<li><a href="#_secured_events_configuration">3.4. Secured events configuration</a></li>
+<li><a href="#_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5. Installing the MaxMind GeoIPLite2 IP lookup database</a></li>
+<li><a href="#_installing_geonames_database">3.6. Installing Geonames database</a></li>
+<li><a href="#_rest_api_security">3.7. REST API Security</a></li>
+<li><a href="#_scripting_security">3.8. Scripting security</a></li>
+<li><a href="#_automatic_profile_merging">3.9. Automatic profile merging</a></li>
+<li><a href="#_securing_a_production_environment">3.10. Securing a production environment</a></li>
+<li><a href="#_integrating_with_an_apache_http_web_server">3.11. Integrating with an Apache HTTP web server</a></li>
+<li><a href="#_changing_the_default_tracking_location">3.12. Changing the default tracking location</a></li>
+<li><a href="#_apache_karaf_ssh_console">3.13. Apache Karaf SSH Console</a></li>
+<li><a href="#_elasticsearch_authentication_and_security">3.14. ElasticSearch authentication and security</a>
<ul class="sectlevel3">
-<li><a href="#_centralized_configuration">2.5.1. Centralized configuration</a></li>
-<li><a href="#_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">2.5.2. Changing the default configuration using environment variables (i.e. Docker configuration)</a></li>
-<li><a href="#_changing_the_default_configuration_using_property_files">2.5.3. Changing the default configuration using property files</a></li>
-<li><a href="#_secured_events_configuration">2.5.4. Secured events configuration</a></li>
-<li><a href="#_installing_the_maxmind_geoiplite2_ip_lookup_database">2.5.5. Installing the MaxMind GeoIPLite2 IP lookup database</a></li>
-<li><a href="#_installing_geonames_database">2.5.6. Installing Geonames database</a></li>
-<li><a href="#_rest_api_security">2.5.7. REST API Security</a></li>
-<li><a href="#_scripting_security">2.5.8. Scripting security</a></li>
-<li><a href="#_automatic_profile_merging">2.5.9. Automatic profile merging</a></li>
-<li><a href="#_securing_a_production_environment">2.5.10. Securing a production environment</a></li>
-<li><a href="#_integrating_with_an_apache_http_web_server">2.5.11. Integrating with an Apache HTTP web server</a></li>
-<li><a href="#_changing_the_default_tracking_location">2.5.12. Changing the default tracking location</a></li>
-<li><a href="#_apache_karaf_ssh_console">2.5.13. Apache Karaf SSH Console</a></li>
-<li><a href="#_elasticsearch_authentication_and_security">2.5.14. ElasticSearch authentication and security</a></li>
+<li><a href="#_user_authentication">3.14.1. User authentication !</a></li>
+<li><a href="#_ssl_communication">3.14.2. SSL communication</a></li>
+</ul>
+</li>
</ul>
</li>
-<li><a href="#_useful_apache_unomi_urls">2.6. Useful Apache Unomi URLs</a></li>
-<li><a href="#_how_profile_tracking_works">2.7. How profile tracking works</a>
+<li><a href="#_migrations">4. Migrations</a>
+<ul class="sectlevel2">
+<li><a href="#_from_version_1_4_to_1_5">4.1. From version 1.4 to 1.5</a>
<ul class="sectlevel3">
-<li><a href="#_steps">2.7.1. Steps</a></li>
+<li><a href="#_data_model_and_elasticsearch_7">4.1.1. Data model and ElasticSearch 7</a></li>
+<li><a href="#_api_changes">4.1.2. API changes</a></li>
+<li><a href="#_migration_steps">4.1.3. Migration steps</a></li>
</ul>
</li>
-<li><a href="#_context_request_flow">2.8. Context Request Flow</a></li>
</ul>
</li>
-<li><a href="#_queries_and_aggregations">3. Queries and aggregations</a>
+<li><a href="#_queries_and_aggregations">5. Queries and aggregations</a>
<ul class="sectlevel2">
-<li><a href="#_query_counts">3.1. Query counts</a></li>
-<li><a href="#_metrics">3.2. Metrics</a></li>
-<li><a href="#_aggregations">3.3. Aggregations</a>
+<li><a href="#_query_counts">5.1. Query counts</a></li>
+<li><a href="#_metrics">5.2. Metrics</a></li>
+<li><a href="#_aggregations">5.3. Aggregations</a>
<ul class="sectlevel3">
-<li><a href="#_aggregation_types">3.3.1. Aggregation types</a></li>
+<li><a href="#_aggregation_types">5.3.1. Aggregation types</a></li>
</ul>
</li>
</ul>
</li>
-<li><a href="#_profile_import_export">4. Profile import & export</a>
+<li><a href="#_profile_import_export">6. Profile import & export</a>
<ul class="sectlevel2">
-<li><a href="#_importing_profiles">4.1. Importing profiles</a>
+<li><a href="#_importing_profiles">6.1. Importing profiles</a>
<ul class="sectlevel3">
-<li><a href="#_import_api">4.1.1. Import API</a></li>
+<li><a href="#_import_api">6.1.1. Import API</a></li>
</ul>
</li>
-<li><a href="#_exporting_profiles">4.2. Exporting profiles</a>
+<li><a href="#_exporting_profiles">6.2. Exporting profiles</a>
<ul class="sectlevel3">
-<li><a href="#_export_api">4.2.1. Export API</a></li>
+<li><a href="#_export_api">6.2.1. Export API</a></li>
</ul>
</li>
-<li><a href="#_configuration_in_details">4.3. Configuration in details</a></li>
+<li><a href="#_configuration_in_details">6.3. Configuration in details</a></li>
</ul>
</li>
-<li><a href="#_consent_management">5. Consent management</a>
+<li><a href="#_consent_management">7. Consent management</a>
<ul class="sectlevel2">
-<li><a href="#_consent_api">5.1. Consent API</a>
+<li><a href="#_consent_api">7.1. Consent API</a>
<ul class="sectlevel3">
-<li><a href="#_profiles_with_consents">5.1.1. Profiles with consents</a></li>
-<li><a href="#_consent_type_definitions">5.1.2. Consent type definitions</a></li>
-<li><a href="#_creating_update_a_visitor_consent">5.1.3. Creating / update a visitor consent</a></li>
-<li><a href="#_how_it_works_internally">5.1.4. How it works (internally)</a></li>
+<li><a href="#_profiles_with_consents">7.1.1. Profiles with consents</a></li>
+<li><a href="#_consent_type_definitions">7.1.2. Consent type definitions</a></li>
+<li><a href="#_creating_update_a_visitor_consent">7.1.3. Creating / update a visitor consent</a></li>
+<li><a href="#_how_it_works_internally">7.1.4. How it works (internally)</a></li>
</ul>
</li>
</ul>
</li>
-<li><a href="#_privacy_management">6. Privacy management</a>
+<li><a href="#_privacy_management">8. Privacy management</a>
<ul class="sectlevel2">
-<li><a href="#_setting_up_access_to_the_privacy_endpoint">6.1. Setting up access to the privacy endpoint</a></li>
-<li><a href="#_anonymizing_a_profile">6.2. Anonymizing a profile</a></li>
-<li><a href="#_downloading_profile_data">6.3. Downloading profile data</a></li>
-<li><a href="#_deleting_a_profile">6.4. Deleting a profile</a></li>
-<li><a href="#_related">6.5. Related</a></li>
+<li><a href="#_setting_up_access_to_the_privacy_endpoint">8.1. Setting up access to the privacy endpoint</a></li>
+<li><a href="#_anonymizing_a_profile">8.2. Anonymizing a profile</a></li>
+<li><a href="#_downloading_profile_data">8.3. Downloading profile data</a></li>
+<li><a href="#_deleting_a_profile">8.4. Deleting a profile</a></li>
+<li><a href="#_related">8.5. Related</a></li>
</ul>
</li>
-<li><a href="#_cluster_setup">7. Cluster setup</a>
+<li><a href="#_cluster_setup">9. Cluster setup</a>
<ul class="sectlevel2">
-<li><a href="#_cluster_setup_2">7.1. Cluster setup</a></li>
+<li><a href="#_cluster_setup_2">9.1. Cluster setup</a></li>
</ul>
</li>
-<li><a href="#_reference">8. Reference</a>
+<li><a href="#_reference">10. Reference</a>
<ul class="sectlevel2">
-<li><a href="#_data_model_overview">8.1. Data Model Overview</a></li>
-<li><a href="#_scope">8.2. Scope</a>
+<li><a href="#_useful_apache_unomi_urls">10.1. Useful Apache Unomi URLs</a></li>
+<li><a href="#_how_profile_tracking_works">10.2. How profile tracking works</a>
<ul class="sectlevel3">
-<li><a href="#_example">8.2.1. Example</a></li>
+<li><a href="#_steps">10.2.1. Steps</a></li>
</ul>
</li>
-<li><a href="#_item">8.3. Item</a>
+<li><a href="#_context_request_flow">10.3. Context Request Flow</a></li>
+<li><a href="#_data_model_overview">10.4. Data Model Overview</a></li>
+<li><a href="#_scope">10.5. Scope</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition">8.3.1. Structure definition</a></li>
+<li><a href="#_example">10.5.1. Example</a></li>
</ul>
</li>
-<li><a href="#_metadata">8.4. Metadata</a>
+<li><a href="#_item">10.6. Item</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_2">8.4.1. Structure definition</a></li>
-<li><a href="#_example_2">8.4.2. Example</a></li>
+<li><a href="#_structure_definition">10.6.1. Structure definition</a></li>
</ul>
</li>
-<li><a href="#_metadataitem">8.5. MetadataItem</a>
+<li><a href="#_metadata">10.7. Metadata</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_3">8.5.1. Structure definition</a></li>
-<li><a href="#_example_3">8.5.2. Example</a></li>
+<li><a href="#_structure_definition_2">10.7.1. Structure definition</a></li>
+<li><a href="#_example_2">10.7.2. Example</a></li>
</ul>
</li>
-<li><a href="#_event">8.6. Event</a>
+<li><a href="#_metadataitem">10.8. MetadataItem</a>
<ul class="sectlevel3">
-<li><a href="#_fields">8.6.1. Fields</a></li>
-<li><a href="#_event_types">8.6.2. Event types</a></li>
+<li><a href="#_structure_definition_3">10.8.1. Structure definition</a></li>
+<li><a href="#_example_3">10.8.2. Example</a></li>
</ul>
</li>
-<li><a href="#_profile">8.7. Profile</a>
+<li><a href="#_event">10.9. Event</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_4">8.7.1. Structure definition</a></li>
-<li><a href="#_example_4">8.7.2. Example</a></li>
+<li><a href="#_fields">10.9.1. Fields</a></li>
+<li><a href="#_event_types">10.9.2. Event types</a></li>
</ul>
</li>
-<li><a href="#_persona">8.8. Persona</a>
+<li><a href="#_profile">10.10. Profile</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_5">8.8.1. Structure definition</a></li>
-<li><a href="#_example_5">8.8.2. Example</a></li>
+<li><a href="#_structure_definition_4">10.10.1. Structure definition</a></li>
+<li><a href="#_example_4">10.10.2. Example</a></li>
</ul>
</li>
-<li><a href="#_consent">8.9. Consent</a>
+<li><a href="#_persona">10.11. Persona</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_6">8.9.1. Structure definition</a></li>
-<li><a href="#_example_6">8.9.2. Example</a></li>
+<li><a href="#_structure_definition_5">10.11.1. Structure definition</a></li>
+<li><a href="#_example_5">10.11.2. Example</a></li>
</ul>
</li>
-<li><a href="#_session">8.10. Session</a>
+<li><a href="#_consent">10.12. Consent</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_7">8.10.1. Structure definition</a></li>
-<li><a href="#_example_7">8.10.2. Example</a></li>
+<li><a href="#_structure_definition_6">10.12.1. Structure definition</a></li>
+<li><a href="#_example_6">10.12.2. Example</a></li>
</ul>
</li>
-<li><a href="#_segment">8.11. Segment</a>
+<li><a href="#_session">10.13. Session</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_8">8.11.1. Structure definition</a></li>
-<li><a href="#_example_8">8.11.2. Example</a></li>
+<li><a href="#_structure_definition_7">10.13.1. Structure definition</a></li>
+<li><a href="#_example_7">10.13.2. Example</a></li>
</ul>
</li>
-<li><a href="#_condition">8.12. Condition</a>
+<li><a href="#_segment">10.14. Segment</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_9">8.12.1. Structure definition</a></li>
-<li><a href="#_example_9">8.12.2. Example</a></li>
+<li><a href="#_structure_definition_8">10.14.1. Structure definition</a></li>
+<li><a href="#_example_8">10.14.2. Example</a></li>
</ul>
</li>
-<li><a href="#_rule">8.13. Rule</a>
+<li><a href="#_condition">10.15. Condition</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_10">8.13.1. Structure definition</a></li>
-<li><a href="#_example_10">8.13.2. Example</a></li>
+<li><a href="#_structure_definition_9">10.15.1. Structure definition</a></li>
+<li><a href="#_example_9">10.15.2. Example</a></li>
</ul>
</li>
-<li><a href="#_action">8.14. Action</a>
+<li><a href="#_rule">10.16. Rule</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_11">8.14.1. Structure definition</a></li>
-<li><a href="#_example_11">8.14.2. Example</a></li>
+<li><a href="#_structure_definition_10">10.16.1. Structure definition</a></li>
+<li><a href="#_example_10">10.16.2. Example</a></li>
</ul>
</li>
-<li><a href="#_list">8.15. List</a>
+<li><a href="#_action">10.17. Action</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_12">8.15.1. Structure definition</a></li>
-<li><a href="#_example_12">8.15.2. Example</a></li>
+<li><a href="#_structure_definition_11">10.17.1. Structure definition</a></li>
+<li><a href="#_example_11">10.17.2. Example</a></li>
</ul>
</li>
-<li><a href="#_goal">8.16. Goal</a>
+<li><a href="#_list">10.18. List</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_13">8.16.1. Structure definition</a></li>
-<li><a href="#_example_13">8.16.2. Example</a></li>
+<li><a href="#_structure_definition_12">10.18.1. Structure definition</a></li>
+<li><a href="#_example_12">10.18.2. Example</a></li>
</ul>
</li>
-<li><a href="#_campaign">8.17. Campaign</a>
+<li><a href="#_goal">10.19. Goal</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_14">8.17.1. Structure definition</a></li>
-<li><a href="#_example_14">8.17.2. Example</a></li>
+<li><a href="#_structure_definition_13">10.19.1. Structure definition</a></li>
+<li><a href="#_example_13">10.19.2. Example</a></li>
</ul>
</li>
-<li><a href="#_scoring_plan">8.18. Scoring plan</a>
+<li><a href="#_campaign">10.20. Campaign</a>
<ul class="sectlevel3">
-<li><a href="#_structure_definition_15">8.18.1. Structure definition</a></li>
-<li><a href="#_example_15">8.18.2. Example</a></li>
+<li><a href="#_structure_definition_14">10.20.1. Structure definition</a></li>
+<li><a href="#_example_14">10.20.2. Example</a></li>
</ul>
</li>
-<li><a href="#_data_model_changes_for_apache_unomi_1_5_0">8.19. Data Model changes for Apache Unomi 1.5.0</a>
+<li><a href="#_scoring_plan">10.21. Scoring plan</a>
<ul class="sectlevel3">
-<li><a href="#_data_model_and_elasticsearch_7">8.19.1. Data model and ElasticSearch 7</a></li>
-<li><a href="#_api_changes">8.19.2. API changes</a></li>
+<li><a href="#_structure_definition_15">10.21.1. Structure definition</a></li>
+<li><a href="#_example_15">10.21.2. Example</a></li>
</ul>
</li>
-<li><a href="#_built_in_event_types">8.20. Built-in Event types</a>
+<li><a href="#_built_in_event_types">10.22. Built-in Event types</a>
<ul class="sectlevel3">
-<li><a href="#_login_event_type">8.20.1. Login event type</a></li>
-<li><a href="#_view_event_type">8.20.2. View event type</a></li>
-<li><a href="#_form_event_type">8.20.3. Form event type</a></li>
-<li><a href="#_update_properties_event_type">8.20.4. Update properties event type</a></li>
-<li><a href="#_identify_event_type">8.20.5. Identify event type</a></li>
-<li><a href="#_session_created_event_type">8.20.6. Session created event type</a></li>
-<li><a href="#_goal_event_type">8.20.7. Goal event type</a></li>
-<li><a href="#_modify_consent_event_type">8.20.8. Modify consent event type</a></li>
+<li><a href="#_login_event_type">10.22.1. Login event type</a></li>
+<li><a href="#_view_event_type">10.22.2. View event type</a></li>
+<li><a href="#_form_event_type">10.22.3. Form event type</a></li>
+<li><a href="#_update_properties_event_type">10.22.4. Update properties event type</a></li>
+<li><a href="#_identify_event_type">10.22.5. Identify event type</a></li>
+<li><a href="#_session_created_event_type">10.22.6. Session created event type</a></li>
+<li><a href="#_goal_event_type">10.22.7. Goal event type</a></li>
+<li><a href="#_modify_consent_event_type">10.22.8. Modify consent event type</a></li>
</ul>
</li>
-<li><a href="#_built_in_condition_types">8.21. Built-in condition types</a>
+<li><a href="#_built_in_condition_types">10.23. Built-in condition types</a>
<ul class="sectlevel3">
-<li><a href="#_existing_condition_type_descriptors">8.21.1. Existing condition type descriptors</a></li>
+<li><a href="#_existing_condition_type_descriptors">10.23.1. Existing condition type descriptors</a></li>
</ul>
</li>
-<li><a href="#_built_in_action_types">8.22. Built-in action types</a>
+<li><a href="#_built_in_action_types">10.24. Built-in action types</a>
<ul class="sectlevel3">
-<li><a href="#_existing_action_types_descriptors">8.22.1. Existing action types descriptors</a></li>
+<li><a href="#_existing_action_types_descriptors">10.24.1. Existing action types descriptors</a></li>
</ul>
</li>
-<li><a href="#_updating_events_using_the_context_servlet">8.23. Updating Events Using the Context Servlet</a>
+<li><a href="#_updating_events_using_the_context_servlet">10.25. Updating Events Using the Context Servlet</a>
<ul class="sectlevel3">
-<li><a href="#_solution">8.23.1. Solution</a></li>
-<li><a href="#_defining_rules">8.23.2. Defining Rules</a></li>
+<li><a href="#_solution">10.25.1. Solution</a></li>
+<li><a href="#_defining_rules">10.25.2. Defining Rules</a></li>
</ul>
</li>
</ul>
</li>
-<li><a href="#_integration_samples">9. Integration samples</a>
+<li><a href="#_integration_samples">11. Integration samples</a>
<ul class="sectlevel2">
-<li><a href="#_samples">9.1. Samples</a></li>
-<li><a href="#_login_sample">9.2. Login sample</a>
+<li><a href="#_samples">11.1. Samples</a></li>
+<li><a href="#_login_sample">11.2. Login sample</a>
<ul class="sectlevel3">
-<li><a href="#_warning">9.2.1. Warning !</a></li>
-<li><a href="#_installing_the_samples">9.2.2. Installing the samples</a></li>
+<li><a href="#_warning">11.2.1. Warning !</a></li>
+<li><a href="#_installing_the_samples">11.2.2. Installing the samples</a></li>
</ul>
</li>
-<li><a href="#_twitter_sample">9.3. Twitter sample</a>
+<li><a href="#_twitter_sample">11.3. Twitter sample</a>
<ul class="sectlevel3">
-<li><a href="#_overview">9.3.1. Overview</a></li>
-<li><a href="#_interacting_with_the_context_server">9.3.2. Interacting with the context server</a></li>
-<li><a href="#_retrieving_context_information_from_unomi_using_the_context_servlet">9.3.3. Retrieving context information from Unomi using the context servlet</a></li>
+<li><a href="#_overview">11.3.1. Overview</a></li>
+<li><a href="#_interacting_with_the_context_server">11.3.2. Interacting with the context server</a></li>
+<li><a href="#_retrieving_context_information_from_unomi_using_the_context_servlet">11.3.3. Retrieving context information from Unomi using the context servlet</a></li>
</ul>
</li>
-<li><a href="#_example_24">9.4. Example</a>
+<li><a href="#_example_24">11.4. Example</a>
<ul class="sectlevel3">
-<li><a href="#_html_page">9.4.1. HTML page</a></li>
-<li><a href="#_javascript">9.4.2. Javascript</a></li>
+<li><a href="#_html_page">11.4.1. HTML page</a></li>
+<li><a href="#_javascript">11.4.2. Javascript</a></li>
</ul>
</li>
-<li><a href="#_conclusion">9.5. Conclusion</a></li>
-<li><a href="#_annex">9.6. Annex</a></li>
-<li><a href="#_weather_update_sample">9.7. Weather update sample</a></li>
+<li><a href="#_conclusion">11.5. Conclusion</a></li>
+<li><a href="#_annex">11.6. Annex</a></li>
+<li><a href="#_weather_update_sample">11.7. Weather update sample</a></li>
</ul>
</li>
-<li><a href="#_connectors">10. Connectors</a>
+<li><a href="#_connectors">12. Connectors</a>
<ul class="sectlevel2">
-<li><a href="#_connectors_2">10.1. Connectors</a>
+<li><a href="#_connectors_2">12.1. Connectors</a>
<ul class="sectlevel3">
-<li><a href="#_call_for_contributors">10.1.1. Call for contributors</a></li>
+<li><a href="#_call_for_contributors">12.1.1. Call for contributors</a></li>
</ul>
</li>
-<li><a href="#_salesforce_connector">10.2. Salesforce Connector</a>
+<li><a href="#_salesforce_connector">12.2. Salesforce Connector</a>
<ul class="sectlevel3">
-<li><a href="#_getting_started_2">10.2.1. Getting started</a></li>
-<li><a href="#_properties">10.2.2. Properties</a></li>
-<li><a href="#_hot_deploying_updates_to_the_salesforce_connector_for_developers">10.2.3. Hot-deploying updates to the Salesforce connector (for developers)</a></li>
-<li><a href="#_using_the_salesforce_workbench_for_testing_rest_api">10.2.4. Using the Salesforce Workbench for testing REST API</a></li>
-<li><a href="#_setting_up_streaming_push_queries">10.2.5. Setting up Streaming Push queries</a></li>
-<li><a href="#_executing_the_unit_tests">10.2.6. Executing the unit tests</a></li>
+<li><a href="#_getting_started_2">12.2.1. Getting started</a></li>
+<li><a href="#_properties">12.2.2. Properties</a></li>
+<li><a href="#_hot_deploying_updates_to_the_salesforce_connector_for_developers">12.2.3. Hot-deploying updates to the Salesforce connector (for developers)</a></li>
+<li><a href="#_using_the_salesforce_workbench_for_testing_rest_api">12.2.4. Using the Salesforce Workbench for testing REST API</a></li>
+<li><a href="#_setting_up_streaming_push_queries">12.2.5. Setting up Streaming Push queries</a></li>
+<li><a href="#_executing_the_unit_tests">12.2.6. Executing the unit tests</a></li>
</ul>
</li>
-<li><a href="#_mailchimp_connector">10.3. MailChimp Connector</a>
+<li><a href="#_mailchimp_connector">12.3. MailChimp Connector</a>
<ul class="sectlevel3">
-<li><a href="#_getting_started_3">10.3.1. Getting started</a></li>
+<li><a href="#_getting_started_3">12.3.1. Getting started</a></li>
</ul>
</li>
</ul>
</li>
-<li><a href="#_developers">11. Developers</a>
+<li><a href="#_developers">13. Developers</a>
<ul class="sectlevel2">
-<li><a href="#_building">11.1. Building</a>
+<li><a href="#_building">13.1. Building</a>
<ul class="sectlevel3">
-<li><a href="#_initial_setup">11.1.1. Initial Setup</a></li>
-<li><a href="#_building_2">11.1.2. Building</a></li>
-<li><a href="#_installing_an_elasticsearch_server">11.1.3. Installing an ElasticSearch server</a></li>
-<li><a href="#_deploying_the_generated_binary_package">11.1.4. Deploying the generated binary package</a></li>
-<li><a href="#_deploying_into_an_existing_karaf_server">11.1.5. Deploying into an existing Karaf server</a></li>
-<li><a href="#_jdk_selection_on_mac_os_x">11.1.6. JDK Selection on Mac OS X</a></li>
-<li><a href="#_running_the_integration_tests">11.1.7. Running the integration tests</a></li>
-<li><a href="#_running_the_performance_tests">11.1.8. Running the performance tests</a></li>
-<li><a href="#_testing_with_an_example_page">11.1.9. Testing with an example page</a></li>
+<li><a href="#_initial_setup">13.1.1. Initial Setup</a></li>
+<li><a href="#_building_2">13.1.2. Building</a></li>
+<li><a href="#_installing_an_elasticsearch_server">13.1.3. Installing an ElasticSearch server</a></li>
+<li><a href="#_deploying_the_generated_binary_package">13.1.4. Deploying the generated binary package</a></li>
+<li><a href="#_deploying_into_an_existing_karaf_server">13.1.5. Deploying into an existing Karaf server</a></li>
+<li><a href="#_jdk_selection_on_mac_os_x">13.1.6. JDK Selection on Mac OS X</a></li>
+<li><a href="#_running_the_integration_tests">13.1.7. Running the integration tests</a></li>
+<li><a href="#_running_the_performance_tests">13.1.8. Running the performance tests</a></li>
+<li><a href="#_testing_with_an_example_page">13.1.9. Testing with an example page</a></li>
</ul>
</li>
-<li><a href="#_ssh_shell_commands">11.2. SSH Shell Commands</a>
+<li><a href="#_ssh_shell_commands">13.2. SSH Shell Commands</a>
<ul class="sectlevel3">
-<li><a href="#_using_the_shell">11.2.1. Using the shell</a></li>
-<li><a href="#_lifecycle_commands">11.2.2. Lifecycle commands</a></li>
-<li><a href="#_runtime_commands">11.2.3. Runtime commands</a></li>
+<li><a href="#_using_the_shell">13.2.1. Using the shell</a></li>
+<li><a href="#_lifecycle_commands">13.2.2. Lifecycle commands</a></li>
+<li><a href="#_runtime_commands">13.2.3. Runtime commands</a></li>
</ul>
</li>
-<li><a href="#_writing_plugins">11.3. Writing Plugins</a></li>
-<li><a href="#_types_vs_instances">11.4. Types vs. instances</a></li>
-<li><a href="#_plugin_structure">11.5. Plugin structure</a></li>
-<li><a href="#_extension_points">11.6. Extension points</a>
+<li><a href="#_writing_plugins">13.3. Writing Plugins</a></li>
+<li><a href="#_types_vs_instances">13.4. Types vs. instances</a></li>
+<li><a href="#_plugin_structure">13.5. Plugin structure</a></li>
+<li><a href="#_extension_points">13.6. Extension points</a>
<ul class="sectlevel3">
-<li><a href="#_actiontype">11.6.1. ActionType</a></li>
-<li><a href="#_conditiontype">11.6.2. ConditionType</a></li>
-<li><a href="#_persona_2">11.6.3. Persona</a></li>
-<li><a href="#_propertymergestrategytype">11.6.4. PropertyMergeStrategyType</a></li>
-<li><a href="#_propertytype">11.6.5. PropertyType</a></li>
-<li><a href="#_rule_2">11.6.6. Rule</a></li>
-<li><a href="#_scoring">11.6.7. Scoring</a></li>
-<li><a href="#_segments">11.6.8. Segments</a></li>
-<li><a href="#_tag">11.6.9. Tag</a></li>
-<li><a href="#_valuetype">11.6.10. ValueType</a></li>
+<li><a href="#_actiontype">13.6.1. ActionType</a></li>
+<li><a href="#_conditiontype">13.6.2. ConditionType</a></li>
+<li><a href="#_persona_2">13.6.3. Persona</a></li>
+<li><a href="#_propertymergestrategytype">13.6.4. PropertyMergeStrategyType</a></li>
+<li><a href="#_propertytype">13.6.5. PropertyType</a></li>
+<li><a href="#_rule_2">13.6.6. Rule</a></li>
+<li><a href="#_scoring">13.6.7. Scoring</a></li>
+<li><a href="#_segments">13.6.8. Segments</a></li>
+<li><a href="#_tag">13.6.9. Tag</a></li>
+<li><a href="#_valuetype">13.6.10. ValueType</a></li>
</ul>
</li>
-<li><a href="#_custom_plugins">11.7. Custom plugins</a>
+<li><a href="#_custom_plugins">13.7. Custom plugins</a>
<ul class="sectlevel3">
-<li><a href="#_creating_a_plugin">11.7.1. Creating a plugin</a></li>
-<li><a href="#_deployment_and_custom_definition">11.7.2. Deployment and custom definition</a></li>
-<li><a href="#_predefined_segments">11.7.3. Predefined segments</a></li>
-<li><a href="#_predefined_rules">11.7.4. Predefined rules</a></li>
-<li><a href="#_predefined_properties">11.7.5. Predefined properties</a></li>
-<li><a href="#_predefined_child_conditions">11.7.6. Predefined child conditions</a></li>
-<li><a href="#_predefined_personas">11.7.7. Predefined personas</a></li>
-<li><a href="#_custom_action_types">11.7.8. Custom action types</a></li>
-<li><a href="#_custom_condition_types">11.7.9. Custom condition types</a></li>
+<li><a href="#_creating_a_plugin">13.7.1. Creating a plugin</a></li>
+<li><a href="#_deployment_and_custom_definition">13.7.2. Deployment and custom definition</a></li>
+<li><a href="#_predefined_segments">13.7.3. Predefined segments</a></li>
+<li><a href="#_predefined_rules">13.7.4. Predefined rules</a></li>
+<li><a href="#_predefined_properties">13.7.5. Predefined properties</a></li>
+<li><a href="#_predefined_child_conditions">13.7.6. Predefined child conditions</a></li>
+<li><a href="#_predefined_personas">13.7.7. Predefined personas</a></li>
+<li><a href="#_custom_action_types">13.7.8. Custom action types</a></li>
+<li><a href="#_custom_condition_types">13.7.9. Custom condition types</a></li>
</ul>
</li>
-<li><a href="#_migration_patches">11.8. Migration patches</a></li>
+<li><a href="#_migration_patches">13.8. Migration patches</a></li>
</ul>
</li>
</ul>
@@ -1563,10 +1573,13 @@ cef09b89-6b99-4e4f-a99c-a4159a66b42b|for
</div>
</div>
</div>
+</div>
+</div>
+<div class="sect1">
+<h2 id="_configuration">3. Configuration</h2>
+<div class="sectionbody">
<div class="sect2">
-<h3 id="_configuration">2.5. Configuration</h3>
-<div class="sect3">
-<h4 id="_centralized_configuration">2.5.1. Centralized configuration</h4>
+<h3 id="_centralized_configuration">3.1. Centralized configuration</h3>
<div class="paragraph">
<p>Apache Unomi uses a centralized configuration file that contains both system properties and configuration properties.
These settings are then fed to the OSGi and other configuration files using placeholder that look something like this:</p>
@@ -1591,8 +1604,8 @@ this file directly, as an override mecha
the values in the <code>$MY_KARAF_HOME/etc/custom.system.properties</code>. See the next section for more information about that.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">2.5.2. Changing the default configuration using environment variables (i.e. Docker configuration)</h4>
+<div class="sect2">
+<h3 id="_changing_the_default_configuration_using_environment_variables_i_e_docker_configuration">3.2. Changing the default configuration using environment variables (i.e. Docker configuration)</h3>
<div class="paragraph">
<p>You might want to use environment variables to change the default system configuration, especially if you intend to run
Apache Unomi inside a Docker container. You can find the list of all the environment variable names in the following file:</p>
@@ -1609,8 +1622,8 @@ Docker Compose you can put the environme
variables you want to use.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_changing_the_default_configuration_using_property_files">2.5.3. Changing the default configuration using property files</h4>
+<div class="sect2">
+<h3 id="_changing_the_default_configuration_using_property_files">3.3. Changing the default configuration using property files</h3>
<div class="paragraph">
<p>If you want to change the default configuration using property files instead of environment variables, you can perform
any modification you want in the <code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code> file.</p>
@@ -1656,8 +1669,8 @@ org.apache.unomi.elasticsearch.addresses
</div>
</div>
</div>
-<div class="sect3">
-<h4 id="_secured_events_configuration">2.5.4. Secured events configuration</h4>
+<div class="sect2">
+<h3 id="_secured_events_configuration">3.4. Secured events configuration</h3>
<div class="paragraph">
<p>Apache Unomi secures some events by default. It comes out of the box with a default configuration that you can adjust
by using the centralized configuration file override in <code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code></p>
@@ -1748,8 +1761,8 @@ thirdparty.provider1.allowedEvents=login
</div>
</div>
</div>
-<div class="sect3">
-<h4 id="_installing_the_maxmind_geoiplite2_ip_lookup_database">2.5.5. Installing the MaxMind GeoIPLite2 IP lookup database</h4>
+<div class="sect2">
+<h3 id="_installing_the_maxmind_geoiplite2_ip_lookup_database">3.5. Installing the MaxMind GeoIPLite2 IP lookup database</h3>
<div class="paragraph">
<p>Apache Unomi requires an IP database in order to resolve IP addresses to user location.
The GeoLite2 database can be downloaded from MaxMind here :
@@ -1759,8 +1772,8 @@ The GeoLite2 database can be downloaded
<p>Simply download the GeoLite2-City.mmdb file into the "etc" directory.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_installing_geonames_database">2.5.6. Installing Geonames database</h4>
+<div class="sect2">
+<h3 id="_installing_geonames_database">3.6. Installing Geonames database</h3>
<div class="paragraph">
<p>Apache Unomi includes a geocoding service based on the geonames database ( <a href="http://www.geonames.org/">http://www.geonames.org/</a> ). It can be
used to create conditions on countries or cities.</p>
@@ -1777,8 +1790,8 @@ Otherwise, import should start at the ne
At the end, you should have about 4 million entries in the geonames index.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_rest_api_security">2.5.7. REST API Security</h4>
+<div class="sect2">
+<h3 id="_rest_api_security">3.7. REST API Security</h3>
<div class="paragraph">
<p>The Apache Unomi Context Server REST API is protected using JAAS authentication and using Basic or Digest HTTP auth.
By default, the login/password for the REST API full administrative access is "karaf/karaf".</p>
@@ -1810,8 +1823,8 @@ org.ops4j.pax.web.ssl.keypassword=${env:
<code>$MY_KARAF_HOME/etc/unomi.custom.system.properties</code> file</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_scripting_security">2.5.8. Scripting security</h4>
+<div class="sect2">
+<h3 id="_scripting_security">3.8. Scripting security</h3>
<div class="paragraph">
<p>By default, scripting (using in conditions, segments and rules) is controlled by a custom classloader that is quite
restrictive and using a white-list/black list system. It is controlled through the following property in the
@@ -1832,8 +1845,8 @@ restrictive configuration.</p>
directly from the context.json personalization conditions and therefore should be kept minimal.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_automatic_profile_merging">2.5.9. Automatic profile merging</h4>
+<div class="sect2">
+<h3 id="_automatic_profile_merging">3.9. Automatic profile merging</h3>
<div class="paragraph">
<p>Apache Unomi is capable of merging profiles based on a common property value. In order to use this, you must
add the MergeProfileOnPropertyAction to a rule (such as a login rule for example), and configure it with the name
@@ -1851,8 +1864,8 @@ will use the merged profile.</p>
Upon sending one of the events, all matching profiles will be merged.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_securing_a_production_environment">2.5.10. Securing a production environment</h4>
+<div class="sect2">
+<h3 id="_securing_a_production_environment">3.10. Securing a production environment</h3>
<div class="paragraph">
<p>Before going live with a project, you should <em>absolutely</em> read the following section that will help you setup a proper
secure environment for running your context server.</p>
@@ -1961,8 +1974,8 @@ node-to-node communication : 9200 (Elast
a proxy, and use it to filter any communication.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_integrating_with_an_apache_http_web_server">2.5.11. Integrating with an Apache HTTP web server</h4>
+<div class="sect2">
+<h3 id="_integrating_with_an_apache_http_web_server">3.11. Integrating with an Apache HTTP web server</h3>
<div class="paragraph">
<p>If you want to setup an Apache HTTP web server in from of Apache Unomi, here is an example configuration using
mod_proxy.</p>
@@ -2059,8 +2072,8 @@ ProxyPassReverse / http://localhost:8181
</div>
</div>
</div>
-<div class="sect3">
-<h4 id="_changing_the_default_tracking_location">2.5.12. Changing the default tracking location</h4>
+<div class="sect2">
+<h3 id="_changing_the_default_tracking_location">3.12. Changing the default tracking location</h3>
<div class="paragraph">
<p>When performing localhost requests to Apache Unomi, a default location will be used to insert values into the session
to make the location-based personalization still work. You can modify the default location settings using the
@@ -2087,8 +2100,8 @@ org.apache.unomi.ip.default.longitude=${
<p>You might want to change these for testing or for demonstration purposes.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_apache_karaf_ssh_console">2.5.13. Apache Karaf SSH Console</h4>
+<div class="sect2">
+<h3 id="_apache_karaf_ssh_console">3.13. Apache Karaf SSH Console</h3>
<div class="paragraph">
<p>The Apache Karaf SSH console is available inside Apache Unomi, but the port has been changed from the default value of
8101 to 8102 to avoid conflicts with other Karaf-based products. So to connect to the SSH console you should use:</p>
@@ -2103,16 +2116,16 @@ org.apache.unomi.ip.default.longitude=${
shell commands in the "Shell commands" section of the documentation.</p>
</div>
</div>
-<div class="sect3">
-<h4 id="_elasticsearch_authentication_and_security">2.5.14. ElasticSearch authentication and security</h4>
+<div class="sect2">
+<h3 id="_elasticsearch_authentication_and_security">3.14. ElasticSearch authentication and security</h3>
<div class="paragraph">
<p>With ElasticSearch 7, it’s possible to secure the access to your data. (<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html">https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html</a>)</p>
</div>
<div class="paragraph">
<p>Depending on your ElasticSearch license you may need to install Kibana and enable xpack security: <a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html">https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html</a></p>
</div>
-<div class="sect4">
-<h5 id="_user_authentication">User authentication !</h5>
+<div class="sect3">
+<h4 id="_user_authentication">3.14.1. User authentication !</h4>
<div class="paragraph">
<p>If your ElasticSearch have been configured to be only accessible by authenticated users (<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html">https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html</a>)</p>
</div>
@@ -2126,8 +2139,8 @@ password=PASSWORD</code></pre>
</div>
</div>
</div>
-<div class="sect4">
-<h5 id="_ssl_communication">SSL communication</h5>
+<div class="sect3">
+<h4 id="_ssl_communication">3.14.2. SSL communication</h4>
<div class="paragraph">
<p>By default Unomi will communicate with ElasticSearch using <code>http</code>
but you can configure your ElasticSearch server(s) to allow encrypted request using <code>https</code>.</p>
@@ -2171,164 +2184,205 @@ of the ElasticSearch server(s). But if y
</div>
</div>
</div>
+</div>
+<div class="sect1">
+<h2 id="_migrations">4. Migrations</h2>
+<div class="sectionbody">
+<div class="paragraph">
+<p>This section contains information and steps to migrate between major Unomi versions.</p>
+</div>
<div class="sect2">
-<h3 id="_useful_apache_unomi_urls">2.6. Useful Apache Unomi URLs</h3>
+<h3 id="_from_version_1_4_to_1_5">4.1. From version 1.4 to 1.5</h3>
+<div class="sect3">
+<h4 id="_data_model_and_elasticsearch_7">4.1.1. Data model and ElasticSearch 7</h4>
<div class="paragraph">
-<p>In this section we will list some useful URLs that can be used to quickly access parts of Apache Unomi that can help
-you understand or diagnose what is going on in the system.</p>
+<p>Since Apache Unomi version 1.5.0 we decided to upgrade the supported ElasticSearch version to the latest 7.4.2.</p>
</div>
<div class="paragraph">
-<p>You can of course find more information about the REST API in the <a href="http://unomi.apache.org/documentation.html">related section</a>
-in the Apache Unomi website.</p>
+<p>To be able to do so, we had to rework the way the data was stored inside ElasticSearch.</p>
</div>
<div class="paragraph">
-<p>For these requests it can be nice to use a browser (such as Firefox) that understands JSON to make it easier to view the
-results as the returned JSON is not beautified (another possiblity is a tool such as Postman).</p>
+<p>Previously every items was stored inside the same ElasticSearch index but this is not allowed anymore in recent ElasticSearch versions.</p>
</div>
<div class="paragraph">
-<p>Important : all URLs are relative to the private Apache Unomi URL, by default: <a href="https://localhost:9443" class="bare">https://localhost:9443</a></p>
+<p>Since Apache Unomi version 1.5.0 every type of items (see section: <a href="#_items">Items</a>) is now stored in a dedicated separated index.</p>
</div>
-<table class="tableblock frame-all grid-all stretch">
-<caption class="title">Table 2. Useful URLs</caption>
-<colgroup>
-<col style="width: 33.3333%;">
-<col style="width: 33.3333%;">
-<col style="width: 33.3334%;">
-</colgroup>
-<thead>
-<tr>
-<th class="tableblock halign-left valign-top">Path</th>
-<th class="tableblock halign-left valign-top">Method</th>
-<th class="tableblock halign-left valign-top">Description</th>
-</tr>
-</thead>
-<tbody>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/properties</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed properties</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/definitions/conditions</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed conditions</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/definitions/actions</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed actions</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/PROFILE_ID</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Dumping a profile in JSON</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/PROFILE_ID/sessions</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing sessions for a profile</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/sessions/SESSION_ID</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Dumping a session in JSON</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/sessions/SESSION_ID/events</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing events for a session. This query can have additional such as eventTypes, q (query), offset, size, sort. See the <a href="https://unomi.apache.org/rest-api-doc/#1019321624">related
-section in the REST API</a> for details.</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/events/search</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">POST</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Listing events for a profile. You will need to provide a query in the body of the request that looks something like this (and <a href="https://unomi.apache.org/rest-api-doc/#1768188821">documentation is available in the REST API</a>) :
-{ "offset" : 0,
- "limit" : 20,
- "condition" : {
- "type": "eventPropertyCondition",
- "parameterValues" : {
- "propertyName" : "profileId",
- "comparisonOperator" : "equals",
- "propertyValue" : "PROFILE_ID"
- }
- }
-}
-where PROFILE_ID is a profile identifier. This will indeed retrieve all the events for a given profile.</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/rules/statistics</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Get all rule execution statistics</p></td>
-</tr>
-<tr>
-<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/rules/statistics</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">DELETE</p></td>
-<td class="tableblock halign-left valign-top"><p class="tableblock">Reset all rule execution statistics to 0</p></td>
-</tr>
-</tbody>
-</table>
</div>
-<div class="sect2">
-<h3 id="_how_profile_tracking_works">2.7. How profile tracking works</h3>
+<div class="sect3">
+<h4 id="_api_changes">4.1.2. API changes</h4>
<div class="paragraph">
-<p>In this section you will learn how Apache Unomi keeps track of visitors.</p>
+<p>To be able to handle the multiple indices the Persistence API implementation
+(<a href="https://github.com/apache/unomi/blob/9f1bab437fd93826dc54d318ed00d3b2e3161437/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java">ElasticSearchPersistenceServiceImpl</a>)
+have been adapted and simplified.</p>
+</div>
+<div class="paragraph">
+<p>The good news is that there is no API changes, the persistence API interface didn’t changed.</p>
+</div>
+<div class="paragraph">
+<p>Any custom Apache Unomi plugins or extensions should continue to work on Apache Unomi 1.5.0.</p>
+</div>
+<div class="paragraph">
+<p>The only notable changes are located at the
+<a href="https://github.com/apache/unomi/blob/9f1bab437fd93826dc54d318ed00d3b2e3161437/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java">ElasticSearchPersistenceServiceImpl Java class</a>.
+This class should not be use directly, instead you should use OSGI service dependency injection using the interface <a href="https://github.com/apache/unomi/blob/9f1bab437fd93826dc54d318ed00d3b2e3161437/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java">PersistenceService</a>.</p>
+</div>
+<div class="paragraph">
+<p>But if you are interested in the implementation changes:</p>
</div>
-<div class="sect3">
-<h4 id="_steps">2.7.1. Steps</h4>
<div class="olist arabic">
<ol class="arabic">
<li>
-<p>A visitor comes to a website</p>
-</li>
-<li>
-<p>The web server resolves a previous request session ID if it exists, or if it doesn’t it create a new sessionID</p>
+<p>The property <code>index.name</code> have been renamed to <code>index.prefix</code>.
+Previously used for the single one index name, now every index is prefixed using this property. (<code>context-</code> by default)</p>
</li>
<li>
-<p>A request to Apache Unomi’s /context.json servlet is made passing the web server session ID as a query parameter</p>
+<p>We removed the property <code>index.names</code> originally used to create additional indices (used by the geonames DB for exemple).
+This property is not needed anymore because the index is automatically created by the peristence service when the mapping configuration is loaded.
+Example of mapping configuration file: (<a href="https://github.com/apache/unomi/blob/9f1bab437fd93826dc54d318ed00d3b2e3161437/extensions/geonames/services/src/main/resources/META-INF/cxs/mappings/geonameEntry.json">geoname index mapping</a>)</p>
</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>Because of this changes the geonames DB index name is now respecting the index naming with prefix like any other item type.
+Previously named: <code>geonames</code> is now using the index name <code>context-geonameentry</code>
+(see: <a href="#_installing_geonames_database">Documentation about geonames extension</a>).</p>
+</div>
+</div>
+<div class="sect3">
+<h4 id="_migration_steps">4.1.3. Migration steps</h4>
+<div class="paragraph">
+<p>In order to migrate the data from ElasticSearch 5 to 7, Unomi provides a migration tool that is directly integrated.</p>
+</div>
+<div class="paragraph">
+<p>In this migration the following is assumed:</p>
+</div>
+<div class="ulist">
+<ul>
<li>
-<p>Unomi uses the sessionID and tries to load an existing session, if none is found a new session is created with the
-ID passed by the web server</p>
+<p>the ElasticSearch 5 cluster installation is referred to as the <code>source</code></p>
</li>
<li>
-<p>If a session was found, the profile ID is extracted from the session and if it not found, Unomi looks for a cookie
-called <code>context-profile-id</code> to read the profileID. If no profileID is found or if the session didn’t exist, a new
-profile ID is created by Apache Unomi</p>
+<p>the ElasticSearch 7 cluster installation is referred to as the <code>target</code></p>
</li>
<li>
-<p>If the profile ID existed, the corresponding profile is loaded by Apache Unomi, otherwise a new profile is created</p>
+<p>the Unomi 1.4 cluster installation is completely stopped</p>
</li>
<li>
-<p>If events were passed along with the request to the context.json endpoint, they are processed against the profile</p>
+<p>the Unomi 1.5 cluster installation has never been started (just uncompressed)</p>
</li>
<li>
-<p>The updated profile is sent back as a response to the context.json request. Along with the response</p>
+<p>the Unomi 1.5 cluster installation has been configured to connect to the <code>target</code> (ElasticSearch 7) cluster</p>
</li>
-</ol>
+</ul>
</div>
<div class="paragraph">
-<p>It is important to note that the profileID is always server-generated. Injecting a custom cookie with a non-valid
-profile ID will result in failure to load the profile. Profile ID are UUIDs, which make them (pretty) safe from brute-
-forcing.</p>
+<p>It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the <code>source</code> clusters (including ElasticSearch and
+Unomi clusters), and ideally to perform the migration on a restored snapshot of the <code>source</code> cluster. For more information
+on ElasticSearch 5 snapshots and restore you can find it here:</p>
</div>
+<div class="literalblock">
+<div class="content">
+<pre>https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html</pre>
</div>
</div>
-<div class="sect2">
-<h3 id="_context_request_flow">2.8. Context Request Flow</h3>
<div class="paragraph">
-<p>Here is an overview of how Unomi processes incoming requests to the <code>ContextServlet</code>.</p>
+<p>The way the migration works is that both ElasticSearch 5 AND an ElasticSearch 7 clusters (or just single nodes) will
+be started at the same time, and data will be migrated from the ES 5 to the ES 7 cluster. Note that it is possible to use
+a single node for both the <code>source</code> and the <code>target</code> clusters to - for example - perform the migration on a single
+machine. If you choose to do that you will have to adjust port numbers on either the <code>source</code> or <code>target</code> cluster node.
+Changing ports requires a restart of the ES cluster you are modifying. In this example we will illustrate how to migrate
+by modifying the <code>source</code> cluster node ports.</p>
</div>
-<div class="imageblock">
+<div class="paragraph">
+<p>So in the <code>source</code> 's ElasticSearch 5 <code>config/elasticsearch.yml</code> file we have modified the default ports to:</p>
+</div>
+<div class="literalblock">
<div class="content">
-<img src="images/unomi-request.png" alt="Unomi request overview">
+<pre>transport.tcp.port: 9310
+http.port: 9210</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Make SURE you change the ports out of the default 9200-9205 and 9300-9305 range (or whatever your cluster uses) otherwise
+both clusters will attempt to merge!</p>
+</div>
+<div class="paragraph">
+<p>On the <code>target</code> ElasticSearch 7 cluster configuration you will need to add the following setting in the <code>config/elasticsearch.yml</code>:</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>reindex.remote.whitelist: "localhost:9210"</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Replace "localhost:9210" which whatever location your <code>source</code> cluster is available at. Restart or start your
+<code>target</code> ElasticSearch 7 cluster.</p>
+</div>
+<div class="paragraph">
+<p>Important: Make sure you haven’t started Apache Unomi before (using the <code>unomi:start</code> command or the autostart command
+line parameter) otherwise you will need to restart your Apache Unomi installation from scratch. The best way to be sure
+of that is to start a new Unomi install by uncompressing the archive and not launching it.</p>
+</div>
+<div class="paragraph">
+<p>You can then start both instances of ElasticSearch 5 and ElasticSearch 7 and finally start Apache Unomi using:</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>./karaf</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Once in the console launch the migration using the following command:</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>migrate 1.4.0</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>Note: the 1.4.0 version is the starting version. If you are starting from a different version (for example a fork), make
+sure that you know what official version of Apache Unomi it corresponds to and you can use the official version number
+as a start version for the migration.</p>
+</div>
+<div class="paragraph">
+<p>Follow the instructions and answer the prompts. If you used the above configuration as an example you can simply use the
+default values.</p>
+</div>
+<div class="paragraph">
+<p>Be careful because the first address that the tool will ask for is the <code>target</code> (ElasticSearch 7) cluster, not the
+ES 5 one.</p>
+</div>
+<div class="paragraph">
+<p>Note that it is also possible to change the index prefix to be different from the default <code>context</code> value
+so that you could host multiple Apache Unomi instances on the same ElasticSearch cluster.</p>
+</div>
+<div class="paragraph">
+<p>Important note: only the data that Apache Unomi manages will be migrated. If you have any other data (for example Kibana
+or ElasticSearch monitoring indices) they will not be migrated by this migration tool.</p>
+</div>
+<div class="paragraph">
+<p>Once the migration has completed, you can start the new Unomi instance using:</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>unomi:start</pre>
+</div>
+</div>
+<div class="paragraph">
+<p>You should then validate that all the data has been properly migrated. For example you could issue a command to list
+the profiles:</p>
+</div>
+<div class="literalblock">
+<div class="content">
+<pre>profile-list</pre>
+</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
-<h2 id="_queries_and_aggregations">3. Queries and aggregations</h2>
+<h2 id="_queries_and_aggregations">5. Queries and aggregations</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Apache Unomi contains a <code>query</code> endpoint that is quite powerful. It provides ways to perform queries that can quickly
@@ -2338,7 +2392,7 @@ get result counts, apply metrics such as
<p>In this section we will show examples of requests that may be built using this API.</p>
</div>
<div class="sect2">
-<h3 id="_query_counts">3.1. Query counts</h3>
+<h3 id="_query_counts">5.1. Query counts</h3>
<div class="paragraph">
<p>Query counts are highly optimized queries that will count the number of objects that match a certain condition without
retrieving the results. This can be used for example to quickly figure out how many objects will match a given condition
@@ -2393,7 +2447,7 @@ EOF</code></pre>
</div>
</div>
<div class="sect2">
-<h3 id="_metrics">3.2. Metrics</h3>
+<h3 id="_metrics">5.2. Metrics</h3>
<div class="paragraph">
<p>Metric queries make it possible to apply functions to the resulting property. The supported metrics are:</p>
</div>
@@ -2455,7 +2509,7 @@ EOF</code></pre>
</div>
</div>
<div class="sect2">
-<h3 id="_aggregations">3.3. Aggregations</h3>
+<h3 id="_aggregations">5.3. Aggregations</h3>
<div class="paragraph">
<p>Aggregations are a very powerful way to build queries in Apache Unomi that will collect and aggregate data by filtering
on certain conditions.</p>
@@ -2467,7 +2521,7 @@ on certain conditions.</p>
- a condition (used to filter the data set that will be aggregated)</p>
</div>
<div class="sect3">
-<h4 id="_aggregation_types">3.3.1. Aggregation types</h4>
+<h4 id="_aggregation_types">5.3.1. Aggregation types</h4>
<div class="paragraph">
<p>Aggregations may be of different types. They are listed here below.</p>
</div>
@@ -2705,14 +2759,14 @@ EOF</code></pre>
</div>
</div>
<div class="sect1">
-<h2 id="_profile_import_export">4. Profile import & export</h2>
+<h2 id="_profile_import_export">6. Profile import & export</h2>
<div class="sectionbody">
<div class="paragraph">
<p>The profile import and export feature in Apache Unomi is based on configurations and consumes or produces CSV files that
contain profiles to be imported and exported.</p>
</div>
<div class="sect2">
-<h3 id="_importing_profiles">4.1. Importing profiles</h3>
+<h3 id="_importing_profiles">6.1. Importing profiles</h3>
<div class="paragraph">
<p>Only <code>ftp</code>, <code>sftp</code>, <code>ftps</code> and <code>file</code> are supported in the source path. For example:</p>
</div>
@@ -2772,7 +2826,7 @@ ftps://USER@HOST?password=PASSWORD&f
</ul>
</div>
<div class="sect3">
-<h4 id="_import_api">4.1.1. Import API</h4>
+<h4 id="_import_api">6.1.1. Import API</h4>
<div class="paragraph">
<p>Apache Unomi provides REST endpoints to manage import configurations:</p>
</div>
@@ -2847,7 +2901,7 @@ ftps://USER@HOST?password=PASSWORD&f
</div>
</div>
<div class="sect2">
-<h3 id="_exporting_profiles">4.2. Exporting profiles</h3>
+<h3 id="_exporting_profiles">6.2. Exporting profiles</h3>
<div class="paragraph">
<p>Only <code>ftp</code>, <code>sftp</code>, <code>ftps</code> and `file are supported in the source path. For example:</p>
</div>
@@ -2868,7 +2922,7 @@ per file.</p>
<p>See <a href="http://camel.apache.org/ftp.html and http://camel.apache.org/file2.html to" class="bare">http://camel.apache.org/ftp.html and http://camel.apache.org/file2.html to</a> build more complex destination path.</p>
</div>
<div class="sect3">
-<h4 id="_export_api">4.2.1. Export API</h4>
+<h4 id="_export_api">6.2.1. Export API</h4>
<div class="paragraph">
<p>Apache Unomi provides REST endpoints to manage export configurations:</p>
</div>
@@ -2936,7 +2990,7 @@ per file.</p>
</div>
</div>
<div class="sect2">
-<h3 id="_configuration_in_details">4.3. Configuration in details</h3>
+<h3 id="_configuration_in_details">6.3. Configuration in details</h3>
<div class="paragraph">
<p>First configuration you need to change would be the configuration type of your import / export feature (code name
router) in the <code>etc/unomi.custom.system.properties</code> file (creating it if necessary):</p>
@@ -3030,17 +3084,17 @@ org.apache.unomi.router.config.allowedEn
</div>
</div>
<div class="sect1">
-<h2 id="_consent_management">5. Consent management</h2>
+<h2 id="_consent_management">7. Consent management</h2>
<div class="sectionbody">
<div class="sect2">
-<h3 id="_consent_api">5.1. Consent API</h3>
+<h3 id="_consent_api">7.1. Consent API</h3>
<div class="paragraph">
<p>Starting with Apache Unomi 1.3, a new API for consent management is now available. This API
is designed to be able to store/retrieve/update visitor consents in order to comply with new
privacy regulations such as the <a href="https://en.wikipedia.org/wiki/General_Data_Protection_Regulation">GDPR</a>.</p>
</div>
<div class="sect3">
-<h4 id="_profiles_with_consents">5.1.1. Profiles with consents</h4>
+<h4 id="_profiles_with_consents">7.1.1. Profiles with consents</h4>
<div class="paragraph">
<p>Visitor profiles now contain a new Consent object that contains the following information:</p>
</div>
@@ -3115,7 +3169,7 @@ EOF</code></pre>
</div>
</div>
<div class="sect3">
-<h4 id="_consent_type_definitions">5.1.2. Consent type definitions</h4>
+<h4 id="_consent_type_definitions">7.1.2. Consent type definitions</h4>
<div class="paragraph">
<p>Apache Unomi does not manage consent definitions, it leaves that to an external system (for example a CMS) so that it
can handle user-facing UIs to create, update, internationalize and present consent definitions to end users.</p>
@@ -3126,7 +3180,7 @@ consent type.</p>
</div>
</div>
<div class="sect3">
-<h4 id="_creating_update_a_visitor_consent">5.1.3. Creating / update a visitor consent</h4>
+<h4 id="_creating_update_a_visitor_consent">7.1.3. Creating / update a visitor consent</h4>
<div class="paragraph">
<p>A new built-in event type called "modifyConsent" can be sent to Apache Unomi to update a consent for the current
profile.</p>
@@ -3209,7 +3263,7 @@ EOF</code></pre>
</div>
</div>
<div class="sect3">
-<h4 id="_how_it_works_internally">5.1.4. How it works (internally)</h4>
+<h4 id="_how_it_works_internally">7.1.4. How it works (internally)</h4>
<div class="paragraph">
<p>Upon receiving this event, Apache Unomi will trigger the modifyAnyConsent rule that has the following definition:</p>
</div>
@@ -3252,14 +3306,14 @@ EOF</code></pre>
</div>
</div>
<div class="sect1">
-<h2 id="_privacy_management">6. Privacy management</h2>
+<h2 id="_privacy_management">8. Privacy management</h2>
<div class="sectionbody">
<div class="paragraph">
<p>Apache Unomi provides an endpoint to manage visitor privacy. You will find in this section information about what it
includes as well as how to use it.</p>
</div>
<div class="sect2">
-<h3 id="_setting_up_access_to_the_privacy_endpoint">6.1. Setting up access to the privacy endpoint</h3>
+<h3 id="_setting_up_access_to_the_privacy_endpoint">8.1. Setting up access to the privacy endpoint</h3>
<div class="paragraph">
<p>The privacy endpoint is a bit special, because despite being protected by basic authentication as the rest of the REST
API is is actually designed to be available to end-users.</p>
@@ -3285,7 +3339,7 @@ geographic location)</p>
</div>
</div>
<div class="sect2">
-<h3 id="_anonymizing_a_profile">6.2. Anonymizing a profile</h3>
+<h3 id="_anonymizing_a_profile">8.2. Anonymizing a profile</h3>
<div class="paragraph">
<p>It is possible to anonymize a profile, meaning it will remove all "identifying" property values from the profile.
Basically all properties with the tag <code>personalIdentifierProperties</code> will be purged from the profile.</p>
@@ -3304,7 +3358,7 @@ and <code>ASCOPE</code> must be replaced
</div>
</div>
<div class="sect2">
-<h3 id="_downloading_profile_data">6.3. Downloading profile data</h3>
+<h3 id="_downloading_profile_data">8.3. Downloading profile data</h3>
<div class="paragraph">
<p>It is possible to download the profile data of a user. This will only download the profile for a user using the
specified ID as a cookie value.</p>
@@ -3325,7 +3379,7 @@ IDs.</p>
</div>
</div>
<div class="sect2">
-<h3 id="_deleting_a_profile">6.4. Deleting a profile</h3>
+<h3 id="_deleting_a_profile">8.4. Deleting a profile</h3>
<div class="paragraph">
<p>It is possible to delete a profile, but this works a little differently than you might expect. In all cases the data
contained in the profile will be completely erased. If the <code>withData</code> optional flag is set to true, all past event and
@@ -3342,7 +3396,7 @@ and the <code>withData</code> specifies
</div>
</div>
<div class="sect2">
-<h3 id="_related">6.5. Related</h3>
+<h3 id="_related">8.5. Related</h3>
<div class="paragraph">
<p>You might also be interested in the <a href="#_consent_api">Consent API</a> section that describe how to manage profile consents.</p>
</div>
@@ -3350,10 +3404,10 @@ and the <code>withData</code> specifies
</div>
</div>
<div class="sect1">
-<h2 id="_cluster_setup">7. Cluster setup</h2>
+<h2 id="_cluster_setup">9. Cluster setup</h2>
<div class="sectionbody">
<div class="sect2">
-<h3 id="_cluster_setup_2">7.1. Cluster setup</h3>
+<h3 id="_cluster_setup_2">9.1. Cluster setup</h3>
<div class="paragraph">
<p>Apache Karaf relies on Apache Karaf Cellar, which in turn uses Hazelcast to discover and configure its cluster.</p>
</div>
@@ -3394,10 +3448,164 @@ placeholders in the hazelcast.xml file i
</div>
</div>
<div class="sect1">
-<h2 id="_reference">8. Reference</h2>
+<h2 id="_reference">10. Reference</h2>
<div class="sectionbody">
<div class="sect2">
-<h3 id="_data_model_overview">8.1. Data Model Overview</h3>
+<h3 id="_useful_apache_unomi_urls">10.1. Useful Apache Unomi URLs</h3>
+<div class="paragraph">
+<p>In this section we will list some useful URLs that can be used to quickly access parts of Apache Unomi that can help
+you understand or diagnose what is going on in the system.</p>
+</div>
+<div class="paragraph">
+<p>You can of course find more information about the REST API in the <a href="http://unomi.apache.org/documentation.html">related section</a>
+in the Apache Unomi website.</p>
+</div>
+<div class="paragraph">
+<p>For these requests it can be nice to use a browser (such as Firefox) that understands JSON to make it easier to view the
+results as the returned JSON is not beautified (another possiblity is a tool such as Postman).</p>
+</div>
+<div class="paragraph">
+<p>Important : all URLs are relative to the private Apache Unomi URL, by default: <a href="https://localhost:9443" class="bare">https://localhost:9443</a></p>
+</div>
+<table class="tableblock frame-all grid-all stretch">
+<caption class="title">Table 2. Useful URLs</caption>
+<colgroup>
+<col style="width: 33.3333%;">
+<col style="width: 33.3333%;">
+<col style="width: 33.3334%;">
+</colgroup>
+<thead>
+<tr>
+<th class="tableblock halign-left valign-top">Path</th>
+<th class="tableblock halign-left valign-top">Method</th>
+<th class="tableblock halign-left valign-top">Description</th>
+</tr>
+</thead>
+<tbody>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/properties</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed properties</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/definitions/conditions</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed conditions</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/definitions/actions</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing deployed actions</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/PROFILE_ID</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Dumping a profile in JSON</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/PROFILE_ID/sessions</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing sessions for a profile</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/sessions/SESSION_ID</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Dumping a session in JSON</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/profiles/sessions/SESSION_ID/events</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing events for a session. This query can have additional such as eventTypes, q (query), offset, size, sort. See the <a href="https://unomi.apache.org/rest-api-doc/#1019321624">related
+section in the REST API</a> for details.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/events/search</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">POST</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Listing events for a profile. You will need to provide a query in the body of the request that looks something like this (and <a href="https://unomi.apache.org/rest-api-doc/#1768188821">documentation is available in the REST API</a>) :
+{ "offset" : 0,
+ "limit" : 20,
+ "condition" : {
+ "type": "eventPropertyCondition",
+ "parameterValues" : {
+ "propertyName" : "profileId",
+ "comparisonOperator" : "equals",
+ "propertyValue" : "PROFILE_ID"
+ }
+ }
+}
+where PROFILE_ID is a profile identifier. This will indeed retrieve all the events for a given profile.</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/rules/statistics</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">GET</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Get all rule execution statistics</p></td>
+</tr>
+<tr>
+<td class="tableblock halign-left valign-top"><p class="tableblock">/cxs/rules/statistics</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">DELETE</p></td>
+<td class="tableblock halign-left valign-top"><p class="tableblock">Reset all rule execution statistics to 0</p></td>
+</tr>
+</tbody>
+</table>
+</div>
+<div class="sect2">
+<h3 id="_how_profile_tracking_works">10.2. How profile tracking works</h3>
+<div class="paragraph">
+<p>In this section you will learn how Apache Unomi keeps track of visitors.</p>
+</div>
+<div class="sect3">
+<h4 id="_steps">10.2.1. Steps</h4>
+<div class="olist arabic">
+<ol class="arabic">
+<li>
+<p>A visitor comes to a website</p>
+</li>
+<li>
+<p>The web server resolves a previous request session ID if it exists, or if it doesn’t it create a new sessionID</p>
+</li>
+<li>
+<p>A request to Apache Unomi’s /context.json servlet is made passing the web server session ID as a query parameter</p>
+</li>
+<li>
+<p>Unomi uses the sessionID and tries to load an existing session, if none is found a new session is created with the
+ID passed by the web server</p>
+</li>
+<li>
+<p>If a session was found, the profile ID is extracted from the session and if it not found, Unomi looks for a cookie
+called <code>context-profile-id</code> to read the profileID. If no profileID is found or if the session didn’t exist, a new
+profile ID is created by Apache Unomi</p>
+</li>
+<li>
+<p>If the profile ID existed, the corresponding profile is loaded by Apache Unomi, otherwise a new profile is created</p>
+</li>
+<li>
+<p>If events were passed along with the request to the context.json endpoint, they are processed against the profile</p>
+</li>
+<li>
+<p>The updated profile is sent back as a response to the context.json request. Along with the response</p>
+</li>
+</ol>
+</div>
+<div class="paragraph">
+<p>It is important to note that the profileID is always server-generated. Injecting a custom cookie with a non-valid
+profile ID will result in failure to load the profile. Profile ID are UUIDs, which make them (pretty) safe from brute-
+forcing.</p>
+</div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="_context_request_flow">10.3. Context Request Flow</h3>
+<div class="paragraph">
+<p>Here is an overview of how Unomi processes incoming requests to the <code>ContextServlet</code>.</p>
+</div>
+<div class="imageblock">
+<div class="content">
+<img src="images/unomi-request.png" alt="Unomi request overview">
+</div>
+</div>
+</div>
+<div class="sect2">
+<h3 id="_data_model_overview">10.4. Data Model Overview</h3>
<div class="paragraph">
<p>Apache Unomi gathers information about users actions, information that is processed and stored by Unomi services.
The collected information can then be used to personalize content, derive insights on user behavior, categorize the
@@ -3417,7 +3625,7 @@ There are other classes that are less ce
</div>
</div>
<div class="sect2">
-<h3 id="_scope">8.2. Scope</h3>
+<h3 id="_scope">10.5. Scope</h3>
<div class="paragraph">
<p>Scopes are not actually an object in the system, but simply unique strings that are used to "classify" objects.
For example, when using scopes with a web content management system, a scope could be associated with a site identifier or even a host name.</p>
@@ -3430,7 +3638,7 @@ For example, when using scopes with a we
</blockquote>
</div>
<div class="sect3">
-<h4 id="_example">8.2.1. Example</h4>
+<h4 id="_example">10.5.1. Example</h4>
<div class="paragraph">
<p>In the following example, the scope uses the unique identifier of a web site called âdigitallâ.</p>
</div>
@@ -3446,7 +3654,7 @@ For example, when using scopes with a we
</div>
</div>
<div class="sect2">
-<h3 id="_item">8.3. Item</h3>
+<h3 id="_item">10.6. Item</h3>
<div class="paragraph">
<p>Unomi structures the information it collects using the concept of <code>Item</code> which provides the base information (an identifier and a type) the context server needs to process and store the data.
Items are persisted according to their type (structure) and identifier (identity).
@@ -3471,7 +3679,7 @@ Scopes allow clients accessing the conte
It contains the following fields, that are inherited by other objects that inherit from it.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition">8.3.1. Structure definition</h4>
+<h4 id="_structure_definition">10.6.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: n/a</p>
</div>
@@ -3509,13 +3717,13 @@ It contains the following fields, that a
</div>
</div>
<div class="sect2">
-<h3 id="_metadata">8.4. Metadata</h3>
+<h3 id="_metadata">10.7. Metadata</h3>
<div class="paragraph">
<p>The Metadata object is an object that contains additional information about an object.
It is usually associated with an Item object (see MetadataItem below).</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_2">8.4.1. Structure definition</h4>
+<h4 id="_structure_definition_2">10.7.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: n/a</p>
</div>
@@ -3587,7 +3795,7 @@ It is usually associated with an Item ob
</table>
</div>
<div class="sect3">
-<h4 id="_example_2">8.4.2. Example</h4>
+<h4 id="_example_2">10.7.2. Example</h4>
<div class="paragraph">
<p>This example of a Metadata object structure was taken from a List associated object.
See the MetadataItem to understand how the two fit together.</p>
@@ -3611,9 +3819,9 @@ See the MetadataItem to understand how t
</div>
</div>
<div class="sect2">
-<h3 id="_metadataitem">8.5. MetadataItem</h3>
+<h3 id="_metadataitem">10.8. MetadataItem</h3>
<div class="sect3">
-<h4 id="_structure_definition_3">8.5.1. Structure definition</h4>
+<h4 id="_structure_definition_3">10.8.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_item">Item</a></p>
</div>
@@ -3640,7 +3848,7 @@ See the MetadataItem to understand how t
</table>
</div>
<div class="sect3">
-<h4 id="_example_3">8.5.2. Example</h4>
+<h4 id="_example_3">10.8.2. Example</h4>
<div class="paragraph">
<p>The following example is actually the definition of a <a href="#_list">List</a> object, which is simply a <a href="#_metadataitem">MetadataItem</a> sub-type with no additional fields.
We can see here the âitemIdâ and âitemTypeâ fields that come from the Item parent class and the âmetadataâ field that contains the object structure coming from the Metadata object type.</p>
@@ -3668,7 +3876,7 @@ We can see here the âitemIdâ a
</div>
</div>
<div class="sect2">
-<h3 id="_event">8.6. Event</h3>
+<h3 id="_event">10.9. Event</h3>
<div class="paragraph">
<p>Events represent something that is happening at a specific time (they are timestamped).
They can be used to track visitor behavior, or even for back-channel system-to-system (as for example for a login) communication.
@@ -3682,7 +3890,7 @@ Examples of events may include a click o
when a rule is triggered.</p>
</div>
<div class="sect3">
-<h4 id="_fields">8.6.1. Fields</h4>
+<h4 id="_fields">10.9.1. Fields</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_item">Item</a></p>
</div>
@@ -3744,7 +3952,7 @@ when a rule is triggered.</p>
</table>
</div>
<div class="sect3">
-<h4 id="_event_types">8.6.2. Event types</h4>
+<h4 id="_event_types">10.9.2. Event types</h4>
<div class="paragraph">
<p>Event types are completely open, and any new event type will be accepted by Apache Unomi.</p>
</div>
@@ -3754,7 +3962,7 @@ when a rule is triggered.</p>
</div>
</div>
<div class="sect2">
-<h3 id="_profile">8.7. Profile</h3>
+<h3 id="_profile">10.10. Profile</h3>
<div class="paragraph">
<p>By processing events, Unomi progressively builds a picture of who the user is and how they behave. This knowledge is
embedded in <code>Profile</code> object. A profile is an <code>Item</code> with any number of properties and optional segments and scores.
@@ -3777,7 +3985,7 @@ permits positive association between dis
action in a context where the user hadnât already been positively identified.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_4">8.7.1. Structure definition</h4>
+<h4 id="_structure_definition_4">10.10.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_item">Item</a></p>
</div>
@@ -3829,7 +4037,7 @@ action in a context where the user hadn�
</table>
</div>
<div class="sect3">
-<h4 id="_example_4">8.7.2. Example</h4>
+<h4 id="_example_4">10.10.2. Example</h4>
<div class="paragraph">
<p>In the example below, a profile for a visitor called âBill Galileoâ is detailed.
A lot of user properties (such as first name, last name, gender, job title and more) were copied over from the CMS upon initial login.
@@ -3940,14 +4148,14 @@ It has also been engaged in some goals (
</div>
</div>
<div class="sect2">
-<h3 id="_persona">8.8. Persona</h3>
+<h3 id="_persona">10.11. Persona</h3>
<div class="paragraph">
<p>A persona is a specialized version of a <a href="#_profile">Profile</a> object. It basically represents a "typical" profile and can be used
notably to simulate personalized for a type of profiles. Usually personas are created from Profile data and then edited
to represent a specific marketing persona.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_5">8.8.1. Structure definition</h4>
+<h4 id="_structure_definition_5">10.11.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_profile">Profile</a></p>
</div>
@@ -3956,7 +4164,7 @@ to represent a specific marketing person
</div>
</div>
<div class="sect3">
-<h4 id="_example_5">8.8.2. Example</h4>
+<h4 id="_example_5">10.11.2. Example</h4>
<div class="paragraph">
<p>In the following example a Persona represents a visitor from Europe, that can be used to match by location.</p>
</div>
@@ -3982,13 +4190,13 @@ to represent a specific marketing person
</div>
</div>
<div class="sect2">
-<h3 id="_consent">8.9. Consent</h3>
+<h3 id="_consent">10.12. Consent</h3>
<div class="paragraph">
<p>A consent represents a single instance of a consent granted/refused or revoked by a profile.
A profile will contain multiple instances of consent identified by unique identifiers.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_6">8.9.1. Structure definition</h4>
+<h4 id="_structure_definition_6">10.12.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: n/a</p>
</div>
@@ -4035,7 +4243,7 @@ A profile will contain multiple instance
</table>
</div>
<div class="sect3">
-<h4 id="_example_6">8.9.2. Example</h4>
+<h4 id="_example_6">10.12.2. Example</h4>
<div class="paragraph">
<p>In this example, the consent called ânewsletterâ was given on the âdigitallâ website.</p>
</div>
@@ -4053,13 +4261,13 @@ A profile will contain multiple instance
</div>
</div>
<div class="sect2">
-<h3 id="_session">8.10. Session</h3>
+<h3 id="_session">10.13. Session</h3>
<div class="paragraph">
<p>A session represents a period of time during which a visitor/profile has been active.
It makes it possible to gather data and then use it for reporting and further analysis by regrouping all the events that occurred during the session.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_7">8.10.1. Structure definition</h4>
+<h4 id="_structure_definition_7">10.13.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_item">Item</a></p>
</div>
@@ -4116,7 +4324,7 @@ It makes it possible to gather data and
</table>
</div>
<div class="sect3">
-<h4 id="_example_7">8.10.2. Example</h4>
+<h4 id="_example_7">10.13.2. Example</h4>
<div class="paragraph">
<p>In this example the session contains a copy of the profile of the visitor.
It is a visitor that has previously authentified in a CMS and whoâse information was copied at the time of login from the CMS user account to the profile.
@@ -4224,7 +4432,7 @@ The visitorâs location is also reso
</div>
</div>
<div class="sect2">
-<h3 id="_segment">8.11. Segment</h3>
+<h3 id="_segment">10.14. Segment</h3>
<div class="paragraph">
<p>Segments are used to group profiles together, and are based on conditions that are executed on profiles to determine
if they are part of a segment or not.</p>
@@ -4234,7 +4442,7 @@ if they are part of a segment or not.</p
highly dynamic concept.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_8">8.11.1. Structure definition</h4>
+<h4 id="_structure_definition_8">10.14.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_metadataitem">MetadataItem</a></p>
</div>
@@ -4261,7 +4469,7 @@ highly dynamic concept.</p>
</table>
</div>
<div class="sect3">
-<h4 id="_example_8">8.11.2. Example</h4>
+<h4 id="_example_8">10.14.2. Example</h4>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
@@ -4347,7 +4555,7 @@ EOF</code></pre>
</div>
</div>
<div class="sect2">
-<h3 id="_condition">8.12. Condition</h3>
+<h3 id="_condition">10.15. Condition</h3>
<div class="paragraph">
<p>Conditions are a very useful notion inside of Apache Unomi, as they are used as the basis for multiple other objects.
Conditions may be used as parts of:</p>
@@ -4385,7 +4593,7 @@ Composition is an essential element of b
<p>For a more complete list of available condition types, see the <a href="#_built_in_condition_types">Built-in condition types</a> reference section.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_9">8.12.1. Structure definition</h4>
+<h4 id="_structure_definition_9">10.15.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: n/a</p>
</div>
@@ -4421,7 +4629,7 @@ that contains a list of conditions to ev
</table>
</div>
<div class="sect3">
-<h4 id="_example_9">8.12.2. Example</h4>
+<h4 id="_example_9">10.15.2. Example</h4>
<div class="paragraph">
<p>Here is an example of a complex condition:</p>
</div>
@@ -4458,7 +4666,7 @@ or <code>sessionReassigned</code>.</p>
</div>
</div>
<div class="sect2">
-<h3 id="_rule">8.13. Rule</h3>
+<h3 id="_rule">10.16. Rule</h3>
<div class="imageblock">
<div class="content">
<img src="images/unomi-rule-engine.png" alt="Unomi Rule Engine">
@@ -4488,7 +4696,7 @@ You can imagine conditions checking inco
<p>For example the Salesforce CRM connector is simply a set of actions that pull and push data into the CRM. It is then just a matter of setting up the proper rules with the proper conditions to determine when and how the data will be pulled or pushed into the third-party system.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_10">8.13.1. Structure definition</h4>
+<h4 id="_structure_definition_10">10.16.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_metadataitem">MetadataItem</a></p>
</div>
@@ -4545,7 +4753,7 @@ You can imagine conditions checking inco
</table>
</div>
<div class="sect3">
-<h4 id="_example_10">8.13.2. Example</h4>
+<h4 id="_example_10">10.16.2. Example</h4>
<div class="paragraph">
<p>In this exmample we should the default <code>updateProperties</code> built-in rule that matches the <code>updateProperties</code> event and
executes the built-in <code>updatePropertiesAction</code></p>
@@ -4587,7 +4795,7 @@ executes the built-in <code>updateProper
</div>
</div>
<div class="sect2">
-<h3 id="_action">8.14. Action</h3>
+<h3 id="_action">10.17. Action</h3>
<div class="paragraph">
<p>Actions are executed by rules in a sequence, and an action is only executed once the previous action has finished executing.
If an action generates an exception, it will be logged and the execution sequence will continue unless in the case of a
@@ -4603,7 +4811,7 @@ an IP address), or even pulling and/or
You may find the list of built-in action types in the <a href="#_built_in_action_types">Built-in action types</a> section.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_11">8.14.1. Structure definition</h4>
+<h4 id="_structure_definition_11">10.17.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: n/a</p>
</div>
@@ -4635,7 +4843,7 @@ You may find the list of built-in action
</table>
</div>
<div class="sect3">
-<h4 id="_example_11">8.14.2. Example</h4>
+<h4 id="_example_11">10.17.2. Example</h4>
<div class="paragraph">
<p>In this example of an action, taking from the <code>form-mapping-example.json</code> rule, the <code>setPropertyAction</code> action is used
to set the <code>properties.firstName</code> profile property to a value read from the event properties called <code>properties.firstName</code>.
@@ -4657,7 +4865,7 @@ overridden or not.</p>
</div>
</div>
<div class="sect2">
-<h3 id="_list">8.15. List</h3>
+<h3 id="_list">10.18. List</h3>
<div class="paragraph">
<p>Lists are a âmanualâ way to organize profiles, whereas Segments are a dynamic way to regroup them.
List objects actually only define the list in terms of name, description and other metadata but the list of members is actually not represented in the object.
@@ -4665,7 +4873,7 @@ The profiles contain references to the l
This property is an array of list identifiers so in order to retrieve all the list names for a given profile, a lookup of List objects is required using the identifiers.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_12">8.15.1. Structure definition</h4>
+<h4 id="_structure_definition_12">10.18.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_metadataitem">MetadataItem</a></p>
</div>
@@ -4689,7 +4897,7 @@ This property is an array of list identi
</table>
</div>
<div class="sect3">
-<h4 id="_example_12">8.15.2. Example</h4>
+<h4 id="_example_12">10.18.2. Example</h4>
<div class="paragraph">
<p>Hereâs an example of a list called âFirst listâ, along with its description, its scope, tags, etc.. . As a List object is basically a MetadataItem sub-class it simply has all the fields defined in that parent class.
Note that the List does not contain Profiles, it is Profiles that reference the Lists, not the reverse.</p>
@@ -4717,7 +4925,7 @@ Note that the List does not contain Prof
</div>
</div>
<div class="sect2">
-<h3 id="_goal">8.16. Goal</h3>
+<h3 id="_goal">10.19. Goal</h3>
<div class="paragraph">
<p>A goal can be defined with two conditions: a start event condition and an target event condition.
Basically the goal will be âactiveâ when its start event condition is satisfied, and âreachedâ when the target event condition is true.
@@ -4725,7 +4933,7 @@ Goals may also (optionally) be associate
Once a goal is âreachedâ, a âgoalâ event triggered and the profile that is currently interacting with the system will see its system properties updated to indicate which goal has been reached.</p>
</div>
<div class="sect3">
-<h4 id="_structure_definition_13">8.16.1. Structure definition</h4>
+<h4 id="_structure_definition_13">10.19.1. Structure definition</h4>
<div class="paragraph">
<p>Inherits all the fields from: <a href="#_metadataitem">MetadataItem</a></p>
</div>
@@ -4762,7 +4970,7 @@ Once a goal is âreachedâ, a â
</table>
</div>
<div class="sect3">
-<h4 id="_example_13">8.16.2. Example</h4>
+<h4 id="_example_13">10.19.2. Example</h4>
<div class="paragraph">
<p>In the following example, a goal called âdownloadGoalExampleâ is started when a new session is created (we use the âsessionCreatedEventConditionâ for that) and is reached when a profile downloads a file called âACME_WP.pdfâ (thatâs what the âdownloadEventConditionâ means).</p>
</div>
[... 910 lines stripped ...]