You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by sh...@apache.org on 2020/02/14 16:46:43 UTC
[unomi] 01/01: UNOMI-270 & UNOMI-175 Data Model Documentation &
Plugin vs Extension clarification - New data model documentation - New
built-in event types documentation - Removed old "Concepts" documentation
(replaced by data model) - Merged extension & plugin documentation to only
talk about plugins - Clarified differences between types & instances -
Reorganized flow of manual (start with practical stuff,
move reference stuff to the reference section)
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch UNOMI-270-UNOMI-175-DATAMODEL-AND-PLUGIN-DOC
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 6883923c726bed255e5a4b9ee3187034659d904e
Author: Serge Huber <sh...@apache.org>
AuthorDate: Fri Feb 14 17:46:07 2020 +0100
UNOMI-270 & UNOMI-175 Data Model Documentation & Plugin vs Extension clarification
- New data model documentation
- New built-in event types documentation
- Removed old "Concepts" documentation (replaced by data model)
- Merged extension & plugin documentation to only talk about plugins
- Clarified differences between types & instances
- Reorganized flow of manual (start with practical stuff, move reference stuff to the reference section)
Signed-off-by: Serge Huber <sh...@apache.org>
---
.../asciidoc/{actions.adoc => action-types.adoc} | 6 +-
manual/src/main/asciidoc/builtin-event-types.adoc | 650 ++++++++++++
manual/src/main/asciidoc/concepts.adoc | 311 ------
manual/src/main/asciidoc/conditions.adoc | 20 +-
manual/src/main/asciidoc/context-request-flow.adoc | 18 +
manual/src/main/asciidoc/datamodel.adoc | 1077 ++++++++++++++++++++
manual/src/main/asciidoc/extending-plugins.adoc | 121 ---
.../main/asciidoc/images/data-model-overview.png | Bin 0 -> 280344 bytes
.../src/main/asciidoc/images/form-event-type.png | Bin 0 -> 1065895 bytes
.../src/main/asciidoc/images/login-event-type.png | Bin 0 -> 16230 bytes
.../asciidoc/images/modify-consent-event-type.png | Bin 0 -> 76508 bytes
manual/src/main/asciidoc/images/profile.png | Bin 0 -> 199798 bytes
.../images/update-properties-event-type.png | Bin 0 -> 199798 bytes
manual/src/main/asciidoc/index.adoc | 20 +-
manual/src/main/asciidoc/new-data-model.adoc | 10 +-
...custom-extensions.adoc => writing-plugins.adoc} | 177 +++-
16 files changed, 1927 insertions(+), 483 deletions(-)
diff --git a/manual/src/main/asciidoc/actions.adoc b/manual/src/main/asciidoc/action-types.adoc
similarity index 90%
rename from manual/src/main/asciidoc/actions.adoc
rename to manual/src/main/asciidoc/action-types.adoc
index 3cbe239..801705c 100644
--- a/manual/src/main/asciidoc/actions.adoc
+++ b/manual/src/main/asciidoc/action-types.adoc
@@ -11,10 +11,10 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//
-=== Built-in actions
+=== Built-in action types
-Unomi comes with quite a lot of built-in actions. Instead of detailing them one by one you will find here an overview of
-what an action descriptor looks like:
+Unomi comes with quite a lot of built-in action typess. Instead of detailing them one by one you will find here an overview of
+what an action type descriptor looks like:
[source,json]
----
diff --git a/manual/src/main/asciidoc/builtin-event-types.adoc b/manual/src/main/asciidoc/builtin-event-types.adoc
new file mode 100644
index 0000000..f95b438
--- /dev/null
+++ b/manual/src/main/asciidoc/builtin-event-types.adoc
@@ -0,0 +1,650 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+=== Built-in Event types
+
+Apache Unomi comes with built-in event types, which we describe below.
+
+==== Login event type
+
+The login event type is used to signal an authentication event has been triggered.
+This event should be “secured”, meaning that it should not be accepted from any location, and by default Apache Unomi will only accept this event from configured “third-party” servers (identified by their IP address and a Unomi application key).
+
+Usually, the login event will contain information passed by the authentication server and may include user properties and any additional information.
+Rules may be set up to copy the information from the event into the profile, but this is not done in the default set of rules provided by Apache Unomi for security reasons.
+You can find an example of such a rule here: https://github.com/apache/unomi/blob/master/samples/login-integration/src/main/resources/META-INF/cxs/rules/exampleLogin.json[https://github.com/apache/unomi/blob/master/samples/login-integration/src/main/resources/META-INF/cxs/rules/exampleLogin.json]
+
+===== Structure overview
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | login
+
+| source | Not used (null)
+
+| target | an Item representing the user that logged in
+
+| scope | the scope in which the user has authenticated
+
+| properties | Not used (empty)
+
+|===
+
+===== Example
+
+In this case, a user has logged into a site called “digitall”, and his user information the following properties are associated with the active user..and perhaps show his visitor profile or user information.
+
+
+image::login-event-type.png[]
+
+[source,json]
+----
+{
+ "itemId": "0b8825a6-efb8-41a6-bea5-d745b33c94cb",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "login",
+ "sessionId": "7b8a5f17-cdb0-4c14-b676-34c1c0de0825",
+ "profileId": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ "timeStamp": "2020-01-30T21:18:28Z",
+ "properties": {},
+ "source": null,
+ "target": {
+ "itemId": "13054a95-092d-4d7b-81f5-e4656c2ebc88",
+ "itemType": "cmsUser",
+ "scope": null,
+ "properties": {
+ "j:function": "Vice President",
+ "preferredLanguage": "en",
+ "j:title": "mister",
+ "emailNotificationsDisabled": "true",
+ "j:organization": "Acme Space",
+ "j:gender": "male",
+ "j:nodename": "bill",
+ "j:lastName": "Galileo",
+ "j:publicProperties": "j:about,j:firstName,j:function,j:gender,j:lastName,j:organization,j:picture,j:title",
+ "j:firstName": "Bill",
+ "j:about": "<p> Lorem Ipsum dolor sit amet.</p> "
+ }
+ }
+}
+----
+
+==== View event type
+
+This event is triggered when a web page is viewed by a user.
+Some integrators might also want to trigger it when a single-page-application screen is displayed or when a mobile application screen is displayed.
+
+===== Structure description
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | view
+
+| source | the source for the view event, could be a web site, an application name, etc…
+
+| target | the page/screen being viewed
+
+| properties | Not used (empty)
+
+|===
+
+===== Example
+
+In this case a use has visited the home page of the digitall site.
+As this is the first page upon login, the destination and referring URL are the same.
+
+[source,json]
+----
+{
+ "itemId": "c75f50c2-ab55-4d95-be69-cbbeee180d6b",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "view",
+ "sessionId": "7b8a5f17-cdb0-4c14-b676-34c1c0de0825",
+ "profileId": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ "timeStamp": "2020-01-30T21:18:32Z",
+ "properties": {},
+ "source": {
+ "itemId": "29f5fe37-28c0-48f3-966b-5353bed87308",
+ "itemType": "site",
+ "scope": "digitall",
+ "properties": {}
+ },
+ "target": {
+ "itemId": "f20836ab-608f-4551-a930-9796ec991340",
+ "itemType": "page",
+ "scope": "digitall",
+ "properties": {
+ "pageInfo": {
+ "templateName": "home",
+ "language": "en",
+ "destinationURL": "http://localhost:8080/sites/digitall/home.html",
+ "categories": [],
+ "pageID": "f20836ab-608f-4551-a930-9796ec991340",
+ "nodeType": "jnt:page",
+ "pagePath": "/sites/digitall/home",
+ "pageName": "Home",
+ "referringURL": "http://localhost:8080/sites/digitall/home.html",
+ "tags": [],
+ "isContentTemplate": false
+ },
+ "attributes": {},
+ "consentTypes": []
+ }
+ }
+}
+----
+
+==== Form event type
+
+This event type is used to track form submissions.
+These could range from login to survey form data captured and processed in Apache Unomi using rules.
+
+===== Structure description
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | form
+
+| source | the page/screen on which the form was submitted
+
+| target | the form that was submitted (there could be more than one form on a page/screen)
+
+| properties | contain the data submitted in the form
+
+|===
+
+===== Example
+
+A form exists on the digitall site, and has been submitted by a visitor.
+In this case it was a search form that contains fields to adjust the search parameters.
+
+image::form-event-type.png[]
+
+[source,json]
+----
+{
+ "itemId": "44177ffe-b5c8-4575-a8e5-f8aa0d4ee792",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "form",
+ "sessionId": "be416c08-8b9b-4611-990f-3a8bf3ed4e68",
+ "profileId": "bc1e1238-a9ac-4b3a-8f63-5eec205cfcd5",
+ "timeStamp": "2020-01-30T21:41:22Z",
+ "properties": {
+ "jcrMethodToCall": "get",
+ "src_originSiteKey": "digitall",
+ "src_terms[0].term": "test",
+ "src_terms[0].applyFilter": "true",
+ "src_terms[0].match": "all_words",
+ "src_terms[0].fields.siteContent": "true",
+ "src_terms[0].fields.tags": "true",
+ "src_terms[0].fields.files": "true",
+ "src_sites.values": "digitall",
+ "src_sitesForReferences.values": "systemsite",
+ "src_languages.values": "en"
+ },
+ "source": {
+ "itemId": "97e14221-33dd-4608-82ae-9724d15d4f12",
+ "itemType": "page",
+ "scope": "digitall",
+ "properties": {
+ "pageInfo": {
+ "templateName": "home",
+ "language": "en",
+ "destinationURL": "http://localhost:8080/sites/digitall/home/search-results.html",
+ "categories": [],
+ "pageID": "97e14221-33dd-4608-82ae-9724d15d4f12",
+ "nodeType": "jnt:page",
+ "pagePath": "/sites/digitall/home/search-results",
+ "pageName": "Search Results",
+ "referringURL": "http://localhost:8080/cms/edit/default/en/sites/digitall/home.html",
+ "tags": [],
+ "isContentTemplate": false
+ },
+ "attributes": {},
+ "consentTypes": []
+ }
+ },
+ "target": {
+ "itemId": "searchForm",
+ "itemType": "form",
+ "scope": "digitall",
+ "properties": {}
+ }
+}
+----
+
+==== Update properties event type
+
+This event is usually used by user interfaces that make it possible to modify profile properties, for example a form where a user can edit his profile properties, or a management UI to modify.
+
+Note that this event type is a protected event type that is only accepted from configured third-party servers.
+
+===== Structure definition
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | updateProperties
+
+| source | the screen that has triggered the update to the profile properties
+
+| target | Not used (null)
+
+| properties | { targetId: the identifier of the profile to update targetType: “profile” if updating a profile or “persona” for personas add/update/delete: properties to be added/updated or deleted on the target profile}
+
+|===
+
+===== Example
+
+In this example, this “updateProperties” event contains properties that must be added to the targetId profile.
+
+image::update-properties-event-type.png[]
+
+[source,json]
+----
+{
+ "itemId": "d8fec330-33cb-42bc-a4e2-bb48ea7ed29b",
+ "itemType": "event",
+ "scope": null,
+ "eventType": "updateProperties",
+ "sessionId": "66e63ec9-66bc-4fac-8a8a-febcc3d6cbb7",
+ "profileId": "bc1e1238-a9ac-4b3a-8f63-5eec205cfcd5",
+ "timeStamp": "2020-01-31T08:51:15Z",
+ "properties": {
+ " *targetId* ": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ " *targetType* ": "profile",
+ " *add* ": {
+ "properties.phoneNumber": "+1-123-555-12345",
+ "properties.countryName": "US",
+ "properties.city": "Las Vegas",
+ "properties.address": "Hotel Flamingo",
+ "properties.zipCode": "89109",
+ "properties.email": "bill@acme.com"
+ }
+ },
+ "source": {
+ "itemId": "wemProfile",
+ "itemType": "wemProfile",
+ "scope": "digitall",
+ "properties": {}
+ },
+ "target": null
+}
+----
+
+==== Identify event type
+
+This event type is used to add information learned about the current profile.
+This could be through a form that has asked the user to provide some information about himself, or it could be information sent by another system (CRM, SSO, DMP, LiveRamp or equivalent) to augment the data for the current profile.
+
+It should be noted that, as in the case of a login event, it might be a good idea to be careful as to who and what system are allowed to send this event.
+Also, in order for this event to perform any modifications, an associated rule will be needed in the Unomi system to perform modifications to a profile (there is no default rule).
+
+|===
+| *Event type* | *Available publicly* | *Default rule* | *Targeted at back-office* | *Can remove/update properties*
+
+| identify | yes | no | no | no
+
+| updateProperties | no | yes | yes | yes
+
+|===
+
+The rule of thumb is: if you need to send profile data from public system to add information to a profile you should use the identify event type and add a rule to only process the data you want to accept.
+If you want to add/update/delete properties in a secure manner from a known server you could use the updateProperties but you should always check first if you can’t use the identify or event form event types with specific rules as this reduces greatly the potential for misuse.
+
+===== Structure description
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | identify
+
+| source | the site/application name that triggered the identify event
+
+| target | the user information contained in the event
+
+| properties | Not used (empty)
+
+|===
+
+===== Example
+
+In this example (coming from the Apache Unomi tracker example), an event containing additional information about the user (his nickname, favorite compiler and industry) was sent to Apache Unomi.
+
+[source,json]
+----
+{
+ "itemId": "18dfd6c7-9055-4ef0-a2eb-14c1482b4544",
+ "itemType": "event",
+ "scope": "myScope",
+ "eventType": "identify",
+ "sessionId": "928d9237-fb3d-4e53-cbee-1aeb1df7f03a",
+ "profileId": "temp_023ded50-bb43-4fe2-acbc-13bfa8de16de",
+ "timeStamp": "2020-01-15T14:13:25Z",
+ "properties": {},
+ "source": {
+ "itemId": "myScope",
+ "itemType": "site",
+ "scope": "myScope",
+ "properties": {
+ "page": {
+ "path": "/tracker/",
+ "referrer": "http://localhost:8181/",
+ "search": "",
+ "title": "Apache Unomi Web Tracker Test Page",
+ "url": "http://localhost:8181/tracker/"
+ }
+ }
+ },
+ "target": {
+ "itemId": "null",
+ "itemType": "analyticsUser",
+ "scope": "myScope",
+ "properties": {
+ "nickname": "Amazing Grace",
+ "favoriteCompiler": "A-0",
+ "industry": "Computer Science"
+ }
+ }
+}
+----
+
+==== Session created event type
+
+The session created event is an internal event created by Apache Unomi when a new session is created.
+This indicates that a new visitor has interacted with a system that is using Apache Unomi to track their behavior.
+
+===== Structure definition
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | sessionCreated
+
+| source | Not used (null)
+
+| target | the Session item that was created with all its fields and properties
+
+| properties | Not used (empty)
+
+|===
+
+===== Example
+
+In this example, a new session was created for a visitor coming to the digitall website.
+The session contains the firstVisit property.
+It may be augmented over time with more information including location.
+
+[source,json]
+----
+{
+ "itemId": "b3f5486f-b317-4182-9bf4-f497271e5363",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "sessionCreated",
+ "sessionId": "be416c08-8b9b-4611-990f-3a8bf3ed4e68",
+ "profileId": "bc1e1238-a9ac-4b3a-8f63-5eec205cfcd5",
+ "timeStamp": "2020-01-30T21:13:26Z",
+ "properties": {},
+ "source": null,
+ "target": {
+ "itemId": "be416c08-8b9b-4611-990f-3a8bf3ed4e68",
+ "itemType": "session",
+ "scope": "digitall",
+ "profileId": "bc1e1238-a9ac-4b3a-8f63-5eec205cfcd5",
+ "profile": {
+ "itemId": "bc1e1238-a9ac-4b3a-8f63-5eec205cfcd5",
+ "itemType": "profile",
+ "properties": {
+ "firstVisit": "2020-01-30T21:13:26Z"
+ },
+ "systemProperties": {},
+ "segments": [],
+ "scores": null,
+ "mergedWith": null,
+ "consents": {}
+ },
+ "properties": {},
+ "systemProperties": {},
+ "timeStamp": "2020-01-30T21:13:26Z",
+ "lastEventDate": null,
+ "size": 0,
+ "duration": 0
+ }
+}
+----
+
+==== Goal event type
+
+A goal event is triggered when the current profile (visitor) reaches a goal.
+
+===== Structure definition
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | goal
+
+| source | the Event that triggered the goal completion
+
+| target | the Goal item that was reached
+
+| properties | Not used (empty)
+
+|===
+
+===== Example
+
+In this example, a visitor has reached a goal by viewing a page called “sub-home” on the site “digitall” (event source).
+This goal event had the goal object as a target.
+The goal object (see Goal object later in this document) has a start event of creating a new session and a target event of a page view on the page “sub-home”.
+
+[source,json]
+----
+{
+ "itemId": "9fa70519-382d-412b-82ea-99b5989fbd0d",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "goal",
+ "sessionId": "42bd3fde-5fe9-4df6-8ae6-8550b8b06a7f",
+ "profileId": "3ec46b2c-fbaa-42d5-99df-54199c807fc8",
+ "timeStamp": "2017-05-29T23:49:16Z",
+ "properties": {},
+ "source": {
+ "itemId": "aadcd86c-9431-43c2-bdc3-06683ac25927",
+ "itemType": "event",
+ "scope": "digitall",
+ "eventType": "view",
+ "sessionId": "42bd3fde-5fe9-4df6-8ae6-8550b8b06a7f",
+ "profileId": "3ec46b2c-fbaa-42d5-99df-54199c807fc8",
+ "timeStamp": "2017-05-29T23:49:16Z",
+ "properties": {},
+ "source": {
+ "itemId": "6d5f4ae3-30c9-4561-81f3-06f82af7da1e",
+ "itemType": "site",
+ "scope": "digitall",
+ "properties": {}
+ },
+ "target": {
+ "itemId": "67dfc299-9b74-4264-a865-aebdc3482539",
+ "itemType": "page",
+ "scope": "digitall",
+ "properties": {
+ "pageInfo": {
+ "language": "en",
+ "destinationURL": "https://acme.com/home/sub-home.html",
+ "pageID": "67dfc299-9b74-4264-a865-aebdc3482539",
+ "pagePath": "/sites/digitall/home/sub-home",
+ "pageName": "sub-home",
+ "referringURL": "https://acme.com/home/perso-on-profile-past-event-page.html"
+ },
+ "category": {},
+ "attributes": {}
+ }
+ }
+ },
+ "target": {
+ "itemId": "_v4ref2mxg",
+ "itemType": "goal",
+ "startEvent": {
+ "parameterValues": {},
+ "type": "sessionCreatedEventCondition"
+ },
+ "targetEvent": {
+ "parameterValues": {
+ "pagePath": "/sites/digitall/home/sub-home"
+ },
+ "type": "pageViewEventCondition"
+ },
+ "campaignId": null,
+ "metadata": {
+ "id": "_v4ref2mxg",
+ "name": "sub-home-visit",
+ "description": "",
+ "scope": "digitall",
+ "tags": [
+ "pageVisitGoal"
+ ],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+ }
+}
+----
+
+==== Modify consent event type
+
+Consent type modification events are used to tell Unomi that consents were modified.
+A built-in rule will update the current profile with the consent modifications contained in the event.
+Consent events may be sent directly by a current profile to update their consents on the profile.
+
+===== Structure definition
+
+Based on the structure of the following object: Event
+
+|===
+| *Field name* | *Value/description*
+
+| eventType | modifyConsent
+
+| source | the page that has triggered the update the consents and that contains the different consent types the current profile could grant or deny
+
+| target | The consent that was modified
+
+| properties | The consent’s new value. See the Consent object type for more information.
+
+|===
+
+===== Example
+
+In this example, a user-generated a consent modification when visiting the home page, possibly by interacting with a consent form that captured his preferences.
+Different consent types were present on the page and he decided to GRANT the “mailchimp” consent.
+
+image::modify-consent-event-type.png[]
+
+[source,json]
+----
+{
+ "scope": "digitall",
+ "eventType": "modifyConsent",
+ "source": {
+ "itemType": "page",
+ "scope": "digitall",
+ "itemId": "f20836ab-608f-4551-a930-9796ec991340",
+ "properties": {
+ "pageInfo": {
+ "pageID": "f20836ab-608f-4551-a930-9796ec991340",
+ "nodeType": "jnt:page",
+ "pageName": "Home",
+ "pagePath": "/sites/digitall/home",
+ "templateName": "home",
+ "destinationURL": "http://localhost:8080/sites/digitall/home.html",
+ "referringURL": "http://localhost:8080/cms/render/default/en/sites/digitall/home.html",
+ "language": "en",
+ "categories": [],
+ "tags": [],
+ "isContentTemplate": false
+ },
+ "attributes": {},
+ "consentTypes": [
+ {
+ "typeIdentifier": "tracking",
+ "activated": true,
+ "title": "Allow tracking",
+ "description": "If approved we are allowed to track the visitor"
+ },
+ {
+ "typeIdentifier": "mailchimp",
+ "activated": true,
+ "title": "Mailchimp",
+ "description": "desc"
+ },
+ {
+ "typeIdentifier": "newsletter1",
+ "activated": true,
+ "title": "Newsletter 1",
+ "description": "desc"
+ },
+ {
+ "typeIdentifier": "newsletter2",
+ "activated": true,
+ "title": "Newsletter 2",
+ "description": "desc"
+ },
+ {
+ "typeIdentifier": "newsletter",
+ "activated": true,
+ "title": "Receive newsletter",
+ "description": "If approved we will send newsletter."
+ }
+ ]
+ }
+ },
+ "target": {
+ "itemType": "consent",
+ "scope": "digitall",
+ "itemId": "mailchimp"
+ },
+ "properties": {
+ "consent": {
+ "scope": "digitall",
+ "typeIdentifier": "mailchimp",
+ "status": "GRANTED",
+ "statusDate": "2020-01-31T20:10:00.463Z",
+ "revokeDate": "2022-01-30T20:10:00.463Z"
+ }
+ }
+}
+----
diff --git a/manual/src/main/asciidoc/concepts.adoc b/manual/src/main/asciidoc/concepts.adoc
deleted file mode 100644
index 1c0af11..0000000
--- a/manual/src/main/asciidoc/concepts.adoc
+++ /dev/null
@@ -1,311 +0,0 @@
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-Apache Unomi gathers information about users actions, information that is processed and stored by Unomi services. The
-collected information can then be used to personalize content, derive insights on user behavior, categorize the user
-profiles into segments along user-definable dimensions or acted upon by algorithms.
-
-=== Items and types
-
-Unomi structures the information it collects using the concept of `Item` which provides the base information (an
-identifier and a type) the context server needs to process and store the data. Items are persisted according to their
-type (structure) and identifier (identity). This base structure can be extended, if needed, using properties in the
-form of key-value pairs.
-
-These properties are further defined by the `Item`’s type definition which explicits the `Item`’s structure and
-semantics. By defining new types, users specify which properties (including the type of values they accept) are
-available to items of that specific type.
-
-Unomi defines default value types: `date`, `email`, `integer` and `string`, all pretty self-explanatory. While you can
-think of these value types as "primitive" types, it is possible to extend Unomi by providing additional value types.
-
-Additionally, most items are also associated to a scope, which is a concept that Unomi uses to group together related
-items. A given scope is represented in Unomi by a simple string identifier and usually represents an application or set
-of applications from which Unomi gathers data, depending on the desired analysis granularity. In the context of web
-sites, a scope could, for example, represent a site or family of related sites being analyzed. Scopes allow clients
-accessing the context server to filter data to only see relevant data.
-
-_Base `Item` structure:_
-
-[source,json]
-----
-{
- "itemType": <type of the item>,
- "scope": <scope>,
- "itemId": <item identifier>,
- "properties": <optional properties>
-}
-----
-
-Some types can be dynamically defined at runtime by calling to the REST API while other extensions are done via Unomi
-plugins. Part of extending Unomi, therefore, is a matter of defining new types and specifying which kind of Unomi
-entity (e.g. profiles) they can be affected to. For example, the following JSON document can be passed to Unomi to
-declare a new property type identified (and named) `tweetNb`, tagged with the `social` tag, targeting profiles and
-using the `integer` value type.
-
-_Example JSON type definition:_
-
-[source,json]
-----
-{
- "itemId": "tweetNb",
- "itemType": "propertyType",
- "metadata": {
- "id": "tweetNb",
- "name": "tweetNb",
- "systemTags": ["social"]
- },
- "target": "profiles",
- "type": "integer"
-}
-----
-
-____
-
-Unomi defines a built-in scope (called `systemscope`) that clients can use to share data across scopes.
-____
-
-=== Events
-
-Users' actions are conveyed from clients to the context server using events. Of course, the required information
-depends on what is collected and users' interactions with the observed systems but events minimally provide a type, a
-scope and source and target items. Additionally, events are timestamped. Conceptually, an event can be seen as a
-sentence, the event's type being the verb, the source the subject and the target the object.
-
-_Event structure:_
-
-[source,json]
-----
-{
- "eventType": <type of the event>,
- "scope": <scope of the event>,
- "source": <Item>,
- "target": <Item>,
- "properties": <optional properties>
-}
-----
-
-Source and target can be any Unomi item but are not limited to them. In particular, as long as they can be described
-using properties and Unomi’s type mechanism and can be processed either natively or via extension plugins, source and
-target can represent just about anything. Events can also be triggered as part of Unomi’s internal processes for example
-when a rule is triggered.
-
-Events are sent to Unomi from client applications using the JSON format and a typical page view event from a web site
-could look something like the following:
-
-_Example page view event:_
-
-[source,json]
-----
-{
- "eventType": "view",
- "scope": "ACMESPACE",
- "source": {
- "itemType": "site",
- "scope": "ACMESPACE",
- "itemId": "c4761bbf-d85d-432b-8a94-37e866410375"
- },
- "target": {
- "itemType": "page",
- "scope": "ACMESPACE",
- "itemId": "b6acc7b3-6b9d-4a9f-af98-54800ec13a71",
- "properties": {
- "pageInfo": {
- "pageID": "b6acc7b3-6b9d-4a9f-af98-54800ec13a71",
- "pageName": "Home",
- "pagePath": "/sites/ACMESPACE/home",
- "destinationURL": "http://localhost:8080/sites/ACMESPACE/home.html",
- "referringURL": "http://localhost:8080/",
- "language": "en"
- },
- "category": {},
- "attributes": {}
- }
- }
-}
-----
-
-=== Profiles
-
-By processing events, Unomi progressively builds a picture of who the user is and how they behave. This knowledge is
-embedded in `Profile` object. A profile is an `Item` with any number of properties and optional segments and scores.
-Unomi provides default properties to cover common data (name, last name, age, email, etc.) as well as default segments
-to categorize users. Unomi users are, however, free and even encouraged to create additional properties and segments to
-better suit their needs.
-
-Contrary to other Unomi items, profiles are not part of a scope since we want to be able to track the associated user
-across applications. For this reason, data collected for a given profile in a specific scope is still available to any
-scoped item that accesses the profile information.
-
-It is interesting to note that there is not necessarily a one to one mapping between users and profiles as users can be
-captured across applications and different observation contexts. As identifying information might not be available in
-all contexts in which data is collected, resolving profiles to a single physical user can become complex because
-physical users are not observed directly. Rather, their portrait is progressively patched together and made clearer as
-Unomi captures more and more traces of their actions. Unomi will merge related profiles as soon as collected data
-permits positive association between distinct profiles, usually as a result of the user performing some identifying
-action in a context where the user hadn’t already been positively identified.
-
-=== Sessions
-
-A session represents a time-bounded interaction between a user (via their associated profile) and a Unomi-enabled
-application. A session represents the sequence of actions the user performed during its duration. For this reason,
-events are associated with the session during which they occurred. In the context of web applications, sessions are
-usually linked to HTTP sessions.
-
-=== Segments
-
-Segments are used to group profiles together, and are based on conditions that are executed on profiles to determine if
-they are part of a segment or not. This also means that a profile may enter or leave a segment based on changes in their
-properties, making segments a highly dynamic concept.
-
-Here is an example of a simple segment definition registered using the REST API:
-
-[source]
-----
-curl -X POST http://localhost:8181/cxs/segments \
---user karaf:karaf \
--H "Content-Type: application/json" \
--d @- <<'EOF'
-{
- "metadata": {
- "id": "leads",
- "name": "Leads",
- "scope": "systemscope",
- "description": "You can customize the list below by editing the leads segment.",
- "readOnly":true
- },
- "condition": {
- "type": "booleanCondition",
- "parameterValues": {
- "operator" : "and",
- "subConditions": [
- {
- "type": "profilePropertyCondition",
- "parameterValues": {
- "propertyName": "properties.leadAssignedTo",
- "comparisonOperator": "exists"
- }
- }
- ]
- }
- }
-}
-EOF
-----
-
-For more details on the conditions and how they are structured using conditions, see the next section.
-
-=== Conditions
-
-Conditions are a very useful notion inside of Apache Unomi, as they are used as the basis for multiple other objects.
-Conditions may be used as parts of:
-
-- Segments
-- Rules
-- Queries
-- Campaigns
-- Goals
-- Profile filters
-
-A condition is composed of two basic elements:
-
-- a condition type identifier
-- a list of parameter values for the condition, that can be of any type, and in some cases may include sub-conditions
-
-A condition type identifier is a string that contains a unique identifier for a condition type. Example condition types
-may include `booleanCondition`, `eventTypeCondition`, `eventPropertyCondition`, and so on. Plugins may implement new
-condition types that may implement any logic that may be needed. The parameter values are simply lists of objects that
-may be used to configure the condition. In the case of a `booleanCondition` for example one of the parameter values will
-be an `operator` that will contain values such as `and` or `or` and a second parameter value called `subConditions`
-that contains a list of conditions to evaluate with that operator. The result of a condition is always a boolean
-value of true or false.
-
-Apache Unomi provides quite a lot of built-in condition types, including boolean types that make it possible to
-compose conditions using operators such as `and`, `or` or `not`. Composition is an essential element of building more
-complex conditions.
-
-Here is an example of a complex condition:
-
-[source,json]
-----
-{
- "condition": {
- "type": "booleanCondition",
- "parameterValues": {
- "operator":"or",
- "subConditions":[
- {
- "type": "eventTypeCondition",
- "parameterValues": {
- "eventTypeId": "sessionCreated"
- }
- },
- {
- "type": "eventTypeCondition",
- "parameterValues": {
- "eventTypeId": "sessionReassigned"
- }
- }
- ]
- }
- }
-}
-----
-
-As we can see in the above example we use the boolean `or` condition to check if the event type is of type `sessionCreated`
-or `sessionReassigned`.
-
-For a more complete list of available conditions, see the <<Built-in conditions>> reference section.
-
-=== Rules
-
-image::unomi-rule-engine.png[Unomi Rule Engine]
-
-Apache Unomi has a built-in rule engine that is one of the most important components of its architecture. Every time
-an event is received by the server, it is evaluated against all the rules and the ones matching the incoming event will
-be executed. You can think of a rule as a structure that looks like this:
-
- when
- conditions
- then
- actions
-
-Basically when a rule is evaluated, all the conditions in the `when` part are evaluated and if the result matches
-(meaning it evaluates to `true`) then the actions will be executed in sequence.
-
-The real power of Apache Unomi comes from the fact that `conditions` and `actions` are fully pluggeable and that plugins
-may implement new conditions and/or actions to perform any task. You can imagine conditions checking incoming event data
-against third-party systems or even against authentication systesm, and actions actually pulling or pushing data to third-party
-systems.
-
-For example the Salesforce CRM connector is simply a set of actions that pull and push data into the CRM. It is then
-just a matter of setting up the proper rules with the proper conditions to determine when and how the data will be
-pulled or pushed into the third-party system.
-
-==== Actions
-
-Actions are executed by rules in a sequence, and an action is only executed once the previous action has finished
-executing. If an action generates an exception, it will be logged and the execution sequence will continue unless in the
-case of a Runtime exception (such as a NullPointerException).
-
-Actions are implemented as Java classes, and as such may perform any kind of tasks that may include calling web hooks,
-setting profile properties, extracting data from the incoming request (such as resolving location from an IP address),
-or even pulling and/or pushing data to third-party systems such as a CRM server.
-
-Apache Unomi also comes with built-in actions. You may find the list of built-in actions in the <<Built-in actions>> section.
-
-=== Request flow
-
-Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
-
-image::unomi-request.png[Unomi request overview]
diff --git a/manual/src/main/asciidoc/conditions.adoc b/manual/src/main/asciidoc/conditions.adoc
index 52a01b8..eaa8219 100644
--- a/manual/src/main/asciidoc/conditions.adoc
+++ b/manual/src/main/asciidoc/conditions.adoc
@@ -12,9 +12,9 @@
// limitations under the License.
//
-=== Built-in conditions
+=== Built-in condition types
-Apache Unomi comes with an extensive collection of built-in conditions. Instead of detailing them one by one you will
+Apache Unomi comes with an extensive collection of built-in condition types. Instead of detailing them one by one you will
find here an overview of what a JSON condition descriptor looks like:
[source,json]
@@ -53,14 +53,14 @@ find here an overview of what a JSON condition descriptor looks like:
}
----
-Note that conditions have two important identifiers:
+Note that condition types have two important identifiers:
- conditionEvaluator
- queryBuilder
-This is because conditions can either be used to build queries or to evaluate a condition in real time. When implementing
+This is because condition types can either be used to build queries or to evaluate a condition in real time. When implementing
a new condition type, both implementations much be provided. Here's an example an OSGi Blueprint registration for the
-above condition descriptor:
+above condition type descriptor:
From https://github.com/apache/unomi/blob/master/plugins/baseplugin/src/main/resources/OSGI-INF/blueprint/blueprint.xml
@@ -85,9 +85,9 @@ From https://github.com/apache/unomi/blob/master/plugins/baseplugin/src/main/res
...
----
-As you can see two Java classes are used to build a single condition. You don't need to understand all these details in
-order to use conditions, but this might be interesting to know if you're interested in building your own condition
-implementations. For more details on building your own custom plugins/extensions, please refer to the corresponding
+As you can see two Java classes are used to build a single condition type. You don't need to understand all these details in
+order to use condition types, but this might be interesting to know if you're interested in building your own condition
+type implementations. For more details on building your own custom plugins/extensions, please refer to the corresponding
sections.
==== Existing condition descriptors
@@ -97,7 +97,7 @@ discover more. But the list below should get you started with the most useful co
- https://github.com/apache/unomi/tree/master/plugins/baseplugin/src/main/resources/META-INF/cxs/conditions
-Of course it is also possible to build your own custom conditions by developing custom Unomi plugins/extensions.
+Of course it is also possible to build your own custom condition types by developing custom Unomi plugins/extensions.
-You will also note that some conditions can re-use a `parentCondition`. This is a way to inherit from another condition
+You will also note that some condition typess can re-use a `parentCondition`. This is a way to inherit from another condition
type to make them more specific.
\ No newline at end of file
diff --git a/manual/src/main/asciidoc/context-request-flow.adoc b/manual/src/main/asciidoc/context-request-flow.adoc
new file mode 100644
index 0000000..49458ed
--- /dev/null
+++ b/manual/src/main/asciidoc/context-request-flow.adoc
@@ -0,0 +1,18 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+=== Context Request flow
+
+Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
+
+image::unomi-request.png[Unomi request overview]
diff --git a/manual/src/main/asciidoc/datamodel.adoc b/manual/src/main/asciidoc/datamodel.adoc
new file mode 100755
index 0000000..507b6f7
--- /dev/null
+++ b/manual/src/main/asciidoc/datamodel.adoc
@@ -0,0 +1,1077 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+
+=== Data Model Overview
+
+Apache Unomi gathers information about users actions, information that is processed and stored by Unomi services.
+The collected information can then be used to personalize content, derive insights on user behavior, categorize the
+user profiles into segments along user-definable dimensions or acted upon by algorithms.
+
+The following data model only contains the classes and properties directly related to the most important objects of Apache Unomi.
+There are other classes that are less central to the functionality but all the major ones are represented in the diagram below:
+
+image::data-model-overview.png[]
+
+We will detail many of these classes in the document below.
+
+=== Scopes
+
+Scopes are not actually an object in the system, but simply unique strings that are used to "classify" objects.
+For example, when using scopes with a web content management system, a scope could be associated with a site identifier or even a host name.
+
+____
+
+Unomi defines a built-in scope (called `systemscope`) that clients can use to share data across scopes.
+____
+
+==== Example
+
+In the following example, the scope uses the unique identifier of a web site called “digitall”.
+
+[source]
+----
+{
+ ... other fields of an object type ...
+ “scope”: “digitall”
+ ... other fields of an object type ...
+}
+----
+
+=== Items
+
+Unomi structures the information it collects using the concept of `Item` which provides the base information (an identifier and a type) the context server needs to process and store the data.
+Items are persisted according to their type (structure) and identifier (identity).
+This base structure can be extended, if needed, using properties in the form of key-value pairs.
+
+These properties are further defined by the `Item`’s type definition which explicits the `Item`’s structure and semantics.
+By defining new types, users specify which properties (including the type of values they accept) are available to items of that specific type.
+
+Unomi defines default value types: `date`, `email`, `integer` and `string`, all pretty self-explanatory.
+While you can think of these value types as "primitive" types, it is possible to extend Unomi by providing additional value types.
+
+Additionally, most items are also associated to a scope, which is a concept that Unomi uses to group together related items.
+A given scope is represented in Unomi by a simple string identifier and usually represents an application or set of applications from which Unomi gathers data, depending on the desired analysis granularity.
+In the context of web sites, a scope could, for example, represent a site or family of related sites being analyzed.
+Scopes allow clients accessing the context server to filter data to only see relevant data.
+
+Items are a generic object, that is common to many objects in the data model.
+It contains the following fields, that are inherited by other objects that inherit from it.
+
+==== Structure definition
+
+Inherits all the fields from: n/a
+
+|===
+| *Field* | *Type* | *Description*
+
+| itemId | String | This field contains a unique identifier (usually a UUID) that uniquely identifies the item in the whole system. It should be unique to a Unomi installation
+
+| itemType | String | A string containing the subtype of this item. Examples are : event, profile, session, … any class that inherits from the Item class will have a unique and different itemType value.
+
+| scope | String (optional) | If present, this will contain a scope identifier. A scope is just a way to regroup objects notably for administrative purposes. For example, when integrating with a CMS a scope could be mapped to a website. The “system” scope value is reserved for values that are used internally by Apache Unomi
+
+|===
+
+=== Metadata
+
+The Metadata object is an object that contains additional information about an object.
+It is usually associated with an Item object (see MetadataItem below).
+
+==== Structure definition
+
+Inherits all the fields from: n/a
+
+|===
+| *Field* | *Type* | *Description*
+
+| id | String | This field contains a unique identifier (UUID) for the object the metadata object is attached to. It is usually a copy of the itemId field on an Item object.
+
+| name | String | A name for the associated object. Usually, this name will be displayed on the user interface
+
+| description | String (optional) | A description of the associated object. Will also usually be used in user interfaces
+
+| scope | String | The scope for the associated object.
+
+| tags | String array | A list of tags for the associated object, this list may be edited through a UI.
+
+| systemTags | String array | A (reserved) list of tags for the associated object. This is usually populated through JSON descriptors and is not meant to be modified by end users. These tags may include values such as “profileProperties” that help classify associated objects.
+
+| enabled | Boolean | Indicates whether the associated is enabled or not. For example, a rule may be disabled using this field.
+
+| missingPlugins | Boolean | This is used for associated objects that require plugins to be deployed to work. If the plugin is not deployed, this object will not perform its function. For example if a rule is registered but the condition or actions it needs are not installed, the rule will not be used.
+
+| hidden | Boolean | Specifies whether the associated object should be visible in UIs or not
+
+| readOnly | Boolean | Specifies whether editing of the associated object should be allowed or not.
+
+|===
+
+==== Example
+
+This example of a Metadata object structure was taken from a List associated object.
+See the MetadataItem to understand how the two fit together.
+
+[source,json]
+----
+{
+ "id": "firstListId",
+ "name": "First list",
+ "description": "Description of the first list.",
+ "scope": "digitall",
+ "tags": [],
+ "systemTags": [],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+}
+----
+
+=== MetadataItem
+
+==== Structure definition
+
+Inherits all the fields from: Item
+
+|===
+| *Field* | *Type* | *Description*
+
+| metadata | Metadata | This object contains just one field, of type Metadata as define just before this object type.
+
+|===
+
+==== Example
+
+The following example is actually the definition of a List object, which is simply a MetadataItem sub-type with no additional fields.
+We can see here the “itemId” and “itemType” fields that come from the Item parent class and the “metadata” field that contains the object structure coming from the Metadata object type.
+
+[source,json]
+----
+{
+ "itemId": "userListId",
+ "itemType": "userList",
+ "metadata": {
+ "id": "userListId",
+ "name": "First list",
+ "description": "Description of the first list.",
+ "scope": "digitall",
+ "tags": [],
+ "systemTags": [],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+}
+----
+
+=== Events
+
+Events represent something that is happening at a specific time (they are timestamped).
+They can be used to track visitor behavior, or even for back-channel system-to-system (as for example for a login) communication.
+Examples of events may include a click on a link on a web page, a login, a form submission, a page view or any other time-stamped action that needs to be tracked.
+
+Events are persisted and immutable, and may be queried or aggregated to produce powerful reports.
+
+Events can also be triggered as part of Unomi’s internal processes for example
+when a rule is triggered.
+
+==== Fields
+
+Inherits all the fields from: Item
+
+|===
+| *Field* | *Type* | *Description*
+
+| eventType | String | Contains an identifier for the field type, which may be any value as Apache Unomi does not come with strict event type definitions and accepts custom events types. The system comes with built-in event types such as “view”, “form”, “login”, “updateProperties” but additional event types may of course be used by developers integrating with Apache Unomi.
+
+| sessionId | String | The unique identifier of a Session object
+
+| profileId | String | The unique identifier of a Profile object
+
+| timestamp | Date | The precise date at which the Event was received by Unomi. This date is in the https://en.wikipedia.org/wiki/ISO_8601[ISO 8601] format.
+
+| scope | String | (Optional, event type specific) An identifier for a scope
+
+| source | Item | An Item that is the source of the event. For example a web site, an application name, a web page
+
+| target | Item | An Item that is the target of the event. For example a button, a link, a file or a page
+
+| properties | Map<String,Object> | Properties for the event. These will change depending on the event type.
+
+|===
+
+==== Event types
+
+Event types are completely open, and any new event type will be accepted by Apache Unomi.
+
+Apache Unomi also comes with an extensive list of built-in event types you can find in the reference section of this manual.
+
+=== Profile
+
+By processing events, Unomi progressively builds a picture of who the user is and how they behave. This knowledge is
+embedded in `Profile` object. A profile is an `Item` with any number of properties and optional segments and scores.
+Unomi provides default properties to cover common data (name, last name, age, email, etc.) as well as default segments
+to categorize users. Unomi users are, however, free and even encouraged to create additional properties and segments to
+better suit their needs.
+
+Contrary to other Unomi items, profiles are not part of a scope since we want to be able to track the associated user
+across applications. For this reason, data collected for a given profile in a specific scope is still available to any
+scoped item that accesses the profile information.
+
+It is interesting to note that there is not necessarily a one to one mapping between users and profiles as users can be
+captured across applications and different observation contexts. As identifying information might not be available in
+all contexts in which data is collected, resolving profiles to a single physical user can become complex because
+physical users are not observed directly. Rather, their portrait is progressively patched together and made clearer as
+Unomi captures more and more traces of their actions. Unomi will merge related profiles as soon as collected data
+permits positive association between distinct profiles, usually as a result of the user performing some identifying
+action in a context where the user hadn’t already been positively identified.
+
+==== Structure definition
+
+Inherits all the fields from: Item
+
+|===
+| *Field name* | *Type* | *Description*
+
+| properties | Map<String,Object> | All the (user-facing) properties for the profile
+
+| systemProperties | Map<String,Object> | Internal properties used to track things such as goals reached, merges with other profiles, lists the profile belongs to.
+
+| segments | String set | A set of Segment identifiers that profile is (currently) associated with
+
+| scores | Map<String,Integer> | A map of scores with the score identifier as the key and the score total value as the value.
+
+| mergedWith | String | If merged with another profile, the profile identifier to the master profile is stored here
+
+| consents | Map<String,Consent> | The consents for the profile, as a map with the consent identifier as a key and the Consent object type as a value.
+
+|===
+
+==== Example
+
+In the example below, a profile for a visitor called “Bill Galileo” is detailed.
+A lot of user properties (such as first name, last name, gender, job title and more) were copied over from the CMS upon initial login.
+The profile is also part of 4 segments (leads, contacts, gender_male, age_60_70) and has a lot of different scores as well.
+It is also part of a list (systemProperties.lists), and has granted two consents for receiving newsletters.
+It has also been engaged in some goals (systemProperties.goals.*StartReached) and completed some goals (systemProperties.goals.*TargetReached)
+
+image::profile.png[]
+
+[source,json]
+----
+{
+ "itemId": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ "itemType": "profile",
+ "properties": {
+ "lastName": "Galileo",
+ "preferredLanguage": "en",
+ "nbOfVisits": 2,
+ "gender": "male",
+ "jobTitle": "Vice President",
+ "lastVisit": "2020-01-31T08:41:22Z",
+ "j:title": "mister",
+ "j:about": "<p> Lorem Ipsum dolor sit amet,consectetur adipisicing elit, sed doeiusmod tempor incididunt ut laboreet dolore magna aliqua. Ut enim adminim veniam, quis nostrudexercitation ullamco laboris nisi utaliquip ex ea commodo consequat.Duis aute irure dolor inreprehenderit in coluptate velit essecillum dolore eu fugiat nulla pariatur.Excepteur sint occaecat cupidatatnon proident, sunt in culpa quiofficia deserunt mollit anim id estlaborum.</p> ",
+ "firstName": "Bill",
+ "pageViewCount": {
+ "digitall": 19
+ },
+ "emailNotificationsDisabled": "true",
+ "company": "Acme Space",
+ "j:nodename": "bill",
+ "j:publicProperties": "j:about,j:firstName,j:function,j:gender,j:lastName,j:organization,j:picture,j:title",
+ "firstVisit": "2020-01-30T21:18:12Z",
+ "phoneNumber": "+1-123-555-12345",
+ "countryName": "US",
+ "city": "Las Vegas",
+ "address": "Hotel Flamingo",
+ "zipCode": "89109",
+ "email": "bill@acme.com",
+ "maritalStatus": "Married",
+ "birthDate": "1959-08-12T23:00:00.000Z",
+ "kids": 2,
+ "age": 60,
+ "income": 1000000,
+ "facebookId": "billgalileo",
+ "twitterId": "billgalileo",
+ "linkedInId": "billgalileo",
+ "leadAssignedTo": "Important Manager",
+ "nationality": "American"
+ },
+ "systemProperties": {
+ "mergeIdentifier": "bill",
+ " *lists* ": [
+ "userListId"
+ ],
+"goals": {
+ "viewLanguagePageGoalTargetReached": "2020-02-10T19:30:31Z",
+ "downloadGoalExampleTargetReached": "2020-02-10T15:22:41Z",
+ "viewLandingPageGoalStartReached": "2020-02-10T19:30:27Z",
+ "downloadGoalExampleStartReached": "2020-02-10T19:30:27Z",
+ "optimizationTestGoalStartReached": "2020-02-10T19:30:27Z"
+}
+ },
+ "segments": [
+ "leads",
+ "age_60_70",
+ "gender_male",
+ "contacts"
+ ],
+ "scores": {
+ "scoring_9": 10,
+ "scoring_8": 0,
+ "scoring_1": 10,
+ "scoring_0": 10,
+ "_s02s6220m": 0,
+ "scoring_3": 10,
+ "_27ir92oa2": 0,
+ "scoring_2": 10,
+ "scoring_5": 10,
+ "scoring_4": 10,
+ "scoring_7": 10,
+ "scoring_6": 10,
+ "_86igp9j1f": 1,
+ "_ps8d573on": 0
+ },
+ "mergedWith": null,
+ "consents": {
+ "digitall/newsletter1": {
+ "scope": "digitall",
+ "typeIdentifier": "newsletter1",
+ "status": "GRANTED",
+ "statusDate": "2019-05-15T14:47:28Z",
+ "revokeDate": "2021-05-14T14:47:28Z"
+ },
+ "digitall/newsletter2": {
+ "scope": "digitall",
+ "typeIdentifier": "newsletter2",
+ "status": "GRANTED",
+ "statusDate": "2019-05-15T14:47:28Z",
+ "revokeDate": "2021-05-14T14:47:28Z"
+ }
+ }
+}
+----
+
+=== Personas
+
+TODO
+
+==== Structure definition
+
+==== Example
+
+=== Consent
+
+A consent represents a single instance of a consent granted/refused or revoked by a profile.
+A profile will contain multiple instances of consent identified by unique identifiers.
+
+==== Structure definition
+
+|===
+| *Field name* | *Type* | *Description*
+
+| scope | String | The scope this consent is associated with. In the case of a website this might be the unique identifier for the site.
+
+| typeIdentifier | String | This is a unique consent type identifier, basically a unique name for the consent. Example of such types might include: “newsletter”, “personalization”, “tracking”.
+
+| status | GRANTED / DENIED / REVOKED | A copy of the profile associated with the session
+
+| statusDate | Date | The date (in ISO 8601 format) at which the current status was set
+
+| revokeDate | Date | The date (in ISO 8106 format) at which time the current status is automatically revoked.
+
+|===
+
+==== Example
+
+In this example, the consent called “newsletter” was given on the “digitall” website.
+
+[source,json]
+----
+{
+ "scope": "digitall",
+ "typeIdentifier": "newsletter",
+ "status": "GRANTED",
+ "statusDate": "2019-05-15T14:47:28Z",
+ "revokeDate": "2021-05-14T14:47:28Z"
+}
+----
+
+=== Session
+
+A session represents a period of time during which a visitor/profile has been active.
+It makes it possible to gather data and then use it for reporting and further analysis by regrouping all the events that occurred during the session.
+
+==== Structure definition
+
+Inherits all the fields from: Item
+
+|===
+| *Field name* | *Type* | *Description*
+
+| properties | Map<String,Object> | All the properties for the session. These contain information such as the browser, operating system and device used, as well as information about the location of the visitor.
+
+| systemProperties | Map<String,Object> | Not used (empty)
+
+| profile | Profile | A copy of the profile associated with the session
+
+| size | Integer | The number of view event types received during this session
+
+| duration | Integer | The duration of the session in milliseconds
+
+| lastEventDate | Date | The date of the last event that occurred in the session, in https://en.wikipedia.org/wiki/ISO_8601[ISO 8601] format.
+
+|===
+
+==== Example
+
+In this example the session contains a copy of the profile of the visitor.
+It is a visitor that has previously authentified in a CMS and who’se information was copied at the time of login from the CMS user account to the profile.
+You can also notice that the session contains the information coming from the browser’s user agent which contains the browser type, version as well as the operating system used.
+The visitor’s location is also resolve based on the IP address that was used to send events.
+
+[source,json]
+----
+{
+ "itemId": "4dcb5b74-6923-45ae-861a-6399ef88a209",
+ "itemType": "session",
+ "scope": "digitall",
+ "profileId": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ "profile": {
+ "itemId": "f7d1f1b9-4415-4ff1-8fee-407b109364f7",
+ "itemType": "profile",
+ "properties": {
+ "preferredLanguage": "en",
+ "nbOfVisits": 2,
+ "gender": "male",
+ "jobTitle": "Vice President",
+ "lastVisit": "2020-01-31T08:41:22Z",
+ "j:title": "mister",
+ "j:about": "<p> Lorem Ipsum dolor sit amet,consectetur adipisicing elit, sed doeiusmod tempor incididunt ut laboreet dolore magna aliqua. Ut enim adminim veniam, quis nostrudexercitation ullamco laboris nisi utaliquip ex ea commodo consequat.Duis aute irure dolor inreprehenderit in coluptate velit essecillum dolore eu fugiat nulla pariatur.Excepteur sint occaecat cupidatatnon proident, sunt in culpa quiofficia deserunt mollit anim id estlaborum.</p> ",
+ "pageViewCount": {
+ "digitall": 19
+ },
+ "emailNotificationsDisabled": "true",
+ "company": "Acme Space",
+ "j:publicProperties": "j:about,j:firstName,j:function,j:gender,j:lastName,j:organization,j:picture,j:title",
+ "firstVisit": "2020-01-30T21:18:12Z",
+ "countryName": "US",
+ "city": "Las Vegas",
+ "zipCode": "89109",
+ "maritalStatus": "Married",
+ "birthDate": "1959-08-12T23:00:00.000Z",
+ "kids": 25,
+ "age": 60,
+ "income": 1000000,
+ "leadAssignedTo": "Important Manager"
+ },
+ "systemProperties": {
+ "mergeIdentifier": "bill",
+ "lists": [
+ "_xb2bcm4wl"
+ ]
+ },
+ "segments": [
+ "leads",
+ "age_60_70",
+ "gender_male",
+ "contacts"
+ ],
+ "scores": {
+ "scoring_9": 10,
+ "scoring_8": 0,
+ "scoring_1": 10,
+ "scoring_0": 10,
+ "_s02s6220m": 0,
+ "scoring_3": 10,
+ "_27ir92oa2": 0,
+ "scoring_2": 10,
+ "scoring_5": 10,
+ "scoring_4": 10,
+ "scoring_7": 10,
+ "scoring_6": 10,
+ "_86igp9j1f": 1,
+ "_ps8d573on": 0
+ },
+ "mergedWith": null,
+ "consents": {}
+ },
+ "properties": {
+ "sessionCity": "Geneva",
+ "operatingSystemFamily": "Desktop",
+ "userAgentNameAndVersion": "Firefox@@72.0",
+ "countryAndCity": "Switzerland@@Geneva@@2660645@@6458783",
+ "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:72.0) Gecko/20100101 Firefox/72.0",
+ "userAgentName": "Firefox",
+ "sessionCountryCode": "CH",
+ "deviceName": null,
+ "sessionCountryName": "Switzerland",
+ "referringURL": "null",
+ "deviceCategory": "Apple Macintosh",
+ "pageReferringURL": "http://localhost:8080/sites/digitall/home/corporate-responsibility.html",
+ "userAgentVersion": "72.0",
+ "sessionAdminSubDiv2": 6458783,
+ "sessionAdminSubDiv1": 2660645,
+ "location": {
+ "lon": 6.1282508,
+ "lat": 46.1884341
+ },
+ "sessionIsp": "Cablecom",
+ "operatingSystemName": "Mac OS X",
+ "deviceBrand": "Apple"
+ },
+ "systemProperties": {},
+ "timeStamp": "2020-01-31T08:41:22Z",
+ "lastEventDate": "2020-01-31T08:53:32Z",
+ "size": 19,
+ "duration": 730317
+}
+----
+
+=== Segment
+
+Segments are used to group profiles together, and are based on conditions that are executed on profiles to determine
+if they are part of a segment or not.
+
+This also means that a profile may enter or leave a segment based on changes in their properties, making segments a
+highly dynamic concept.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Type* | *Description*
+
+| condition | Condition | The root condition for the segment. Conditions may be composed by using built-in conditions such as `booleanCondition` that can accept sub-conditions.
+
+|===
+
+==== Example
+
+[source,json]
+----
+{
+ "itemId": "age_20_30",
+ "itemType": "segment",
+ "condition": {
+ "parameterValues": {
+ "subConditions": [
+ {
+ "parameterValues": {
+ "propertyName": "properties.age",
+ "comparisonOperator": "greaterThanOrEqualTo",
+ "propertyValueInteger": 20
+ },
+ "type": "profilePropertyCondition"
+ },
+ {
+ "parameterValues": {
+ "propertyName": "properties.age",
+ "comparisonOperator": "lessThan",
+ "propertyValueInteger": 30
+ },
+ "type": "profilePropertyCondition"
+ }
+ ],
+ "operator": "and"
+ },
+ "type": "booleanCondition"
+ },
+ "metadata": {
+ "id": "age_20_30",
+ "name": "age_20_30",
+ "description": null,
+ "scope": "digitall",
+ "tags": [],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+}
+----
+
+Here is an example of a simple segment definition registered using the REST API:
+
+[source]
+----
+curl -X POST http://localhost:8181/cxs/segments \
+--user karaf:karaf \
+-H "Content-Type: application/json" \
+-d @- <<'EOF'
+{
+ "metadata": {
+ "id": "leads",
+ "name": "Leads",
+ "scope": "systemscope",
+ "description": "You can customize the list below by editing the leads segment.",
+ "readOnly":true
+ },
+ "condition": {
+ "type": "booleanCondition",
+ "parameterValues": {
+ "operator" : "and",
+ "subConditions": [
+ {
+ "type": "profilePropertyCondition",
+ "parameterValues": {
+ "propertyName": "properties.leadAssignedTo",
+ "comparisonOperator": "exists"
+ }
+ }
+ ]
+ }
+ }
+}
+EOF
+----
+
+For more details on the conditions and how they are structured using conditions, see the next section.
+
+=== Condition
+
+Conditions are a very useful notion inside of Apache Unomi, as they are used as the basis for multiple other objects.
+Conditions may be used as parts of:
+
+- Segments
+- Rules
+- Queries
+- Campaigns
+- Goals
+- Profile filters
+
+The result of a condition is always a boolean value of true or false.
+
+Apache Unomi provides quite a lot of built-in condition types, including boolean types that make it possible to compose conditions using operators such as `and`, `or` or `not`.
+Composition is an essential element of building more complex conditions.
+
+For a more complete list of available conditions, see the <<Built-in conditions>> reference section.
+
+==== Structure definition
+
+Inherits all the fields from: n/a
+
+|===
+| *Field name* | *Type* | *Description*
+
+| conditionTypeId | String | A condition type identifier is a string that contains a unique identifier for a condition
+type. Example condition types may include `booleanCondition`, `eventTypeCondition`, `eventPropertyCondition`, and so on.
+Plugins may implement new condition types that may implement any logic that may be needed.
+
+| parameterValues | Map<String,Object> | The parameter values are simply key-value paris that may be used to configure the condition.
+In the case of a `booleanCondition` for example one of the parameter values will be an `operator` that will contain values such as `and` or `or` and a second parameter value called `subConditions`
+that contains a list of conditions to evaluate with that operator.
+
+|===
+
+==== Example
+
+Here is an example of a complex condition:
+
+[source,json]
+----
+{
+ "condition": {
+ "type": "booleanCondition",
+ "parameterValues": {
+ "operator":"or",
+ "subConditions":[
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionCreated"
+ }
+ },
+ {
+ "type": "eventTypeCondition",
+ "parameterValues": {
+ "eventTypeId": "sessionReassigned"
+ }
+ }
+ ]
+ }
+ }
+}
+----
+
+As we can see in the above example we use the boolean `or` condition to check if the event type is of type `sessionCreated`
+or `sessionReassigned`.
+
+=== Rule
+
+image::unomi-rule-engine.png[Unomi Rule Engine]
+
+Apache Unomi has a built-in rule engine that is one of the most important components of its architecture.
+Every time an event is received by the server, it is evaluated against all the rules and the ones matching the incoming event will be executed.
+You can think of a rule as a structure that looks like this:
+
+ when
+ conditions
+ then
+ actions
+
+Basically when a rule is evaluated, all the conditions in the `when` part are evaluated and if the result matches (meaning it evaluates to `true`) then the actions will be executed in sequence.
+
+The real power of Apache Unomi comes from the fact that `conditions` and `actions` are fully pluggeable and that plugins may implement new conditions and/or actions to perform any task.
+You can imagine conditions checking incoming event data against third-party systems or even against authentication systesm, and actions actually pulling or pushing data to third-party systems.
+
+For example the Salesforce CRM connector is simply a set of actions that pull and push data into the CRM. It is then just a matter of setting up the proper rules with the proper conditions to determine when and how the data will be pulled or pushed into the third-party system.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Type* | *Description*
+
+| condition | Condition | The root condition for the rule. Conditions may be composed by using built-in conditions such as `booleanCondition` that can accept sub-conditions.
+
+| action | Action array | A list of <<Action>> object that will be executed if the condition is true.
+
+| linkedItems | String array | A list of references to objects that may have generated this rule. Goals and segments dynamically generate rules to react to incoming events. It is not recommend to manipulate rules that have linkedItems as it may break functionality.
+
+| raiseEventOnlyOnceForProfile | Boolean | If true, the rule will only be executed once for a given profile and a matching event. Warning: this functionality has a performance impact since it looks up past events.
+
+| raiseEventOnlyOnceForSession | Boolean | If true, the rule will only be executed once for a given session and a matching event. Warning: this functionality has a performance impact since it looks up past events.
+
+| priority | Integer | The priority for the rule. The lower the priority value the higher the effective priority (they are sorted by ascending order of priority)
+
+|===
+
+==== Example
+
+In this exmample we should the default `updateProperties` built-in rule that matches the `updateProperties` event and
+executes the built-in `updatePropertiesAction`
+
+[source,json]
+----
+{
+ "itemId": "updateProperties",
+ "itemType": "rule",
+ "condition": {
+ "parameterValues": {},
+ "type": "updatePropertiesEventCondition"
+ },
+ "actions": [
+ {
+ "parameterValues": {},
+ "type": "updatePropertiesAction"
+ }
+ ],
+ "linkedItems": null,
+ "raiseEventOnlyOnceForProfile": false,
+ "raiseEventOnlyOnceForSession": false,
+ "priority": 0,
+ "metadata": {
+ "id": "updateProperties",
+ "name": "Update profile/persona properties",
+ "description": "Update profile/persona properties",
+ "scope": "systemscope",
+ "tags": [],
+ "systemTags": [],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": true
+ }
+}
+----
+
+=== Action
+
+Actions are executed by rules in a sequence, and an action is only executed once the previous action has finished executing.
+If an action generates an exception, it will be logged and the execution sequence will continue unless in the case of a
+Runtime exception (such as a NullPointerException).
+
+Action use Action types that are implemented as Java classes, and as such may perform any kind of tasks that may include
+calling web hooks, setting profile properties, extracting data from the incoming request (such as resolving location from
+an IP address), or even pulling and/or pushing data to third-party systems such as a CRM server.
+
+Apache Unomi also comes with built-in actions.
+You may find the list of built-in actions in the <<Built-in actions>> section.
+
+==== Structure definition
+
+Inherits all the fields from: n/a
+
+|===
+| *Field name* | *Type* | *Description*
+
+| actionTypeId | String | An action type identifier is a string that contains a unique identifier for a action type.
+
+| parameterValues | Map<String,Object> | The parameter values are simply key-value paris that may be used to configure the action.
+
+|===
+
+==== Example
+
+In this example of an action, taking from the `form-mapping-example.json` rule, the `setPropertyAction` action is used
+to set the `properties.firstName` profile property to a value read from the event properties called `properties.firstName`.
+The `setPropertyStrategy` is a parameter specific to this action that allows to define if existing values should be
+overridden or not.
+
+[source,json]
+----
+ {
+ "type": "setPropertyAction",
+ "parameterValues": {
+ "setPropertyName": "properties(firstName)",
+ "setPropertyValue": "eventProperty::properties(firstName)",
+ "setPropertyStrategy": "alwaysSet"
+ }
+ }
+----
+
+=== Lists
+
+Lists are a “manual” way to organize profiles, whereas Segments are a dynamic way to regroup them.
+List objects actually only define the list in terms of name, description and other metadata but the list of members is actually not represented in the object.
+The profiles contain references to the lists in their “systemProperties.lists” property.
+This property is an array of list identifiers so in order to retrieve all the list names for a given profile, a lookup of List objects is required using the identifiers.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Description*
+
+| | No additional fields are present in this object type
+
+|===
+
+==== Example
+
+Here’s an example of a list called “First list”, along with its description, its scope, tags, etc.. . As a List object is basically a MetadataItem sub-class it simply has all the fields defined in that parent class.
+Note that the List does not contain Profiles, it is Profiles that reference the Lists, not the reverse.
+
+[source,json]
+----
+{
+ "itemId": "userListId",
+ "itemType": "userList",
+ "metadata": {
+ "id": "userListId",
+ "name": "First list",
+ "description": "Description of the first list.",
+ "scope": "digitall",
+ "tags": [],
+ "systemTags": [],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+}
+----
+
+=== Goals
+
+A goal can be defined with two conditions: a start event condition and an target event condition.
+Basically the goal will be “active” when its start event condition is satisfied, and “reached” when the target event condition is true.
+Goals may also (optionally) be associated with Campaigns.
+Once a goal is “reached”, a “goal” event triggered and the profile that is currently interacting with the system will see its system properties updated to indicate which goal has been reached.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Type* | *Description*
+
+| startEvent | Condition | The condition that will be used to determine if this goal was activated by the current profile
+
+| targetEvent | Condition | The condition that will be used to determine if the current profile has reached the goal.
+
+| campaignId | String | If this goal was setup as part of a Campaign, the unique identifier for the campaign is stored in this field.
+
+|===
+
+==== Example
+
+In the following example, a goal called “downloadGoalExample” is started when a new session is created (we use the “sessionCreatedEventCondition” for that) and is reached when a profile downloads a file called “ACME_WP.pdf” (that’s what the “downloadEventCondition” means).
+
+[source,json]
+----
+{
+ "itemId": "downloadGoalExample",
+ "itemType": "goal",
+ "startEvent": {
+ "parameterValues": {},
+ "type": "sessionCreatedEventCondition"
+ },
+ "targetEvent": {
+ "parameterValues": {
+ "filePath": "/sites/digitall/files/PDF/Publications/ACME_WP.pdf"
+ },
+ "type": "downloadEventCondition"
+ },
+ "campaignId": "firstCampaignExample",
+ "metadata": {
+ "id": "downloadGoalExample",
+ "name": "downloadGoalExample",
+ "description": null,
+ "scope": "digitall",
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false,
+ "systemTags": [
+ "goal",
+ "downloadGoal"
+ ]
+ }
+}
+----
+
+=== Campaigns
+
+A Campaign object represents a digital marketing campaign, along with conditions to enter the campaign and a specific duration, target and costs.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Type* | *Description*
+
+| startDate | Date | The start date of the Campaign (in ISO 8601 format)
+
+| endDate | Date | The end date of the Campaign (in ISO 8601 format)
+
+| entryCondition | Condition | The condition that must be satisfied for a profile to become a participant in the campaign
+
+| cost | Double | An indicative cost for the campaign
+
+| currency | String | The currency code (3-letter) for the cost of the campaign
+
+| primaryGoal | String | A unique identifier of the primary Goal for the campaign.
+
+| timezone | String | The timezone of the campaign identified by the TZ database name (see https://en.wikipedia.org/wiki/List_of_tz_database_time_zones)
+
+|===
+
+==== Example
+
+In the following example a campaign that starts January 1st 31, 2020 at 8:38am and finished on February 29th, 2020 at the same time has the following entry condition: the session duration must be less or equal to 3000 milliseconds (3 seconds) and the profile has viewed the “about” page on the “digitall” website.
+The cost of the campaign is USD 1’000’000 and the timezone is Europe/Zurich.
+The primary goal for the campaign is the goal we should as an example in the Goal section.
+
+[source,json]
+----
+{
+ "itemId": "firstCampaignExample",
+ "itemType": "campaign",
+ "startDate": "2020-01-31T08:38:00Z",
+ "endDate": "2020-02-29T08:38:00Z",
+ "entryCondition": {
+ "parameterValues": {
+ "subConditions": [
+ {
+ "parameterValues": {
+ "propertyName": "duration",
+ "comparisonOperator": "lessThanOrEqualTo",
+ "propertyValueInteger": 3000
+ },
+ "type": "sessionPropertyCondition"
+ },
+ {
+ "parameterValues": {
+ "pagePath": "/sites/digitall/home/about"
+ },
+ "type": "pageViewEventCondition"
+ }
+ ],
+ "operator": "and"
+ },
+ "type": "booleanCondition"
+ },
+ "cost": 1000000,
+ "currency": "USD",
+ "primaryGoal": "downloadGoalExample",
+ "timezone": "Europe/Zurich",
+ "metadata": {
+ "id": "firstCampaignExample",
+ "name": "firstCampaign",
+ "description": "Example of a campaign",
+ "scope": "digitall",
+ "tags": [],
+ "systemTags": [
+ "landing",
+ "campaign"
+ ],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+}
+----
+
+=== Scoring plans
+
+Scoring plans make it possible to define scores that will be tracked for profiles and use conditions to increment a score when the conditions are met.
+This makes it possible to then use threshold conditions on profiles when they reach a certain score.
+
+==== Structure definition
+
+Inherits all the fields from: MetadataItem
+
+|===
+| *Field name* | *Type* | *Description*
+
+| elements | ScoringElement array | A ScoringElement is composed of:* A Condition* A Score increment valueEach element defines a separate condition (tree) that will increment the defined score for this scoring plan, making it possible to have completely different conditions to augment a score
+
+|===
+
+==== Example
+
+In this example a scoring plan contains a single element that will increment a score with an increment one 1 once the profile has viewed at least 3 pages (using the “hasSeenNPagesCondition” condition).
+
+[source,json]
+----
+{
+ "itemId": "viewMoreThan3PagesId",
+ "itemType": "scoring",
+ "elements": [
+ {
+ "condition": {
+ "parameterValues": {
+ "value": 3,
+ "scope": "digitall",
+ "comparisonOperator": "greaterThanOrEqualTo"
+ },
+ "type": "hasSeenNPagesCondition"
+ },
+ "value": 1
+ }
+ ],
+ "metadata": {
+ "id": "viewMoreThan3PagesId",
+ "name": "Viewed more than 3 pages",
+ "description": null,
+ "scope": "digitall",
+ "tags": [],
+ "systemTags": [
+ "st:behavioral"
+ ],
+ "enabled": true,
+ "missingPlugins": false,
+ "hidden": false,
+ "readOnly": false
+ }
+}
+----
diff --git a/manual/src/main/asciidoc/extending-plugins.adoc b/manual/src/main/asciidoc/extending-plugins.adoc
deleted file mode 100644
index 0752396..0000000
--- a/manual/src/main/asciidoc/extending-plugins.adoc
+++ /dev/null
@@ -1,121 +0,0 @@
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-Unomi is architected so that users can provided extensions in the form of plugins.
-
-=== Types vs. instances
-
-Several extension points in Unomi rely on the concept of type: the extension defines a prototype for what the actual
-items will be once parameterized with values known only at runtime. This is similar to the concept of classes in
-object-oriented programming: types define classes, providing the expected structure and which fields are expected to
-be provided at runtime, that are then instantiated when needed with actual values.
-
-=== Plugin structure
-
-Being built on top of Apache Karaf, Unomi leverages OSGi to support plugins. A Unomi plugin is, thus, an OSGi
-bundle specifying some specific metadata to tell Unomi the kind of entities it provides. A plugin can provide the
-following entities to extend Unomi, each with its associated definition (as a JSON file), located in a specific spot
-within the `META-INF/cxs/` directory of the bundle JAR file:
-
-|====
-|Entity |Location in `cxs` directory
-
-|ActionType |actions
-|ConditionType |conditions
-|Persona |personas
-|PropertyMergeStrategyType |mergers
-|PropertyType |properties then profiles or sessions subdirectory then `<category name>` directory
-|Rule |rules
-|Scoring |scorings
-|Segment |segments
-|ValueType |values
-|====
-
-http://aries.apache.org/modules/blueprint.html[Blueprint] is used to declare what the plugin provides and inject
-any required dependency. The Blueprint file is located, as usual, at `OSGI-INF/blueprint/blueprint.xml` in the bundle JAR file.
-
-The plugin otherwise follows a regular maven project layout and should depend on the Unomi API maven artifact:
-
-[source,xml]
-----
-<dependency>
- <groupId>org.apache.unomi</groupId>
- <artifactId>unomi-api</artifactId>
- <version>...</version>
-</dependency>
-----
-
-Some plugins consists only of JSON definitions that are used to instantiate the appropriate structures at runtime
-while some more involved plugins provide code that extends Unomi in deeper ways.
-
-In both cases, plugins can provide more that one type of extension. For example, a plugin could provide both `ActionType`s and `ConditionType`s.
-
-=== Extension points
-
-==== ActionType
-
-`ActionType`s define new actions that can be used as consequences of Rules being triggered. When a rule triggers, it creates new actions based on the event data and the rule internal processes, providing values for parameters defined in the associated `ActionType`. Example actions include: “Set user property x to value y” or “Send a message to service x”.
-
-==== ConditionType
-
-`ConditionType`s define new conditions that can be applied to items (for example to decide whether a rule needs to be triggered or if a profile is considered as taking part in a campaign) or to perform queries against the stored Unomi data. They may be implemented in Java when attempting to define a particularly complex test or one that can better be optimized by coding it. They may also be defined as combination of other conditions. A simple condition could be: “User is male”, while a m [...]
-
-==== Persona
-
-A persona is a "virtual" profile used to represent categories of profiles, and may also be used to test how a personalized experience would look like using this virtual profile. A persona can define predefined properties and sessions. Persona definition make it possible to “emulate” a certain type of profile, e.g : US visitor, non-US visitor, etc.
-
-==== PropertyMergeStrategyType
-
-A strategy to resolve how to merge properties when merging profile together.
-
-==== PropertyType
-
-Definition for a profile or session property, specifying how possible values are constrained, if the value is multi-valued (a vector of values as opposed to a scalar value). `PropertyType`s can also be categorized using systemTags or file system structure, using sub-directories to organize definition files.
-
-==== Rule
-
-`Rule`s are conditional sets of actions to be executed in response to incoming events. Triggering of rules is guarded by a condition: the rule is only triggered if the associated condition is satisfied. That condition can test the event itself, but also the profile or the session. Once a rule triggers, a list of actions can be performed as consequences. Also, when rules trigger, a specific event is raised so that other parts of Unomi can react accordingly.
-
-==== Scoring
-
-`Scoring`s are set of conditions associated with a value to assign to profiles when matching so that the associated users can be scored along that dimension. Each scoring element is evaluated and matching profiles' scores are incremented with the associated value.
-
-==== Segments
-
-`Segment`s represent dynamically evaluated groups of similar profiles in order to categorize the associated users. To be considered part of a given segment, users must satisfies the segment’s condition. If they match, users are automatically added to the segment. Similarly, if at any given point during, they cease to satisfy the segment’s condition, they are automatically removed from it.
-
-==== Tag
-
-`Tag`s are simple labels that are used to classify all other objects inside Unomi.
-
-==== ValueType
-
-Definition for values that can be assigned to properties ("primitive" types).
-
-=== Other Unomi entities
-
-==== UserList
-
-User list are simple static lists of users. The associated profile stores the lists it belongs to in a specific property.
-
-==== Goal
-
-Goals represent tracked activities / actions that can be accomplished by site (or more precisely scope) visitors. These are tracked in general because they relate to specific business objectives or are relevant to measure site/scope performance.
-
-Goals can be defined at the scope level or in the context of a particular `Campaign`. Either types of goals behave exactly the same way with the exception of two notable differences:
- - duration: scope-level goals are considered until removed while campaign-level goals are only considered for the campaign duration
- - audience filtering: any visitor is considered for scope-level goals while campaign-level goals only consider visitors who match the campaign's conditions
-
-==== Campaign
-
-A goal-oriented, time-limited marketing operation that needs to be evaluated for return on investment performance by tracking the ratio of visits to conversions.
\ No newline at end of file
diff --git a/manual/src/main/asciidoc/images/data-model-overview.png b/manual/src/main/asciidoc/images/data-model-overview.png
new file mode 100755
index 0000000..8243b9e
Binary files /dev/null and b/manual/src/main/asciidoc/images/data-model-overview.png differ
diff --git a/manual/src/main/asciidoc/images/form-event-type.png b/manual/src/main/asciidoc/images/form-event-type.png
new file mode 100755
index 0000000..9312059
Binary files /dev/null and b/manual/src/main/asciidoc/images/form-event-type.png differ
diff --git a/manual/src/main/asciidoc/images/login-event-type.png b/manual/src/main/asciidoc/images/login-event-type.png
new file mode 100755
index 0000000..572bc8f
Binary files /dev/null and b/manual/src/main/asciidoc/images/login-event-type.png differ
diff --git a/manual/src/main/asciidoc/images/modify-consent-event-type.png b/manual/src/main/asciidoc/images/modify-consent-event-type.png
new file mode 100755
index 0000000..7329026
Binary files /dev/null and b/manual/src/main/asciidoc/images/modify-consent-event-type.png differ
diff --git a/manual/src/main/asciidoc/images/profile.png b/manual/src/main/asciidoc/images/profile.png
new file mode 100755
index 0000000..65db0ac
Binary files /dev/null and b/manual/src/main/asciidoc/images/profile.png differ
diff --git a/manual/src/main/asciidoc/images/update-properties-event-type.png b/manual/src/main/asciidoc/images/update-properties-event-type.png
new file mode 100755
index 0000000..65db0ac
Binary files /dev/null and b/manual/src/main/asciidoc/images/update-properties-event-type.png differ
diff --git a/manual/src/main/asciidoc/index.adoc b/manual/src/main/asciidoc/index.adoc
index 8c89727..a88b793 100644
--- a/manual/src/main/asciidoc/index.adoc
+++ b/manual/src/main/asciidoc/index.adoc
@@ -28,10 +28,6 @@ image::asf_logo_url.png[pdfwidth=35%,align=center]
include::5-min-quickstart.adoc[]
-== Concepts
-
-include::concepts.adoc[]
-
== First steps with Apache Unomi
include::getting-started.adoc[]
@@ -48,6 +44,8 @@ include::useful-unomi-urls.adoc[]
include::how-profile-tracking-works.adoc[]
+include::context-request-flow.adoc[]
+
== Queries and aggregations
include::queries-and-aggregations.adoc[]
@@ -70,9 +68,15 @@ include::clustering.adoc[]
== Reference
+include::datamodel.adoc[]
+
+include::new-data-model.adoc[]
+
+include::builtin-event-types.adoc[]
+
include::conditions.adoc[]
-include::actions.adoc[]
+include::action-types.adoc[]
== Integration samples
@@ -98,10 +102,6 @@ include::building-and-deploying.adoc[]
include::shell-commands.adoc[]
-include::extending-plugins.adoc[]
-
-include::custom-extensions.adoc[]
+include::writing-plugins.adoc[]
include::patches.adoc[]
-
-include::new-data-model.adoc[]
\ No newline at end of file
diff --git a/manual/src/main/asciidoc/new-data-model.adoc b/manual/src/main/asciidoc/new-data-model.adoc
index 48d19f5..0b98dfd 100644
--- a/manual/src/main/asciidoc/new-data-model.adoc
+++ b/manual/src/main/asciidoc/new-data-model.adoc
@@ -22,7 +22,7 @@ To be able to do so, we had to rework the way the data was stored inside Elastic
Previously every items was stored inside the same ElasticSearch index but this is not allowed anymore in recent ElasticSearch versions.
-Since Apache Unomi version 1.5.0 every type of items (see section: link:#_items_and_types[Items and types]) is now stored in a dedicated separated index.
+Since Apache Unomi version 1.5.0 every type of items (see section: link:#_items[Items]) is now stored in a dedicated separated index.
==== API changes
@@ -55,6 +55,14 @@ Previously named: `geonames` is now using the index name `context-geonameentry`
In order to migrate the data from ElasticSearch 5 to 7, Unomi provides a migration tool that is directly integrated.
+In these migration steps the following is assumed:
+
+- the ElasticSearch 5 installation is referred to as the `source`
+- the ElasticSearch 7 installation is referred to as the `target`
+- the Unomi 1.4 installation is completely stopped
+- the Unomi 1.5 installation has never been started (just uncompressed)
+- the Unomi 1.5 installation has been configured to connect to the `target` (ElasticSearch 7) cluster
+
It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the `source` cluster, and ideally to perform the
migration on a restore of the `source` cluster. For more information on ElasticSearch 5 snapshots and restore you can
find it here: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
diff --git a/manual/src/main/asciidoc/custom-extensions.adoc b/manual/src/main/asciidoc/writing-plugins.adoc
similarity index 62%
rename from manual/src/main/asciidoc/custom-extensions.adoc
rename to manual/src/main/asciidoc/writing-plugins.adoc
index cb170e4..f29daf8 100644
--- a/manual/src/main/asciidoc/custom-extensions.adoc
+++ b/manual/src/main/asciidoc/writing-plugins.adoc
@@ -11,41 +11,162 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//
-=== Custom extensions
+=== Writing Plugins
+
+Unomi is architected so that users can provided extensions in the form of plugins.
+
+=== Types vs. instances
+
+Several extension points in Unomi rely on the concept of type: a plugin defines a prototype for what the actual
+items will be once parameterized with values known only at runtime. This is similar to the concept of classes in
+object-oriented programming: types define classes, providing the expected structure and which fields are expected to
+be provided at runtime, that are then instantiated when needed with actual values.
+
+So for example we have the following types vs instances:
+
+- ConditionTypes vs Conditions
+- ActionTypes vs Actions
+- PropertyTypes vs Properties (for profiles and sessions)
+
+=== Plugin structure
+
+Being built on top of Apache Karaf, Unomi leverages OSGi to support plugins. A Unomi plugin is, thus, an OSGi
+bundle specifying some specific metadata to tell Unomi the kind of entities it provides. A plugin can provide the
+following entities to extend Unomi, each with its associated definition (as a JSON file), located in a specific spot
+within the `META-INF/cxs/` directory of the bundle JAR file:
+
+|====
+|Entity |Location in `cxs` directory
+
+|ActionType |actions
+|ConditionType |conditions
+|Persona |personas
+|PropertyMergeStrategyType |mergers
+|PropertyType |properties then profiles or sessions subdirectory then `<category name>` directory
+|Rule |rules
+|Scoring |scorings
+|Segment |segments
+|ValueType |values
+|====
+
+http://aries.apache.org/modules/blueprint.html[Blueprint] is used to declare what the plugin provides and inject
+any required dependency. The Blueprint file is located, as usual, at `OSGI-INF/blueprint/blueprint.xml` in the bundle JAR file.
+
+The plugin otherwise follows a regular maven project layout and should depend on the Unomi API maven artifact:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.unomi</groupId>
+ <artifactId>unomi-api</artifactId>
+ <version>...</version>
+</dependency>
+----
+
+Some plugins consists only of JSON definitions that are used to instantiate the appropriate structures at runtime
+while some more involved plugins provide code that extends Unomi in deeper ways.
+
+In both cases, plugins can provide more that one type of extension. For example, a plugin could provide both `ActionType`s and `ConditionType`s.
+
+=== Extension points
+
+In this section the value types that may be used as extension points are presented. Examples of these types will be
+given in the next section with more details.
+
+==== ActionType
+
+`ActionType`s define new actions that can be used as consequences of Rules being triggered. When a rule triggers, it
+creates new actions based on the event data and the rule internal processes, providing values for parameters defined
+in the associated `ActionType`. Example actions include: “Set user property x to value y” or “Send a message to service x”.
+
+==== ConditionType
+
+`ConditionType`s define new conditions that can be applied to items (for example to decide whether a rule needs to be
+triggered or if a profile is considered as taking part in a campaign) or to perform queries against the stored Unomi
+data. They may be implemented in Java when attempting to define a particularly complex test or one that can better be
+optimized by coding it. They may also be defined as combination of other conditions. A simple condition could be:
+“User is male”, while a more generic condition with parameters may test whether a given property has a specific value:
+“User property x has value y”.
+
+==== Persona
+
+A persona is a "virtual" profile used to represent categories of profiles, and may also be used to test how a
+personalized experience would look like using this virtual profile. A persona can define predefined properties and
+sessions. Persona definition make it possible to “emulate” a certain type of profile, e.g : US visitor, non-US visitor, etc.
+
+==== PropertyMergeStrategyType
+
+A strategy to resolve how to merge properties when merging profile together.
+
+==== PropertyType
+
+Definition for a profile or session property, specifying how possible values are constrained, if the value is
+multi-valued (a vector of values as opposed to a scalar value). `PropertyType`s can also be categorized using
+systemTags or file system structure, using sub-directories to organize definition files.
+
+==== Rule
+
+`Rule`s are conditional sets of actions to be executed in response to incoming events. Triggering of rules is guarded
+by a condition: the rule is only triggered if the associated condition is satisfied. That condition can test the
+event itself, but also the profile or the session. Once a rule triggers, a list of actions can be performed as
+consequences. Also, when rules trigger, a specific event is raised so that other parts of Unomi can react accordingly.
+
+==== Scoring
+
+`Scoring`s are set of conditions associated with a value to assign to profiles when matching so that the associated
+users can be scored along that dimension. Each scoring element is evaluated and matching profiles' scores are
+incremented with the associated value.
+
+==== Segments
+
+`Segment`s represent dynamically evaluated groups of similar profiles in order to categorize the associated users.
+To be considered part of a given segment, users must satisfies the segment’s condition. If they match, users are
+automatically added to the segment. Similarly, if at any given point during, they cease to satisfy the segment’s
+condition, they are automatically removed from it.
+
+==== Tag
+
+`Tag`s are simple labels that are used to classify all other objects inside Unomi.
+
+==== ValueType
+
+Definition for values that can be assigned to properties ("primitive" types).
+
+=== Custom plugins
Apache Unomi is a pluggeable server that may be extended in many ways. This document assumes you are familiar with the
<<_concepts,Apache Unomi concepts>> . This document is mostly a reference document on the different things that may
be used inside an extension. If you are looking for complete samples, please see the <<_samples,samples page>>.
-==== Creating an extension
+==== Creating a plugin
-An extension is simply a Maven project, with a Maven pom that looks like this:
+An plugin is simply a Maven project, with a Maven pom that looks like this:
[source]
----
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<groupId>org.apache.unomi</groupId>
- <artifactId>unomi-extensions</artifactId>
+ <artifactId>unomi-plugins</artifactId>
<version>${project.version}</version>
</parent>
<modelVersion>4.0.0</modelVersion>
- <artifactId>unomi-extension-example</artifactId>
- <name>Apache Unomi :: Extensions :: Example</name>
- <description>Service implementation for the Apache Unomi Context Server extension that integrates with the Geonames database</description>
+ <artifactId>unomi-plugin-example</artifactId>
+ <name>Apache Unomi :: Plugins :: Example</name>
+ <description>A sample example of a Unomi plugin</description>
<version>${project.version}</version>
<packaging>bundle</packaging>
<dependencies>
- <!-- This dependency is not required but generally used in extensions -->
+ <!-- This dependency is not required but generally used in plugins -->
<dependency>
<groupId>org.apache.unomi</groupId>
<artifactId>unomi-api</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
- </dependency>
+ </dependency>
</dependencies>
<build>
@@ -63,19 +184,21 @@ An extension is simply a Maven project, with a Maven pom that looks like this:
</Import-Package>
</instructions>
</configuration>
- </plugin>
+ </plugin>
</plugins>
</build>
</project>
----
-An extension may contain many different kinds of Apache Unomi objects, as well as custom OSGi services or anything that
+A plugin may contain many different kinds of Apache Unomi objects, as well as custom OSGi services or anything that
is needed to build your application.
==== Deployment and custom definition
-When you deploy a custom bundle with a custom definition (see "Predefined xxx" chapters under) for the first time, the definition will automatically be deployed at your bundle start event *if it does not exist*.
-After that if you redeploy the same bundle, the definition will not be redeployed, but you can redeploy it manually using the command `unomi:deploy-definition <bundleId> <fileName>` If you need to modify an existing
+When you deploy a custom bundle with a custom definition (see "Predefined xxx" chapters under) for the first time, the
+definition will automatically be deployed at your bundle start event *if it does not exist*.
+After that if you redeploy the same bundle, the definition will not be redeployed, but you can redeploy it manually
+using the command `unomi:deploy-definition <bundleId> <fileName>` If you need to modify an existing
definition when deploying the module, see <<Migration patches>>.
==== Predefined segments
@@ -150,7 +273,7 @@ In this example we provide a rule that will execute when a predefined composed c
"profileUpdatedEventCondition" is received. See below to see how predefined composed conditions are declared.
Once the condition is matched, the actions will be executed in sequence. In this example there is only a single
action of type "evaluateProfileSegmentsAction" that is defined so it will be executed by Apache Unomi's rule engine.
-You can also see below how custom actions may be defined.
+You can also see below how custom actions may be defined.
==== Predefined properties
@@ -192,7 +315,7 @@ Here is an example of a property definition JSON file
==== Predefined child conditions
You can define new predefined conditions that are actually conditions inheriting from a parent condition and setting
-pre-defined parameter values. You can do this by creating a JSON file in:
+pre-defined parameter values. You can do this by creating a JSON file in:
[source]
----
@@ -200,7 +323,7 @@ src/main/resources/META-INF/cxs/conditions
----
Here is an example of a JSON file that defines a profileUpdateEventCondition that inherits from a parent condition of
-type eventTypeCondition.
+type eventTypeCondition.
[source]
----
@@ -236,7 +359,7 @@ Personas may also be pre-defined by creating JSON files in the following directo
src/main/resources/META-INF/cxs/personas
----
-Here is an example of a persona definition JSON file:
+Here is an example of a persona definition JSON file:
[source]
----
@@ -278,10 +401,10 @@ Here is an example of a persona definition JSON file:
You can see that it's also possible to define sessions for personas.
-==== Custom actions
+==== Custom action types
-Custom actions are a powerful way to integrate with external systems by being able to define custom logic that will
-be executed by an Apache Unomi rule. An action is defined by a JSON file created in the following directory:
+Custom action types are a powerful way to integrate with external systems by being able to define custom logic that will
+be executed by an Apache Unomi rule. An action type is defined by a JSON file created in the following directory:
[source]
----
@@ -311,7 +434,7 @@ Here is an example of a JSON action definition:
"multivalued": true
}
]
-}
+}
----
The `actionExecutor` identifier refers to a service property that is defined in the OSGi Blueprint service registration.
@@ -347,17 +470,17 @@ You can note here the `actionExecutorId` that corresponds to the `actionExecutor
The implementation of the action is available here : https://github.com/apache/unomi/blob/master/extensions/lists-extension/actions/src/main/java/org/apache/unomi/lists/actions/AddToListsAction.java[org.apache.unomi.lists.actions.AddToListsAction]
-==== Custom conditions
+==== Custom condition types
-Custom conditions are different from predefined child conditions because they implement their logic using Java classes.
-They are also declared by adding a JSON file into the conditions directory:
+Custom condition types are different from predefined child conditions because they implement their logic using Java classes.
+They are also declared by adding a JSON file into the `conditions` directory:
[source]
----
src/main/resources/META-INF/cxs/conditions
----
-Here is an example of JSON custom condition definition:
+Here is an example of JSON custom condition type definition:
[source]
----
@@ -386,7 +509,7 @@ Here is an example of JSON custom condition definition:
Note the `conditionEvaluator` and the `queryBuilder` values. These reference OSGi service properties that are declared
in an OSGi Blueprint configuration file (other service definitions may also be used such as Declarative Services or even
Java registered services). Here is an example of an OSGi Blueprint definition corresponding to the above JSON condition
-definition file.
+type definition file.
[source]
----
@@ -413,7 +536,7 @@ src/main/resources/OSGI-INF/blueprint/blueprint.xml
</blueprint>
----
-You can find the implementation of the two classes here :
+You can find the implementation of the two classes here :
* https://github.com/apache/unomi/blob/master/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/MatchAllConditionESQueryBuilder.java[org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionESQueryBuilder]
* https://github.com/apache/unomi/blob/master/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/MatchAllConditionEvaluator.java[org.apache.unomi.plugins.baseplugin.conditions.MatchAllConditionEvaluator]