You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by sh...@apache.org on 2021/02/08 17:17:06 UTC
[unomi] branch unomi-1.5.x updated (f524d22 -> db21180)
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a change to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git.
from f524d22 UNOMI-417: fix profile persist when receivint multiple events that modify the profile (#240)
new d4d6562 Fix bug in profile service it (#222)
new d9372b0 Setup code analysis
new f16e473 Fix build broken after committing github configuration file.
new 55c8bcd UNOMI-371 add optional support for optimistic concurrency control (if_seq_no) (#223)
new de2a8cf UNOMI-371 Fix compilation error & other minor improvements
new 3503efc Skip tests in jenkins deploy stage (#227)
new 2d680cc UNOMI-409 add .gitignore to unomi-web-tracker (#228)
new f64e5c2 feat(updatePropertiesAction): support add values to a SET in a profile property (#230)
new 2f9ed6e UNOMI-413-mergeProfileOnPropertyAction: support take Events profile as master, fix persistence bug when new profile is merged and it was the event profile (#232)
new a2e2bb5 Fix URL for testing in twitter sample documentation (#234)
new b5b6a3b UNOMI-422 improve PropertiesUpdateActionIT (#241)
new 8b50dbf mergePropertyAction: when merge identifier change, if forceEventProfile, change identifier and do not create a new profile (#239)
new fd43611 UNOMI-424 Fix buildQuery for BooleanConditionESQueryBuilder (#243)
new 30e05c8 UNOMI-425 Fix IT (#245)
new db21180 Return error to the client when exception is thrown from Elasticsearch impl (#202)
The 15 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.github/workflows/codeql-analysis.yml | 67 ++++
Jenkinsfile | 4 +-
Jenkinsfile-jdk11 | 4 +-
api/src/main/java/org/apache/unomi/api/Item.java | 10 +
.../apache/unomi/services/UserListServiceImpl.java | 2 +-
.../unomi/privacy/internal/PrivacyServiceImpl.java | 2 +-
extensions/web-tracker/javascript/.gitignore | 1 +
extensions/web-tracker/javascript/package.json | 7 +-
.../web-tracker/javascript/{ => src}/snippet.js | 0
.../test/java/org/apache/unomi/itests/BasicIT.java | 1 -
.../apache/unomi/itests/ConditionEvaluatorIT.java | 105 +++---
.../org/apache/unomi/itests/ProfileServiceIT.java | 26 ++
.../itests/ProfileServiceWithoutOverwriteIT.java | 124 +++++++
.../unomi/itests/PropertiesUpdateActionIT.java | 132 ++++++-
.../1.3/asciidoc/samples/twitter-sample.adoc | 4 +-
.../1.4/asciidoc/samples/twitter-sample.adoc | 4 +-
.../1.5/asciidoc/samples/twitter-sample.adoc | 4 +-
.../src/main/asciidoc/samples/twitter-sample.adoc | 4 +-
.../main/resources/etc/custom.system.properties | 2 +
.../ElasticSearchPersistenceServiceImpl.java | 135 +++++--
.../resources/OSGI-INF/blueprint/blueprint.xml | 4 +
.../org.apache.unomi.persistence.elasticsearch.cfg | 8 +-
.../unomi/persistence/spi/PersistenceService.java | 64 +++-
.../actions/MergeProfilesOnPropertyAction.java | 35 +-
.../baseplugin/actions/UpdatePropertiesAction.java | 6 +
.../conditions/BooleanConditionESQueryBuilder.java | 6 +-
.../cxs/actions/mergeProfilesOnPropertyAction.json | 5 +
.../unomi/plugins/mail/actions/SendMailAction.java | 2 +-
pom.xml | 2 +
.../apache/unomi/rest/RuntimeExceptionMapper.java | 23 +-
.../services/impl/events/EventServiceImpl.java | 2 +-
.../services/impl/rules/RulesServiceImpl.java | 2 +-
.../services/impl/segments/SegmentServiceImpl.java | 12 +-
.../java/org/apache/unomi/web/ClientServlet.java | 38 +-
.../java/org/apache/unomi/web/ContextServlet.java | 389 +++++++++++----------
.../apache/unomi/web/EventsCollectorServlet.java | 24 +-
36 files changed, 914 insertions(+), 346 deletions(-)
create mode 100644 .github/workflows/codeql-analysis.yml
create mode 100644 extensions/web-tracker/javascript/.gitignore
rename extensions/web-tracker/javascript/{ => src}/snippet.js (100%)
create mode 100644 itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
copy extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ConfigRestExceptionMapper.java => rest/src/main/java/org/apache/unomi/rest/RuntimeExceptionMapper.java (60%)
[unomi] 01/15: Fix bug in profile service it (#222)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit d4d6562b11630aa1f527f4d39bfd5c613f8a74ec
Author: liatiusim <62...@users.noreply.github.com>
AuthorDate: Fri Dec 4 11:18:49 2020 +0100
Fix bug in profile service it (#222)
(cherry picked from commit cc1306f6b6475ce557bea40ac17f602c8b15587e)
---
.../org/apache/unomi/itests/ProfileServiceIT.java | 25 +++++++++++++++++++
.../ElasticSearchPersistenceServiceImpl.java | 21 ++++++++++++++++
.../unomi/persistence/spi/PersistenceService.java | 28 ++++++++++++++++++++++
3 files changed, 74 insertions(+)
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
index db4e735..d75f608 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
@@ -24,6 +24,8 @@ import org.apache.unomi.api.services.DefinitionsService;
import org.apache.unomi.api.PartialList;
import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.fail;
+
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.Before;
@@ -112,4 +114,27 @@ public class ProfileServiceIT extends BaseIT {
assertEquals(0, profiles.getList().size());
}
+ // Relevant only when throwExceptions system property is true
+ @Test
+ public void testGetProfileWithWrongScrollerIdThrowException() throws InterruptedException, NoSuchFieldException, IllegalAccessException {
+ boolean throwExceptionCurrent = (boolean) persistenceService.getSetting("throwExceptions");
+ persistenceService.setSetting("throwExceptions", true);
+
+ Query query = new Query();
+ query.setLimit(2);
+ query.setScrollTimeValidity("10m");
+ query.setScrollIdentifier("dummyScrollId");
+
+ try {
+ profileService.search(query, Profile.class);
+ fail("search method didn't throw when expected");
+ } catch (RuntimeException ex) {
+ // Should get here since this scenario should throw exception
+ }
+ finally {
+ persistenceService.setSetting("throwExceptions", throwExceptionCurrent);
+ }
+ }
+
+
}
diff --git a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
index f51a5c0..62aaa81 100644
--- a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
+++ b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
@@ -112,6 +112,7 @@ import javax.net.ssl.X509TrustManager;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
+import java.lang.reflect.Field;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
@@ -694,6 +695,26 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
}
@Override
+ public void setSettings(Map<String, Object> settings) throws NoSuchFieldException, IllegalAccessException {
+ for (Map.Entry<String, Object> setting : settings.entrySet())
+ setSetting(setting.getKey(), setting.getValue());
+ }
+
+ @Override
+ public void setSetting(String fieldName, Object value) throws NoSuchFieldException, IllegalAccessException {
+ Field field = this.getClass().getDeclaredField(fieldName);
+ field.set(getClass(), value);
+ }
+
+ @Override
+ public Object getSetting(String fieldName) throws NoSuchFieldException, IllegalAccessException {
+ Field field = this.getClass().getDeclaredField(fieldName);
+ return field.get(getClass());
+ }
+
+
+
+ @Override
public <T extends Item> T load(final String itemId, final Class<T> clazz) {
return load(itemId, null, clazz);
}
diff --git a/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java b/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
index ce7d28e..d85059e 100644
--- a/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
+++ b/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
@@ -80,6 +80,34 @@ public interface PersistenceService {
<T extends Item> PartialList<T> getAllItems(final Class<T> clazz, int offset, int size, String sortBy, String scrollTimeValidity);
/**
+ * Set settings of the persistence service
+ *
+ * @param settings map of setting name and it's value
+ * @throws NoSuchFieldException if the field does not exist
+ * @throws IllegalAccessException field is not accessible to be changed
+ */
+ void setSettings(Map<String, Object> settings) throws NoSuchFieldException, IllegalAccessException;
+
+ /**
+ * Set settings of the persistence service
+ *
+ * @param fieldName name of the field to set
+ * @param value value of the field to set
+ * @throws NoSuchFieldException if the field does not exist
+ * @throws IllegalAccessException field is not accessible to be changed
+ */
+ void setSetting(String fieldName, Object value) throws NoSuchFieldException, IllegalAccessException;
+
+ /**
+ * Set settings of the persistence service
+ *
+ * @param fieldName name of the field to get
+ * @throws NoSuchFieldException if the field does not exist
+ * @throws IllegalAccessException field is not accessible to be changed
+ */
+ Object getSetting(String fieldName) throws NoSuchFieldException, IllegalAccessException;
+
+ /**
* Persists the specified Item in the context server.
*
* @param item the item to persist
[unomi] 14/15: UNOMI-425 Fix IT (#245)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 30e05c8d8ecae13ed407862a7aa13bec9b22aac2
Author: MT BENTERKI <be...@gmail.com>
AuthorDate: Fri Feb 5 17:10:00 2021 +0100
UNOMI-425 Fix IT (#245)
* UNOMI-425 Fix GraphQLProfilePropertiesIT
* UNOMI-425 remove system.out.println
(cherry picked from commit 71783712ca08d8a2a68236245dcddc7e6f7ca5e6)
---
itests/src/test/java/org/apache/unomi/itests/BasicIT.java | 1 -
1 file changed, 1 deletion(-)
diff --git a/itests/src/test/java/org/apache/unomi/itests/BasicIT.java b/itests/src/test/java/org/apache/unomi/itests/BasicIT.java
index 0ae2feb..033b762 100644
--- a/itests/src/test/java/org/apache/unomi/itests/BasicIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/BasicIT.java
@@ -116,7 +116,6 @@ public class BasicIT extends BaseIT {
// by the connection manager.
String responseContent;
try (CloseableHttpResponse response = HttpClientBuilder.create().build().execute(request)) {
- System.out.println(response.getStatusLine());
HttpEntity entity = response.getEntity();
// do something useful with the response body
// and ensure it is fully consumed
[unomi] 09/15: UNOMI-413-mergeProfileOnPropertyAction: support take
Events profile as master,
fix persistence bug when new profile is merged and it was the event profile
(#232)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 2f9ed6ed8d32c9888c0bf2670076d3e7fd639eed
Author: giladw <gw...@yotpo.com>
AuthorDate: Wed Jan 6 17:20:34 2021 +0200
UNOMI-413-mergeProfileOnPropertyAction: support take Events profile as master, fix persistence bug when new profile is merged and it was the event profile (#232)
(cherry picked from commit 1c9b970d6d21ea3d2c6744292e748c32278b6876)
---
.../actions/MergeProfilesOnPropertyAction.java | 22 ++++++++++++++++++----
.../cxs/actions/mergeProfilesOnPropertyAction.json | 5 +++++
2 files changed, 23 insertions(+), 4 deletions(-)
diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
index ffdf626..b01990b 100644
--- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
+++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
@@ -132,11 +132,18 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
return StringUtils.isEmpty(mergeProfilePreviousPropertyValue) ? EventService.PROFILE_UPDATED : EventService.NO_CHANGE;
}
- // Use oldest profile for master profile
- final Profile masterProfile = profileService.mergeProfiles(profiles.get(0), profiles);
+ Profile markedMasterProfile;
+ boolean forceEventProfileAsMaster = action.getParameterValues().containsKey("forceEventProfileAsMaster") ?
+ (boolean) action.getParameterValues().get("forceEventProfileAsMaster") : false;
+ if (forceEventProfileAsMaster)
+ markedMasterProfile = event.getProfile();
+ else
+ markedMasterProfile = profiles.get(0);// Use oldest profile for master profile
+
+ final Profile masterProfile = profileService.mergeProfiles(markedMasterProfile, profiles);
// Profile has changed
- if (!masterProfile.getItemId().equals(profileId)) {
+ if (forceEventProfileAsMaster || !masterProfile.getItemId().equals(profileId)) {
HttpServletResponse httpServletResponse = (HttpServletResponse) event.getAttributes().get(Event.HTTP_RESPONSE_ATTRIBUTE);
// we still send back the current profile cookie. It will be changed on the next request to the ContextServlet.
// The current profile will be deleted only then because we cannot delete it right now (too soon)
@@ -192,7 +199,14 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
sourceMap.put("mergedWith", masterProfileId);
profile.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profile.getSystemProperties());
- persistenceService.update(profile, null, Profile.class, sourceMap);
+
+ boolean isExist = persistenceService.load(profile.getItemId(), Profile.class) != null;
+
+ if (isExist == false) //save the original event profile is it has been changed
+ persistenceService.save(profile);
+ else
+ persistenceService.update(profile, null, Profile.class, sourceMap,true);
+
}
}
} catch (Exception e) {
diff --git a/plugins/baseplugin/src/main/resources/META-INF/cxs/actions/mergeProfilesOnPropertyAction.json b/plugins/baseplugin/src/main/resources/META-INF/cxs/actions/mergeProfilesOnPropertyAction.json
index 6082f9f..07bf38c 100644
--- a/plugins/baseplugin/src/main/resources/META-INF/cxs/actions/mergeProfilesOnPropertyAction.json
+++ b/plugins/baseplugin/src/main/resources/META-INF/cxs/actions/mergeProfilesOnPropertyAction.json
@@ -15,6 +15,11 @@
"id": "mergeProfilePropertyName",
"type": "string",
"multivalued": false
+ },
+ {
+ "id": "forceEventProfileAsMaster",
+ "type": "boolean",
+ "multivalued": false
}
]
}
\ No newline at end of file
[unomi] 13/15: UNOMI-424 Fix buildQuery for
BooleanConditionESQueryBuilder (#243)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit fd43611153e00012044aa842a49e5a84ea8872ae
Author: MT BENTERKI <be...@gmail.com>
AuthorDate: Fri Feb 5 14:48:25 2021 +0100
UNOMI-424 Fix buildQuery for BooleanConditionESQueryBuilder (#243)
add and fix IT
(cherry picked from commit ad6742020714f2c3f63fe50cea41892caa8bb491)
---
.../apache/unomi/itests/ConditionEvaluatorIT.java | 105 ++++++++++++---------
.../conditions/BooleanConditionESQueryBuilder.java | 6 +-
2 files changed, 59 insertions(+), 52 deletions(-)
diff --git a/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java b/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java
index 5a2421e..a1f93a6 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ConditionEvaluatorIT.java
@@ -43,7 +43,6 @@ import static org.junit.Assert.*;
@RunWith(PaxExam.class)
@ExamReactorStrategy(PerSuite.class)
public class ConditionEvaluatorIT extends BaseIT {
-
protected ConditionBuilder builder;
protected Item item;
protected Date lastVisit;
@@ -61,8 +60,9 @@ public class ConditionEvaluatorIT extends BaseIT {
public void setUp() {
assertNotNull("Definition service should be available", definitionsService);
assertNotNull("Persistence service should be available", persistenceService);
+ builder = new ConditionBuilder(definitionsService);
- lastVisit = new GregorianCalendar(2015,1,1,20,30,0).getTime();
+ lastVisit = new GregorianCalendar(2015, Calendar.FEBRUARY,1,20,30,0).getTime();
Profile profile = new Profile("profile-" + UUID.randomUUID().toString());
profile.setProperty("firstVisit", lastVisit);
@@ -70,37 +70,34 @@ public class ConditionEvaluatorIT extends BaseIT {
profile.setProperty("gender", "female");
profile.setProperty("lastVisit", lastVisit);
profile.setProperty("randomStats", 0.15);
- profile.setSegments(new HashSet<String>(Arrays.asList("s1", "s2", "s3")));
- this.item = profile;
- builder = new ConditionBuilder(definitionsService);
-
+ profile.setSegments(new HashSet<>(Arrays.asList("s1", "s2", "s3")));
+ item = profile;
}
@Test
public void testCompound() {
// test AND
assertTrue(eval(builder.and(builder.profileProperty("properties.gender").equalTo("female"),
- builder.profileProperty("properties.age").equalTo(Integer.valueOf(30))).build()));
- //assertFalse(eval(builder.and(builder.profileProperty("properties.gender").equalTo("male"),
- // builder.profileProperty("properties.age").equalTo(Integer.valueOf(30))).build()));
- //assertFalse(eval(builder.and(builder.profileProperty("properties.gender").equalTo("female"),
- // builder.profileProperty("properties.age").equalTo(Integer.valueOf(40))).build()));
+ builder.profileProperty("properties.age").equalTo(30)).build()));
+ assertFalse(eval(builder.and(builder.profileProperty("properties.gender").equalTo("male"),
+ builder.profileProperty("properties.age").equalTo(30)).build()));
+ assertFalse(eval(builder.and(builder.profileProperty("properties.gender").equalTo("female"),
+ builder.profileProperty("properties.age").equalTo(40)).build()));
// test OR
assertTrue(eval(builder.or(builder.profileProperty("properties.gender").equalTo("female"),
- builder.profileProperty("properties.age").equalTo(Integer.valueOf(40))).build()));
+ builder.profileProperty("properties.age").equalTo(40)).build()));
assertTrue(eval(builder.or(builder.profileProperty("properties.gender").equalTo("male"),
- builder.profileProperty("properties.age").equalTo(Integer.valueOf(30))).build()));
- //assertFalse(eval(builder.or(builder.profileProperty("properties.gender").equalTo("male"),
- // builder.profileProperty("properties.age").equalTo(Integer.valueOf(40))).build()));
+ builder.profileProperty("properties.age").equalTo(30)).build()));
+ assertFalse(eval(builder.or(builder.profileProperty("properties.gender").equalTo("male"),
+ builder.profileProperty("properties.age").equalTo(40)).build()));
// test NOT
assertTrue(eval(builder.not(builder.profileProperty("properties.gender").equalTo("male")).build()));
- //assertFalse(eval(builder.not(builder.profileProperty("properties.age").equalTo(Integer.valueOf(30))).build()));
-
+ assertFalse(eval(builder.not(builder.profileProperty("properties.age").equalTo(30)).build()));
}
- //@Test
+ @Test
public void testDate() {
assertTrue(eval(builder.profileProperty("properties.lastVisit").equalTo(lastVisit).build()));
assertTrue(eval(builder.profileProperty("properties.lastVisit")
@@ -118,34 +115,31 @@ public class ConditionEvaluatorIT extends BaseIT {
.all(new Date(lastVisit.getTime() + 10000), lastVisit).build()));
assertTrue(eval(builder.profileProperty("properties.lastVisit").isDay(lastVisit).build()));
- assertTrue(eval(builder.profileProperty("properties.lastVisit").isNotDay(new Date(lastVisit.getTime() + 10000)).build()));
+ assertTrue(eval(builder.profileProperty("properties.lastVisit").isNotDay(new Date(lastVisit.getTime() + (24*60*60*1000))).build()));
}
@Test
public void testExistence() {
- /*assertTrue("Gender property does not exist",
- eval(builder.profileProperty("properties.gender").exists().build()));
+ assertTrue("Gender property does not exist", eval(builder.profileProperty("properties.gender").exists().build()));
assertFalse("Gender property missing", eval(builder.profileProperty("properties.gender").missing().build()));
assertTrue("Strange property exists", eval(builder.profileProperty("properties.unknown").missing().build()));
- assertFalse("Strange property exists", eval(builder.profileProperty("properties.unknown").exists().build()));*/
+ assertFalse("Strange property exists", eval(builder.profileProperty("properties.unknown").exists().build()));
}
@Test
public void testInteger() {
- assertTrue(eval(builder.profileProperty("properties.age").equalTo(Integer.valueOf(30)).build()));
- assertTrue(eval(builder.not(builder.profileProperty("properties.age").equalTo(Integer.valueOf(40))).build()));
- assertTrue(eval(builder.profileProperty("properties.age").notEqualTo(Integer.valueOf(40)).build()));
- assertTrue(eval(builder.profileProperty("properties.age").lessThan(Integer.valueOf(40)).build()));
- assertTrue(eval(builder.profileProperty("properties.age").greaterThan(Integer.valueOf(20)).build()));
- assertTrue(eval(builder.profileProperty("properties.age").greaterThanOrEqualTo(Integer.valueOf(30)).build()));
- //assertFalse(eval(builder.profileProperty("properties.age").greaterThanOrEqualTo(Integer.valueOf(31)).build()));
-
- assertTrue(eval(builder.profileProperty("properties.age").in(Integer.valueOf(30)).build()));
- assertTrue(eval(builder.profileProperty("properties.age").in(Integer.valueOf(31), Integer.valueOf(30)).build()));
- assertTrue(eval(builder.profileProperty("properties.age").notIn(Integer.valueOf(25), Integer.valueOf(26))
- .build()));
- //assertFalse(eval(builder.profileProperty("properties.age").notIn(Integer.valueOf(25), Integer.valueOf(30))
- // .build()));
+ assertTrue(eval(builder.profileProperty("properties.age").equalTo(30).build()));
+ assertTrue(eval(builder.not(builder.profileProperty("properties.age").equalTo(40)).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").notEqualTo(40).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").lessThan(40).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").greaterThan(20).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").greaterThanOrEqualTo(30).build()));
+ assertFalse(eval(builder.profileProperty("properties.age").greaterThanOrEqualTo(31).build()));
+
+ assertTrue(eval(builder.profileProperty("properties.age").in(30).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").in(31, 30).build()));
+ assertTrue(eval(builder.profileProperty("properties.age").notIn(25, 26).build()));
+ assertFalse(eval(builder.profileProperty("properties.age").notIn(25, 30).build()));
}
@Test
@@ -167,34 +161,51 @@ public class ConditionEvaluatorIT extends BaseIT {
public void testMultiValue() {
assertTrue(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "in")
.parameter("segments", "s10", "s20", "s2").build()));
- //assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "in")
- // .parameter("segments", "s10", "s20", "s30").build()));
+ assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "in")
+ .parameter("segments", "s10", "s20", "s30").build()));
assertTrue(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "notIn")
.parameter("segments", "s10", "s20", "s30").build()));
- //assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "notIn")
- // .parameter("segments", "s10", "s20", "s2").build()));
+ assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "notIn")
+ .parameter("segments", "s10", "s20", "s2").build()));
assertTrue(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "all")
.parameter("segments", "s1", "s2").build()));
- //assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "all")
- // .parameter("segments", "s1", "s5").build()));
+ assertFalse(eval(builder.property("profileSegmentCondition", "segments").parameter("matchType", "all")
+ .parameter("segments", "s1", "s5").build()));
}
@Test
public void testString() {
assertTrue(eval(builder.profileProperty("properties.gender").equalTo("female").build()));
- //assertFalse(eval(builder.not(builder.profileProperty("properties.gender").equalTo("female")).build()));
+ assertFalse(eval(builder.not(builder.profileProperty("properties.gender").equalTo("female")).build()));
assertTrue(eval(builder.profileProperty("properties.gender").notEqualTo("male").build()));
- //assertFalse(eval(builder.not(builder.profileProperty("properties.gender").notEqualTo("male")).build()));
+ assertFalse(eval(builder.not(builder.profileProperty("properties.gender").notEqualTo("male")).build()));
assertTrue(eval(builder.profileProperty("properties.gender").startsWith("fe").build()));
assertTrue(eval(builder.profileProperty("properties.gender").endsWith("le").build()));
assertTrue(eval(builder.profileProperty("properties.gender").contains("fem").build()));
- //assertFalse(eval(builder.profileProperty("properties.gender").contains("mu").build()));
+ assertFalse(eval(builder.profileProperty("properties.gender").contains("mu").build()));
assertTrue(eval(builder.profileProperty("properties.gender").matchesRegex(".*ale").build()));
assertTrue(eval(builder.profileProperty("properties.gender").in("male", "female").build()));
assertTrue(eval(builder.profileProperty("properties.gender").notIn("one", "two").build()));
- //assertFalse(eval(builder.profileProperty("properties.gender").notIn("one", "two", "female").build()));
+ assertFalse(eval(builder.profileProperty("properties.gender").notIn("one", "two", "female").build()));
assertTrue(eval(builder.profileProperty("properties.gender").all("female").build()));
- //assertFalse(eval(builder.profileProperty("properties.gender").all("male", "female").build()));
+ assertFalse(eval(builder.profileProperty("properties.gender").all("male", "female").build()));
+ }
+
+ @Test
+ public void testRange() {
+ // test AND
+ Condition condition = builder.and(
+ builder.profileProperty("properties.gender").equalTo("female"),
+ builder.profileProperty("properties.age").greaterThanOrEqualTo(40)
+ ).build();
+ assertFalse(eval(condition));
+
+ // test OR
+ condition = builder.or(
+ builder.profileProperty("properties.gender").equalTo("female"),
+ builder.profileProperty("properties.age").greaterThanOrEqualTo(40)
+ ).build();
+ assertTrue(eval(condition));
}
}
diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/BooleanConditionESQueryBuilder.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/BooleanConditionESQueryBuilder.java
index 8a7a381..574fcc8 100644
--- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/BooleanConditionESQueryBuilder.java
+++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/conditions/BooleanConditionESQueryBuilder.java
@@ -65,11 +65,7 @@ public class BooleanConditionESQueryBuilder implements ConditionESQueryBuilder {
} else {
QueryBuilder orFilter = dispatcher.buildFilter(conditions.get(i), context);
if (orFilter != null) {
- if (orFilter.getName().equals("range")) {
- boolQueryBuilder.filter(orFilter);
- } else {
- boolQueryBuilder.should(orFilter);
- }
+ boolQueryBuilder.should(orFilter);
} else {
logger.warn("Null filter for boolean OR sub condition " + conditions.get(i));
}
[unomi] 12/15: mergePropertyAction: when merge identifier change,
if forceEventProfile,
change identifier and do not create a new profile (#239)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 8b50dbf81144cec039a2dd807dc4800163a6225b
Author: giladw <gw...@yotpo.com>
AuthorDate: Fri Feb 5 15:44:43 2021 +0200
mergePropertyAction: when merge identifier change, if forceEventProfile, change identifier and do not create a new profile (#239)
(cherry picked from commit 107011a1d948db6ef8fc5b0217bd58afeb6a20b4)
---
.../baseplugin/actions/MergeProfilesOnPropertyAction.java | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
index b01990b..93036d7 100644
--- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
+++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
@@ -70,6 +70,9 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
final Session currentSession = event.getSession();
+ boolean forceEventProfileAsMaster = action.getParameterValues().containsKey("forceEventProfileAsMaster") ?
+ (boolean) action.getParameterValues().get("forceEventProfileAsMaster") : false;
+
// store the profile id in case the merge change it to a previous one
String profileId = profile.getItemId();
@@ -95,8 +98,12 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
profile = profiles.get(0);
} else {
// Create a new profile
- profile = new Profile(UUID.randomUUID().toString());
- profile.setProperty("firstVisit", event.getTimeStamp());
+ if (forceEventProfileAsMaster)
+ profile = event.getProfile();
+ else {
+ profile = new Profile(UUID.randomUUID().toString());
+ profile.setProperty("firstVisit", event.getTimeStamp());
+ }
profile.getSystemProperties().put(mergeProfilePropertyName, mergeProfilePropertyValue);
}
@@ -133,8 +140,6 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
}
Profile markedMasterProfile;
- boolean forceEventProfileAsMaster = action.getParameterValues().containsKey("forceEventProfileAsMaster") ?
- (boolean) action.getParameterValues().get("forceEventProfileAsMaster") : false;
if (forceEventProfileAsMaster)
markedMasterProfile = event.getProfile();
else
[unomi] 11/15: UNOMI-422 improve PropertiesUpdateActionIT (#241)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit b5b6a3b3c6e2ccefc54b14192b46dd476ec90db0
Author: MT BENTERKI <be...@gmail.com>
AuthorDate: Wed Feb 3 10:40:04 2021 +0100
UNOMI-422 improve PropertiesUpdateActionIT (#241)
(cherry picked from commit 40cac9bb99698fc35c2de5877e00c12c9671fe40)
---
.../unomi/itests/PropertiesUpdateActionIT.java | 132 +++++++++++++++++++--
1 file changed, 124 insertions(+), 8 deletions(-)
diff --git a/itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java b/itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
index 1e64b91..f1b6304 100644
--- a/itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/PropertiesUpdateActionIT.java
@@ -35,9 +35,7 @@ import org.slf4j.LoggerFactory;
import javax.inject.Inject;
import java.io.IOException;
-import java.util.Date;
-import java.util.HashMap;
-import java.util.Map;
+import java.util.*;
/**
* Created by amidani on 12/10/2017.
@@ -57,7 +55,7 @@ public class PropertiesUpdateActionIT extends BaseIT {
protected EventService eventService;
@Before
- public void setUp() throws IOException {
+ public void setUp() throws IOException, InterruptedException {
Profile profile = new Profile();
profile.setItemId(PROFILE_TEST_ID);
profileService.save(profile);
@@ -67,6 +65,8 @@ public class PropertiesUpdateActionIT extends BaseIT {
profileTarget.setItemId(PROFILE_TARGET_TEST_ID);
profileService.save(profileTarget);
LOGGER.info("Profile saved with ID [{}].", profileTarget.getItemId());
+
+ refreshPersistence();
}
@Test
@@ -81,16 +81,19 @@ public class PropertiesUpdateActionIT extends BaseIT {
propertyToUpdate.put("properties.firstName", "UPDATED FIRST NAME CURRENT PROFILE");
updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_UPDATE, propertyToUpdate);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TARGET_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+
int changes = eventService.send(updateProperties);
LOGGER.info("Changes of the event : {}", changes);
Assert.assertTrue(changes > 0);
+ Assert.assertEquals("UPDATED FIRST NAME CURRENT PROFILE", profile.getProperty("firstName"));
}
@Test
public void testUpdateProperties_NotCurrentProfile() {
-
Profile profile = profileService.load(PROFILE_TARGET_TEST_ID);
Profile profileToUpdate = profileService.load(PROFILE_TEST_ID);
Assert.assertNull(profileToUpdate.getProperty("firstName"));
@@ -104,12 +107,125 @@ public class PropertiesUpdateActionIT extends BaseIT {
updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_UPDATE, propertyToUpdate);
updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
- int changes = eventService.send(updateProperties);
-
- LOGGER.info("Changes of the event : {}", changes);
+ eventService.send(updateProperties);
profileToUpdate = profileService.load(PROFILE_TEST_ID);
Assert.assertEquals("UPDATED FIRST NAME", profileToUpdate.getProperty("firstName"));
+ }
+
+ @Test
+ public void testUpdateProperties_CurrentProfile_PROPS_TO_ADD() throws InterruptedException {
+ Profile profile = profileService.load(PROFILE_TEST_ID);
+
+ Event updateProperties = new Event("updateProperties", null, profile, null, null, profile, new Date());
+ updateProperties.setPersistent(false);
+
+ Map<String, Object> propertyToAdd = new HashMap<>();
+ propertyToAdd.put("properties.prop1", "New property 1");
+ propertyToAdd.put("properties.prop2", "New property 2");
+ propertyToAdd.put("properties.prop3", "New property 3");
+
+ updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_ADD, propertyToAdd);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+ eventService.send(updateProperties);
+ profileService.save(profile);
+ refreshPersistence();
+
+ profile = profileService.load(PROFILE_TEST_ID);
+ Assert.assertEquals("New property 1", profile.getProperty("prop1"));
+ Assert.assertEquals("New property 2", profile.getProperty("prop2"));
+ Assert.assertEquals("New property 3", profile.getProperty("prop3"));
+ }
+
+ @Test
+ public void testUpdateProperties_CurrentProfile_PROPS_TO_ADD_TO_SET() throws InterruptedException {
+ Profile profile = profileService.load(PROFILE_TEST_ID);
+ Event updateProperties = new Event("updateProperties", null, profile, null, null, profile, new Date());
+ updateProperties.setPersistent(false);
+
+ Map<String, Object> propertyToAddToSet = new HashMap<>();
+ propertyToAddToSet.put("properties.prop1", "New property 1");
+ propertyToAddToSet.put("properties.prop2", "New property 2");
+ propertyToAddToSet.put("properties.prop3", "New property 3");
+
+ updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_ADD, propertyToAddToSet);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+ eventService.send(updateProperties);
+ profileService.save(profile);
+ refreshPersistence();
+
+ profile = profileService.load(PROFILE_TEST_ID);
+ Assert.assertEquals("New property 1", profile.getProperty("prop1"));
+ Assert.assertEquals("New property 2", profile.getProperty("prop2"));
+ Assert.assertEquals("New property 3", profile.getProperty("prop3"));
+
+ // Add set and check
+ propertyToAddToSet = new HashMap<>();
+ propertyToAddToSet.put("properties.prop1", "New property 1 bis");
+ propertyToAddToSet.put("properties.prop3", "New property 3 bis");
+
+ updateProperties = new Event("updateProperties", null, profile, null, null, profile, new Date());
+ updateProperties.setPersistent(false);
+ updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_ADD_TO_SET, propertyToAddToSet);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+ eventService.send(updateProperties);
+ profileService.save(profile);
+ refreshPersistence();
+
+ profile = profileService.load(PROFILE_TEST_ID);
+ Assert.assertEquals(2, ((List<String>) profile.getProperty("prop1")).size());
+ Assert.assertEquals(2, ((List<String>) profile.getProperty("prop3")).size());
+ Assert.assertEquals("New property 1", ((List<String>) profile.getProperty("prop1")).get(1));
+ Assert.assertEquals("New property 2", profile.getProperty("prop2"));
+ Assert.assertEquals("New property 3 bis", ((List<String>) profile.getProperty("prop3")).get(0));
+ }
+
+ @Test
+ public void testUpdateProperties_CurrentProfile_PROPS_TO_DELETE() throws InterruptedException {
+ Profile profile = profileService.load(PROFILE_TEST_ID);
+ Event updateProperties = new Event("updateProperties", null, profile, null, null, profile, new Date());
+ updateProperties.setPersistent(false);
+
+ Map<String, Object> propertyToAdd = new HashMap<>();
+ propertyToAdd.put("properties.prop1", "New property 1");
+ propertyToAdd.put("properties.prop1bis", "New property 1 bis");
+ propertyToAdd.put("properties.prop2", "New property 2");
+ propertyToAdd.put("properties.prop3", "New property 3");
+
+ updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_ADD, propertyToAdd);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+ eventService.send(updateProperties);
+ profileService.save(profile);
+ refreshPersistence();
+
+ profile = profileService.load(PROFILE_TEST_ID);
+ Assert.assertEquals("New property 1", profile.getProperty("prop1"));
+ Assert.assertEquals("New property 1 bis", profile.getProperty("prop1bis"));
+ Assert.assertEquals("New property 2", profile.getProperty("prop2"));
+ Assert.assertEquals("New property 3", profile.getProperty("prop3"));
+
+ // Delete property and check
+ List<String> propertyToDelete = new ArrayList<>();
+ propertyToDelete.add("properties.prop1bis");
+
+ updateProperties = new Event("updateProperties", null, profile, null, null, profile, new Date());
+ updateProperties.setPersistent(false);
+ updateProperties.setProperty(UpdatePropertiesAction.PROPS_TO_DELETE, propertyToDelete);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_ID_KEY, PROFILE_TEST_ID);
+ updateProperties.setProperty(UpdatePropertiesAction.TARGET_TYPE_KEY, "profile");
+
+ eventService.send(updateProperties);
+ profileService.save(profile);
+ refreshPersistence();
+ profile = profileService.load(PROFILE_TEST_ID);
+ Assert.assertNull(profile.getProperty("prop1bis"));
+ Assert.assertEquals("New property 1", profile.getProperty("prop1"));
+ Assert.assertEquals("New property 2", profile.getProperty("prop2"));
+ Assert.assertEquals("New property 3", profile.getProperty("prop3"));
}
}
[unomi] 05/15: UNOMI-371 Fix compilation error & other minor
improvements
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit de2a8cf99a86b8df263227ee5434448d66d9e4bd
Author: Serge Huber <sh...@jahia.com>
AuthorDate: Tue Jan 5 14:30:40 2021 +0100
UNOMI-371 Fix compilation error & other minor improvements
(cherry picked from commit fbfaba2b8c920264e5a5faebf2c8cc74e606ceb9)
---
.../apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
index 5c2ed5b..bcd427e 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
@@ -96,16 +96,17 @@ public class ProfileServiceWithoutOverwriteIT extends BaseIT {
public void testSaveProfileWithoutOverwriteSavesAfterReload() throws InterruptedException {
Profile profile = setupWithoutOverwriteTests();
String profileId = profile.getItemId();
- Thread.sleep(4000);
+
+ profileService.refresh();
Profile updatedProfile = profileService.load(profileId);
updatedProfile.setProperty("country", "test2-country");
profileService.save(updatedProfile);
- Thread.sleep(4000);
+ profileService.refresh();
Profile profileWithNewCountry = profileService.load(profileId);
- assertEquals(profileWithNewCountry.getProperty("country"), "test2-country");
+ assertEquals("Country property doesn't have expected value", "test2-country", profileWithNewCountry.getProperty("country"));
}
@Test(expected = RuntimeException.class)
@@ -113,11 +114,11 @@ public class ProfileServiceWithoutOverwriteIT extends BaseIT {
Profile profile = setupWithoutOverwriteTests();
String profileId = profile.getItemId();
- Thread.sleep(4000);
+ profileService.refresh();
Profile updatedProfile = profileService.load(profileId);
updatedProfile.setProperty("country", "test2-country");
- updatedProfile.setMetadata("seq_no", 1L);
+ updatedProfile.setSystemMetadata("seq_no", 1L);
profileService.save(updatedProfile);
}
}
[unomi] 07/15: UNOMI-409 add .gitignore to unomi-web-tracker (#228)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 2d680cc5b328106efbf7078d523d634c55041f08
Author: MT BENTERKI <be...@gmail.com>
AuthorDate: Wed Jan 6 08:47:04 2021 +0100
UNOMI-409 add .gitignore to unomi-web-tracker (#228)
* UNOMI-409 add .gitignore to unomi-web-tracker
* UNOMI-409 move snippet.js into src update npm scripts
(cherry picked from commit dbfd3d9acb6bea334cf2ef5cd0468ff6fa8bfa7c)
---
extensions/web-tracker/javascript/.gitignore | 1 +
extensions/web-tracker/javascript/package.json | 7 ++++---
extensions/web-tracker/javascript/{ => src}/snippet.js | 0
3 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/extensions/web-tracker/javascript/.gitignore b/extensions/web-tracker/javascript/.gitignore
new file mode 100644
index 0000000..1521c8b
--- /dev/null
+++ b/extensions/web-tracker/javascript/.gitignore
@@ -0,0 +1 @@
+dist
diff --git a/extensions/web-tracker/javascript/package.json b/extensions/web-tracker/javascript/package.json
index 3ed01b0..fc30f96 100644
--- a/extensions/web-tracker/javascript/package.json
+++ b/extensions/web-tracker/javascript/package.json
@@ -11,12 +11,13 @@
"author": "Apache Software Foundation",
"license": "Apache-2.0",
"scripts": {
- "build": "yarn browserify && yarn replace && yarn minify && yarn snippet:minify",
+ "build": "yarn browserify && yarn snippet:browserify && yarn replace && yarn minify && yarn snippet:minify",
"browserify": "browserify src/index.js -p [ browserify-header --file src/license.js ] -s unomiTracker -o dist/unomi-tracker.js",
+ "snippet:browserify": "browserify src/snippet.js -o dist/snippet.js",
"replace": "replace-in-file 'analytics.require = require' '//analytics.require = require' dist/unomi-tracker.js",
"minify": "cd dist && uglifyjs -c -m --comments '/@license/' -o unomi-tracker.min.js --source-map url=unomi-tracker.min.js.map -- unomi-tracker.js",
- "snippet:minify": "uglifyjs -c -m -o snippet.min.js --source-map url=snipper.min.js.map -- snippet.js",
- "clean": "rimraf *.log dist/unomi-tracker.js dist/unomi-tracker.min.js",
+ "snippet:minify": "cd dist && uglifyjs -c -m -o snippet.min.js --source-map url=snipper.min.js.map -- snippet.js",
+ "clean": "rimraf *.log dist/*",
"clean:all": "yarn clean && rimraf node_modules"
},
"dependencies": {
diff --git a/extensions/web-tracker/javascript/snippet.js b/extensions/web-tracker/javascript/src/snippet.js
similarity index 100%
rename from extensions/web-tracker/javascript/snippet.js
rename to extensions/web-tracker/javascript/src/snippet.js
[unomi] 10/15: Fix URL for testing in twitter sample documentation
(#234)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit a2e2bb58a08b8be3692c4f6b9239feeffd62bfe2
Author: wmnedel <52...@users.noreply.github.com>
AuthorDate: Thu Jan 7 06:07:31 2021 -0500
Fix URL for testing in twitter sample documentation (#234)
(cherry picked from commit b7216160195fa22fc294ac3237fe10ebaa0c6437)
---
manual/src/archives/1.3/asciidoc/samples/twitter-sample.adoc | 4 ++--
manual/src/archives/1.4/asciidoc/samples/twitter-sample.adoc | 4 ++--
manual/src/archives/1.5/asciidoc/samples/twitter-sample.adoc | 4 ++--
manual/src/main/asciidoc/samples/twitter-sample.adoc | 4 ++--
4 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/manual/src/archives/1.3/asciidoc/samples/twitter-sample.adoc b/manual/src/archives/1.3/asciidoc/samples/twitter-sample.adoc
index 0590606..e33c5bb 100644
--- a/manual/src/archives/1.3/asciidoc/samples/twitter-sample.adoc
+++ b/manual/src/archives/1.3/asciidoc/samples/twitter-sample.adoc
@@ -47,7 +47,7 @@ cp target/tweet-button-plugin-1.0.0-incubating-SNAPSHOT.jar ../../package/target
===== Testing the samples
-You can now go to http://localhost:8181/index.html[http://localhost:8181/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
+You can now go to http://localhost:8181/twitter/index.html[http://localhost:8181/twitter/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
We will now explain in greater details some concepts and see how the example works.
@@ -430,4 +430,4 @@ We have seen a simple example how to interact with Unomi using a combination of
Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
-image::unomi-request.png[Unomi request overview]
\ No newline at end of file
+image::unomi-request.png[Unomi request overview]
diff --git a/manual/src/archives/1.4/asciidoc/samples/twitter-sample.adoc b/manual/src/archives/1.4/asciidoc/samples/twitter-sample.adoc
index da660a3..1f85ef3 100644
--- a/manual/src/archives/1.4/asciidoc/samples/twitter-sample.adoc
+++ b/manual/src/archives/1.4/asciidoc/samples/twitter-sample.adoc
@@ -47,7 +47,7 @@ cp target/tweet-button-plugin-1.5.1-SNAPSHOT.jar ../../package/target/unomi-1.5.
===== Testing the samples
-You can now go to http://localhost:8181/index.html[http://localhost:8181/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
+You can now go to http://localhost:8181/twitter/index.html[http://localhost:8181/twitter/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
We will now explain in greater details some concepts and see how the example works.
@@ -430,4 +430,4 @@ We have seen a simple example how to interact with Unomi using a combination of
Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
-image::unomi-request.png[Unomi request overview]
\ No newline at end of file
+image::unomi-request.png[Unomi request overview]
diff --git a/manual/src/archives/1.5/asciidoc/samples/twitter-sample.adoc b/manual/src/archives/1.5/asciidoc/samples/twitter-sample.adoc
index e6e59db..b0d6064 100644
--- a/manual/src/archives/1.5/asciidoc/samples/twitter-sample.adoc
+++ b/manual/src/archives/1.5/asciidoc/samples/twitter-sample.adoc
@@ -47,7 +47,7 @@ cp target/tweet-button-plugin-2.0.0-SNAPSHOT.jar ../../package/target/unomi-2.0.
===== Testing the samples
-You can now go to http://localhost:8181/index.html[http://localhost:8181/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
+You can now go to http://localhost:8181/twitter/index.html[http://localhost:8181/twitter/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
We will now explain in greater details some concepts and see how the example works.
@@ -430,4 +430,4 @@ We have seen a simple example how to interact with Unomi using a combination of
Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
-image::unomi-request.png[Unomi request overview]
\ No newline at end of file
+image::unomi-request.png[Unomi request overview]
diff --git a/manual/src/main/asciidoc/samples/twitter-sample.adoc b/manual/src/main/asciidoc/samples/twitter-sample.adoc
index 3f1e031..ac1057d 100644
--- a/manual/src/main/asciidoc/samples/twitter-sample.adoc
+++ b/manual/src/main/asciidoc/samples/twitter-sample.adoc
@@ -47,7 +47,7 @@ cp target/tweet-button-plugin-1.5.1-SNAPSHOT.jar ../../package/target/unomi-1.5.
===== Testing the samples
-You can now go to http://localhost:8181/index.html[http://localhost:8181/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
+You can now go to http://localhost:8181/twitter/index.html[http://localhost:8181/twitter/index.html] to test the samples code. The page is very simple, you will see a Twitter button, which, once clicked, will open a new window to tweet about the current page. The original page should be updated with the new values of the properties coming from Unomi. Additionnally, the raw JSON response is displayed.
We will now explain in greater details some concepts and see how the example works.
@@ -597,4 +597,4 @@ We have seen a simple example how to interact with Unomi using a combination of
Here is an overview of how Unomi processes incoming requests to the `ContextServlet`.
-image::unomi-request.png[Unomi request overview]
\ No newline at end of file
+image::unomi-request.png[Unomi request overview]
[unomi] 08/15: feat(updatePropertiesAction): support add values to
a SET in a profile property (#230)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit f64e5c243cfb34e6f5d21903116917a18edaf02f
Author: giladw <gw...@yotpo.com>
AuthorDate: Wed Jan 6 09:55:42 2021 +0200
feat(updatePropertiesAction): support add values to a SET in a profile property (#230)
(cherry picked from commit 9512fa7991e2b889673fdc712301d2466d054ea3)
---
.../unomi/plugins/baseplugin/actions/UpdatePropertiesAction.java | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/UpdatePropertiesAction.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/UpdatePropertiesAction.java
index 04d2e9b..ef175d9 100644
--- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/UpdatePropertiesAction.java
+++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/UpdatePropertiesAction.java
@@ -37,6 +37,7 @@ public class UpdatePropertiesAction implements ActionExecutor {
public static final String PROPS_TO_ADD = "add";
public static final String PROPS_TO_UPDATE = "update";
public static final String PROPS_TO_DELETE = "delete";
+ public static final String PROPS_TO_ADD_TO_SET = "addToSet";
public static final String TARGET_ID_KEY = "targetId";
public static final String TARGET_TYPE_KEY = "targetType";
@@ -76,6 +77,11 @@ public class UpdatePropertiesAction implements ActionExecutor {
isProfileOrPersonaUpdated |= processProperties(target, propsToUpdate, "alwaysSet");
}
+ Map<String, Object> propsToAddToSet = (HashMap<String, Object>) event.getProperties().get(PROPS_TO_ADD_TO_SET);
+ if (propsToAddToSet != null) {
+ isProfileOrPersonaUpdated |= processProperties(target, propsToAddToSet, "addValues");
+ }
+
List<String> propsToDelete = (List<String>) event.getProperties().get(PROPS_TO_DELETE);
if (propsToDelete != null) {
for (String prop : propsToDelete) {
[unomi] 04/15: UNOMI-371 add optional support for optimistic
concurrency control (if_seq_no) (#223)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 55c8bcdd930d52ef6bee412e3fa8264357af2b42
Author: giladw <gw...@yotpo.com>
AuthorDate: Tue Jan 5 14:23:55 2021 +0100
UNOMI-371 add optional support for optimistic concurrency control (if_seq_no) (#223)
(cherry picked from commit 714a643efe0e0ccd66f14d4a8cff12545f7f94cd)
---
api/src/main/java/org/apache/unomi/api/Item.java | 10 ++
.../apache/unomi/services/UserListServiceImpl.java | 2 +-
.../unomi/privacy/internal/PrivacyServiceImpl.java | 2 +-
.../itests/ProfileServiceWithoutOverwriteIT.java | 123 +++++++++++++++++++++
.../ElasticSearchPersistenceServiceImpl.java | 110 ++++++++++++++----
.../resources/OSGI-INF/blueprint/blueprint.xml | 4 +
.../org.apache.unomi.persistence.elasticsearch.cfg | 8 +-
.../unomi/persistence/spi/PersistenceService.java | 36 +++++-
.../actions/MergeProfilesOnPropertyAction.java | 6 +-
.../unomi/plugins/mail/actions/SendMailAction.java | 2 +-
.../services/impl/events/EventServiceImpl.java | 2 +-
.../services/impl/rules/RulesServiceImpl.java | 2 +-
.../services/impl/segments/SegmentServiceImpl.java | 12 +-
13 files changed, 276 insertions(+), 43 deletions(-)
diff --git a/api/src/main/java/org/apache/unomi/api/Item.java b/api/src/main/java/org/apache/unomi/api/Item.java
index 72f0ae6..2d5ff71 100644
--- a/api/src/main/java/org/apache/unomi/api/Item.java
+++ b/api/src/main/java/org/apache/unomi/api/Item.java
@@ -21,6 +21,7 @@ import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.Serializable;
+import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@@ -63,6 +64,7 @@ public abstract class Item implements Serializable {
protected String itemType;
protected String scope;
protected Long version;
+ protected Map<String, Object> systemMetadata = new HashMap<>();
public Item() {
this.itemType = getItemType(this.getClass());
@@ -140,4 +142,12 @@ public abstract class Item implements Serializable {
public void setVersion(Long version) {
this.version = version;
}
+
+ public Object getSystemMetadata(String key) {
+ return systemMetadata.get(key);
+ }
+
+ public void setSystemMetadata(String key, Object value) {
+ systemMetadata.put(key, value);
+ }
}
diff --git a/extensions/lists-extension/services/src/main/java/org/apache/unomi/services/UserListServiceImpl.java b/extensions/lists-extension/services/src/main/java/org/apache/unomi/services/UserListServiceImpl.java
index dc3bbc8..37ca72e 100644
--- a/extensions/lists-extension/services/src/main/java/org/apache/unomi/services/UserListServiceImpl.java
+++ b/extensions/lists-extension/services/src/main/java/org/apache/unomi/services/UserListServiceImpl.java
@@ -94,7 +94,7 @@ public class UserListServiceImpl implements UserListService {
if(index != -1){
((List) profileSystemProperties.get("lists")).remove(index);
profileSystemProperties.put("lastUpdated", new Date());
- persistenceService.update(p.getItemId(), null, Profile.class, "systemProperties", profileSystemProperties);
+ persistenceService.update(p, null, Profile.class, "systemProperties", profileSystemProperties);
}
}
}
diff --git a/extensions/privacy-extension/services/src/main/java/org/apache/unomi/privacy/internal/PrivacyServiceImpl.java b/extensions/privacy-extension/services/src/main/java/org/apache/unomi/privacy/internal/PrivacyServiceImpl.java
index 1675e11..3d3d68d 100644
--- a/extensions/privacy-extension/services/src/main/java/org/apache/unomi/privacy/internal/PrivacyServiceImpl.java
+++ b/extensions/privacy-extension/services/src/main/java/org/apache/unomi/privacy/internal/PrivacyServiceImpl.java
@@ -137,7 +137,7 @@ public class PrivacyServiceImpl implements PrivacyService {
persistenceService.save(session);
List<Event> events = eventService.searchEvents(session.getItemId(), new String[0], null, 0, -1, null).getList();
for (Event event : events) {
- persistenceService.update(event.getItemId(), event.getTimeStamp(), Event.class, "profileId", newProfile.getItemId());
+ persistenceService.update(event, event.getTimeStamp(), Event.class, "profileId", newProfile.getItemId());
}
}
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
new file mode 100644
index 0000000..5c2ed5b
--- /dev/null
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceWithoutOverwriteIT.java
@@ -0,0 +1,123 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License
+ */
+package org.apache.unomi.itests;
+
+import org.apache.unomi.api.Profile;
+import org.apache.unomi.api.services.DefinitionsService;
+import org.apache.unomi.api.services.ProfileService;
+import org.apache.unomi.persistence.spi.PersistenceService;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.ops4j.pax.exam.Configuration;
+import org.ops4j.pax.exam.Option;
+import org.ops4j.pax.exam.junit.PaxExam;
+import org.ops4j.pax.exam.spi.reactors.ExamReactorStrategy;
+import org.ops4j.pax.exam.spi.reactors.PerSuite;
+import org.ops4j.pax.exam.util.Filter;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import javax.inject.Inject;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.List;
+
+import static org.junit.Assert.assertEquals;
+import static org.ops4j.pax.exam.CoreOptions.systemProperty;
+
+/**
+ * An integration test for the profile service
+ */
+@RunWith(PaxExam.class)
+@ExamReactorStrategy(PerSuite.class)
+public class ProfileServiceWithoutOverwriteIT extends BaseIT {
+ private final static Logger LOGGER = LoggerFactory.getLogger(ProfileServiceWithoutOverwriteIT.class);
+
+ private final static String TEST_PROFILE_ID = "test-profile-id";
+
+ @Configuration
+ public Option[] config() throws InterruptedException {
+ List<Option> options = new ArrayList<>();
+ options.addAll(Arrays.asList(super.config()));
+ options.add(systemProperty("org.apache.unomi.elasticsearch.throwExceptions").value("true"));
+ options.add(systemProperty("org.apache.unomi.elasticsearch.alwaysOverwrite").value("false"));
+ return options.toArray(new Option[0]);
+ }
+
+ @Inject @Filter(timeout = 600000)
+ protected ProfileService profileService;
+
+ @Inject
+ @Filter(timeout = 600000)
+ protected PersistenceService persistenceService;
+
+ @Inject
+ @Filter(timeout = 600000)
+ protected DefinitionsService definitionsService;
+
+ @Before
+ public void setUp() {
+ TestUtils.removeAllProfiles(definitionsService, persistenceService);
+ }
+
+ private Profile setupWithoutOverwriteTests() {
+ Profile profile = new Profile();
+ profile.setItemId(TEST_PROFILE_ID);
+ profile.setProperty("country", "test-country");
+ profile.setProperty("state", "test-state");
+ profileService.save(profile);
+
+ return profile;
+ }
+
+ @Test(expected = RuntimeException.class)
+ public void testSaveProfileWithoutOverwriteSameProfileThrowsException() {
+ Profile profile = setupWithoutOverwriteTests();
+ profile.setProperty("country", "test2-country");
+ profileService.save(profile);
+ }
+
+ @Test
+ public void testSaveProfileWithoutOverwriteSavesAfterReload() throws InterruptedException {
+ Profile profile = setupWithoutOverwriteTests();
+ String profileId = profile.getItemId();
+ Thread.sleep(4000);
+
+ Profile updatedProfile = profileService.load(profileId);
+ updatedProfile.setProperty("country", "test2-country");
+ profileService.save(updatedProfile);
+
+ Thread.sleep(4000);
+
+ Profile profileWithNewCountry = profileService.load(profileId);
+ assertEquals(profileWithNewCountry.getProperty("country"), "test2-country");
+ }
+
+ @Test(expected = RuntimeException.class)
+ public void testSaveProfileWithoutOverwriteWrongSeqNoThrowsException() throws InterruptedException {
+ Profile profile = setupWithoutOverwriteTests();
+ String profileId = profile.getItemId();
+
+ Thread.sleep(4000);
+
+ Profile updatedProfile = profileService.load(profileId);
+ updatedProfile.setProperty("country", "test2-country");
+ updatedProfile.setMetadata("seq_no", 1L);
+ profileService.save(updatedProfile);
+ }
+}
diff --git a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
index 62aaa81..5c4a251 100644
--- a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
+++ b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
@@ -23,7 +23,6 @@ import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
-import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.lucene.search.TotalHits;
@@ -40,6 +39,7 @@ import org.apache.unomi.persistence.elasticsearch.conditions.*;
import org.apache.unomi.persistence.spi.PersistenceService;
import org.apache.unomi.persistence.spi.aggregate.*;
import org.elasticsearch.ElasticsearchStatusException;
+import org.elasticsearch.action.DocWriteRequest;
import org.elasticsearch.action.admin.cluster.health.ClusterHealthRequest;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.admin.indices.refresh.RefreshRequest;
@@ -49,6 +49,7 @@ import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
+import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.ClearScrollRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
@@ -63,6 +64,7 @@ import org.elasticsearch.client.HttpAsyncResponseConsumerFactory;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import org.elasticsearch.client.RestHighLevelClient;
+import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.core.CountRequest;
import org.elasticsearch.client.core.CountResponse;
import org.elasticsearch.client.core.MainResponse;
@@ -139,6 +141,9 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
public static final String BULK_PROCESSOR_FLUSH_INTERVAL = "bulkProcessor.flushInterval";
public static final String BULK_PROCESSOR_BACKOFF_POLICY = "bulkProcessor.backoffPolicy";
public static final String INDEX_DATE_PREFIX = "date-";
+ public static final String SEQ_NO = "seq_no";
+ public static final String PRIMARY_TERM = "primary_term";
+
private static final Logger logger = LoggerFactory.getLogger(ElasticSearchPersistenceServiceImpl.class.getName());
private RestHighLevelClient client;
private BulkProcessor bulkProcessor;
@@ -192,6 +197,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
private Integer aggQueryMaxResponseSizeHttp = null;
private Integer clientSocketTimeout = null;
+ private boolean alwaysOverwrite = true;
private Map<String, Map<String, Map<String, Object>>> knownMappings = new HashMap<>();
@@ -362,6 +368,15 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
public void setAggQueryThrowOnMissingDocs(boolean aggQueryThrowOnMissingDocs) {
this.aggQueryThrowOnMissingDocs = aggQueryThrowOnMissingDocs;
}
+
+ public void setThrowExceptions(boolean throwExceptions) {
+ this.throwExceptions = throwExceptions;
+ }
+ public void setAlwaysOverwrite(boolean alwaysOverwrite) {
+ this.alwaysOverwrite = alwaysOverwrite;
+ }
+
+
public void start() throws Exception {
// on startup
@@ -747,8 +762,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
if (response.isExists()) {
String sourceAsString = response.getSourceAsString();
final T value = ESCustomObjectMapper.getObjectMapper().readValue(sourceAsString, clazz);
- value.setItemId(response.getId());
- value.setVersion(response.getVersion());
+ setMetadata(value, response.getId(), response.getVersion(), response.getSeqNo(), response.getPrimaryTerm());
putInCache(itemId, value);
return value;
} else {
@@ -772,13 +786,28 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
}
+ private void setMetadata(Item item, String id, long version, long seqNo, long primaryTerm) {
+ item.setItemId(id);
+ item.setVersion(version);
+ item.setSystemMetadata(SEQ_NO, seqNo);
+ item.setSystemMetadata(PRIMARY_TERM, primaryTerm);
+ }
+
@Override
public boolean save(final Item item) {
- return save(item, useBatchingForSave);
+ return save(item, useBatchingForSave, alwaysOverwrite);
}
@Override
public boolean save(final Item item, final boolean useBatching) {
+ return save(item, useBatching, alwaysOverwrite);
+ }
+
+ @Override
+ public boolean save(final Item item, final Boolean useBatchingOption, final Boolean alwaysOverwriteOption) {
+ final boolean useBatching = useBatchingOption == null ? this.useBatchingForSave : useBatchingOption;
+ final boolean alwaysOverwrite = alwaysOverwriteOption == null ? this.alwaysOverwrite : alwaysOverwriteOption;
+
Boolean result = new InClassLoaderExecute<Boolean>(metricsService, this.getClass().getName() + ".saveItem", this.bundleContext, this.fatalIllegalStateErrors) {
protected Boolean execute(Object... args) throws Exception {
try {
@@ -790,13 +819,28 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
IndexRequest indexRequest = new IndexRequest(index);
indexRequest.id(itemId);
indexRequest.source(source, XContentType.JSON);
+
+ if (!alwaysOverwrite) {
+ Long seqNo = (Long)item.getSystemMetadata(SEQ_NO);
+ Long primaryTerm = (Long)item.getSystemMetadata(PRIMARY_TERM);
+
+ if (seqNo != null && primaryTerm != null) {
+ indexRequest.setIfSeqNo(seqNo);
+ indexRequest.setIfPrimaryTerm(primaryTerm);
+ }
+ else {
+ indexRequest.opType(DocWriteRequest.OpType.CREATE);
+ }
+ }
+
if (routingByType.containsKey(itemType)) {
indexRequest.routing(routingByType.get(itemType));
}
try {
if (bulkProcessor == null || !useBatching) {
- client.index(indexRequest, RequestOptions.DEFAULT);
+ IndexResponse response = client.index(indexRequest, RequestOptions.DEFAULT);
+ setMetadata(item, response.getId(), response.getVersion(), response.getSeqNo(), response.getPrimaryTerm());
} else {
bulkProcessor.add(indexRequest);
}
@@ -819,26 +863,43 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
}
@Override
- public boolean update(final String itemId, final Date dateHint, final Class clazz, final String propertyName, final Object propertyValue) {
- return update(itemId, dateHint, clazz, Collections.singletonMap(propertyName, propertyValue));
+ public boolean update(final Item item, final Date dateHint, final Class clazz, final String propertyName, final Object propertyValue) {
+ return update(item, dateHint, clazz, Collections.singletonMap(propertyName, propertyValue));
}
@Override
- public boolean update(final String itemId, final Date dateHint, final Class clazz, final Map source) {
+ public boolean update(final Item item, final Date dateHint, final Class clazz, final Map source) {
+ return update(item, dateHint, clazz, source, alwaysOverwrite);
+ }
+
+ @Override
+ public boolean update(final Item item, final Date dateHint, final Class clazz, final Map source, final boolean alwaysOverwrite) {
Boolean result = new InClassLoaderExecute<Boolean>(metricsService, this.getClass().getName() + ".updateItem", this.bundleContext, this.fatalIllegalStateErrors) {
protected Boolean execute(Object... args) throws Exception {
try {
String itemType = Item.getItemType(clazz);
- UpdateRequest updateRequest = new UpdateRequest(getIndex(itemType, dateHint), itemId);
+ UpdateRequest updateRequest = new UpdateRequest(getIndex(itemType, dateHint), item.getItemId());
updateRequest.doc(source);
+
+ if (!alwaysOverwrite) {
+ Long seqNo = (Long)item.getSystemMetadata(SEQ_NO);
+ Long primaryTerm = (Long)item.getSystemMetadata(PRIMARY_TERM);
+
+ if (seqNo != null && primaryTerm != null) {
+ updateRequest.setIfSeqNo(seqNo);
+ updateRequest.setIfPrimaryTerm(primaryTerm);
+ }
+ }
+
if (bulkProcessor == null) {
- client.update(updateRequest, RequestOptions.DEFAULT);
+ UpdateResponse response = client.update(updateRequest, RequestOptions.DEFAULT);
+ setMetadata(item, response.getId(), response.getVersion(), response.getSeqNo(), response.getPrimaryTerm());
} else {
bulkProcessor.add(updateRequest);
}
return true;
} catch (IndexNotFoundException e) {
- throw new Exception("No index found for itemType=" + clazz.getName() + "itemId=" + itemId, e);
+ throw new Exception("No index found for itemType=" + clazz.getName() + "itemId=" + item.getItemId(), e);
}
}
}.catchingExecuteInClassLoader(true);
@@ -907,7 +968,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
}
@Override
- public boolean updateWithScript(final String itemId, final Date dateHint, final Class<?> clazz, final String script, final Map<String, Object> scriptParams) {
+ public boolean updateWithScript(final Item item, final Date dateHint, final Class<?> clazz, final String script, final Map<String, Object> scriptParams) {
Boolean result = new InClassLoaderExecute<Boolean>(metricsService, this.getClass().getName() + ".updateWithScript", this.bundleContext, this.fatalIllegalStateErrors) {
protected Boolean execute(Object... args) throws Exception {
try {
@@ -917,17 +978,26 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
Script actualScript = new Script(ScriptType.INLINE, "painless", script, scriptParams);
- UpdateRequest updateRequest = new UpdateRequest(index, itemId);
+ UpdateRequest updateRequest = new UpdateRequest(index, item.getItemId());
+
+ Long seqNo = (Long)item.getSystemMetadata(SEQ_NO);
+ Long primaryTerm = (Long)item.getSystemMetadata(PRIMARY_TERM);
+
+ if (seqNo != null && primaryTerm != null) {
+ updateRequest.setIfSeqNo(seqNo);
+ updateRequest.setIfPrimaryTerm(primaryTerm);
+ }
updateRequest.script(actualScript);
if (bulkProcessor == null) {
- client.update(updateRequest, RequestOptions.DEFAULT);
+ UpdateResponse response = client.update(updateRequest, RequestOptions.DEFAULT);
+ setMetadata(item, response.getId(), response.getVersion(), response.getSeqNo(), response.getPrimaryTerm());
} else {
bulkProcessor.add(updateRequest);
}
return true;
} catch (IndexNotFoundException e) {
- throw new Exception("No index found for itemType=" + clazz.getName() + "itemId=" + itemId, e);
+ throw new Exception("No index found for itemType=" + clazz.getName() + "itemId=" + item.getItemId(), e);
}
}
}.catchingExecuteInClassLoader(true);
@@ -1471,6 +1541,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
SearchRequest searchRequest = new SearchRequest(getIndexNameForQuery(itemType));
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder()
.fetchSource(true)
+ .seqNoAndPrimaryTerm(true)
.query(query)
.size(size < 0 ? defaultQueryLimit : size)
.from(offset);
@@ -1528,8 +1599,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
// add hit to results
String sourceAsString = searchHit.getSourceAsString();
final T value = ESCustomObjectMapper.getObjectMapper().readValue(sourceAsString, clazz);
- value.setItemId(searchHit.getId());
- value.setVersion(searchHit.getVersion());
+ setMetadata(value, searchHit.getId(), searchHit.getVersion(), searchHit.getSeqNo(), searchHit.getPrimaryTerm());
results.add(value);
}
@@ -1559,8 +1629,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
for (SearchHit searchHit : searchHits) {
String sourceAsString = searchHit.getSourceAsString();
final T value = ESCustomObjectMapper.getObjectMapper().readValue(sourceAsString, clazz);
- value.setItemId(searchHit.getId());
- value.setVersion(searchHit.getVersion());
+ setMetadata(value, searchHit.getId(), searchHit.getVersion(), searchHit.getSeqNo(), searchHit.getPrimaryTerm());
results.add(value);
}
}
@@ -1606,8 +1675,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
// add hit to results
String sourceAsString = searchHit.getSourceAsString();
final T value = ESCustomObjectMapper.getObjectMapper().readValue(sourceAsString, clazz);
- value.setItemId(searchHit.getId());
- value.setVersion(searchHit.getVersion());
+ setMetadata(value, searchHit.getId(), searchHit.getVersion(), searchHit.getSeqNo(), searchHit.getPrimaryTerm());
results.add(value);
}
}
diff --git a/persistence-elasticsearch/core/src/main/resources/OSGI-INF/blueprint/blueprint.xml b/persistence-elasticsearch/core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
index 48402f3..6f40799 100644
--- a/persistence-elasticsearch/core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
+++ b/persistence-elasticsearch/core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
@@ -63,6 +63,8 @@
<cm:property name="password" value="" />
<cm:property name="sslEnable" value="false" />
<cm:property name="sslTrustAllCertificates" value="false" />
+ <cm:property name="throwExceptions" value="false" />
+ <cm:property name="alwaysOverwrite" value="true" />
</cm:default-properties>
</cm:property-placeholder>
@@ -136,6 +138,8 @@
<property name="password" value="${es.password}" />
<property name="sslEnable" value="${es.sslEnable}" />
<property name="sslTrustAllCertificates" value="${es.sslTrustAllCertificates}" />
+ <property name="throwExceptions" value="${es.throwExceptions}" />
+ <property name="alwaysOverwrite" value="${es.alwaysOverwrite}" />
</bean>
<!-- We use a listener here because using the list directly for listening to proxies coming from the same bundle didn't seem to work -->
diff --git a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg
index c0e7b46..ac30c91 100644
--- a/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg
+++ b/persistence-elasticsearch/core/src/main/resources/org.apache.unomi.persistence.elasticsearch.cfg
@@ -26,7 +26,6 @@ monthlyIndex.numberOfShards=${org.apache.unomi.elasticsearch.monthlyIndex.nbShar
monthlyIndex.numberOfReplicas=${org.apache.unomi.elasticsearch.monthlyIndex.nbReplicas:-0}
monthlyIndex.indexMappingTotalFieldsLimit=${org.apache.unomi.elasticsearch.monthlyIndex.indexMappingTotalFieldsLimit:-1000}
monthlyIndex.indexMaxDocValueFieldsSearch=${org.apache.unomi.elasticsearch.monthlyIndex.indexMaxDocValueFieldsSearch:-1000}
-monthlyIndex.itemsMonthlyIndexedOverride=${org.apache.unomi.elasticsearch.monthlyIndex.itemsMonthlyIndexedOverride:-event,session}
numberOfShards=${org.apache.unomi.elasticsearch.defaultIndex.nbShards:-5}
numberOfReplicas=${org.apache.unomi.elasticsearch.defaultIndex.nbReplicas:-0}
indexMappingTotalFieldsLimit=${org.apache.unomi.elasticsearch.defaultIndex.indexMappingTotalFieldsLimit:-1000}
@@ -70,4 +69,9 @@ aggQueryMaxResponseSizeHttp=${org.apache.unomi.elasticsearch.aggQueryMaxResponse
username=${org.apache.unomi.elasticsearch.username:-}
password=${org.apache.unomi.elasticsearch.password:-}
sslEnable=${org.apache.unomi.elasticsearch.sslEnable:-false}
-sslTrustAllCertificates=${org.apache.unomi.elasticsearch.sslTrustAllCertificates:-false}
\ No newline at end of file
+sslTrustAllCertificates=${org.apache.unomi.elasticsearch.sslTrustAllCertificates:-false}
+
+# Errors
+throwExceptions=${org.apache.unomi.elasticsearch.throwExceptions:-false}
+
+alwaysOverwrite=${org.apache.unomi.elasticsearch.alwaysOverwrite:-true}
\ No newline at end of file
diff --git a/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java b/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
index d85059e..f92d3bb 100644
--- a/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
+++ b/persistence-spi/src/main/java/org/apache/unomi/persistence/spi/PersistenceService.java
@@ -127,41 +127,65 @@ public interface PersistenceService {
boolean save(Item item, boolean useBatching);
/**
+ * Persists the specified Item in the context server.
+ *
+ * @param item the item to persist
+ * @param useBatching whether to use batching or not for saving the item. If activating there may be a delay between
+ * the call to this method and the actual saving in the persistence backend
+ * @param alwaysOverwrite whether to overwrite a document even if we are holding an old item when saving
+ *
+ * @return {@code true} if the item was properly persisted, {@code false} otherwise
+ */
+ boolean save(Item item, Boolean useBatching, Boolean alwaysOverwrite);
+
+ /**
* Updates the item of the specified class and identified by the specified identifier with new property values provided as name - value pairs in the specified Map.
*
- * @param itemId the identifier of the item we want to update
+ * @param item the item we want to update
* @param dateHint a Date helping in identifying where the item is located
* @param clazz the Item subclass of the item to update
* @param source a Map with entries specifying as key the property name to update and as value its new value
* @return {@code true} if the update was successful, {@code false} otherwise
*/
- boolean update(String itemId, Date dateHint, Class<?> clazz, Map<?, ?> source);
+ boolean update(Item item, Date dateHint, Class<?> clazz, Map<?, ?> source);
/**
* Updates the item of the specified class and identified by the specified identifier with a new property value for the specified property name. Same as
* {@code update(itemId, dateHint, clazz, Collections.singletonMap(propertyName, propertyValue))}
*
- * @param itemId the identifier of the item we want to update
+ * @param item the item we want to update
* @param dateHint a Date helping in identifying where the item is located
* @param clazz the Item subclass of the item to update
* @param propertyName the name of the property to update
* @param propertyValue the new value of the property
* @return {@code true} if the update was successful, {@code false} otherwise
*/
- boolean update(String itemId, Date dateHint, Class<?> clazz, String propertyName, Object propertyValue);
+ boolean update(Item item, Date dateHint, Class<?> clazz, String propertyName, Object propertyValue);
+
+ /**
+ * Updates the item of the specified class and identified by the specified identifier with new property values provided as name - value pairs in the specified Map.
+ *
+ * @param item the item we want to update
+ * @param dateHint a Date helping in identifying where the item is located
+ * @param clazz the Item subclass of the item to update
+ * @param source a Map with entries specifying as key the property name to update and as value its new value
+ * @param alwaysOverwrite whether to overwrite a document even if we are holding an old item when saving
+ * @return {@code true} if the update was successful, {@code false} otherwise
+ */
+ boolean update(Item item, Date dateHint, Class<?> clazz, Map<?, ?> source, final boolean alwaysOverwrite);
/**
* Updates the item of the specified class and identified by the specified identifier with a new property value for the specified property name. Same as
* {@code update(itemId, dateHint, clazz, Collections.singletonMap(propertyName, propertyValue))}
*
- * @param itemId the identifier of the item we want to update
+ * @param item the item we want to update
* @param dateHint a Date helping in identifying where the item is located
* @param clazz the Item subclass of the item to update
* @param script inline script
* @param scriptParams script params
* @return {@code true} if the update was successful, {@code false} otherwise
*/
- boolean updateWithScript(String itemId, Date dateHint, Class<?> clazz, String script, Map<String, Object> scriptParams);
+ boolean updateWithScript(Item item, Date dateHint, Class<?> clazz, String script, Map<String, Object> scriptParams);
/**
* Updates the items of the specified class by a query with a new property value for the specified property name
diff --git a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
index a496ddb..ffdf626 100644
--- a/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
+++ b/plugins/baseplugin/src/main/java/org/apache/unomi/plugins/baseplugin/actions/MergeProfilesOnPropertyAction.java
@@ -178,12 +178,12 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
}
for (Session session : sessions) {
- persistenceService.update(session.getItemId(), session.getTimeStamp(), Session.class, "profileId", anonymousBrowsing ? null : masterProfileId);
+ persistenceService.update(session, session.getTimeStamp(), Session.class, "profileId", anonymousBrowsing ? null : masterProfileId);
}
List<Event> events = persistenceService.query("profileId", profileId, null, Event.class);
for (Event event : events) {
- persistenceService.update(event.getItemId(), event.getTimeStamp(), Event.class, "profileId", anonymousBrowsing ? null : masterProfileId);
+ persistenceService.update(event, event.getTimeStamp(), Event.class, "profileId", anonymousBrowsing ? null : masterProfileId);
}
// we must mark all the profiles that we merged into the master as merged with the master, and they will
// be deleted upon next load
@@ -192,7 +192,7 @@ public class MergeProfilesOnPropertyAction implements ActionExecutor {
sourceMap.put("mergedWith", masterProfileId);
profile.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profile.getSystemProperties());
- persistenceService.update(profile.getItemId(), null, Profile.class, sourceMap);
+ persistenceService.update(profile, null, Profile.class, sourceMap);
}
}
} catch (Exception e) {
diff --git a/plugins/mail/src/main/java/org/apache/unomi/plugins/mail/actions/SendMailAction.java b/plugins/mail/src/main/java/org/apache/unomi/plugins/mail/actions/SendMailAction.java
index 8e03175..acdb1eb 100644
--- a/plugins/mail/src/main/java/org/apache/unomi/plugins/mail/actions/SendMailAction.java
+++ b/plugins/mail/src/main/java/org/apache/unomi/plugins/mail/actions/SendMailAction.java
@@ -116,7 +116,7 @@ public class SendMailAction implements ActionExecutor {
event.getProfile().setSystemProperty("notificationAck", profileNotif);
event.getProfile().setSystemProperty("lastUpdated", new Date());
- persistenceService.update(event.getProfile().getItemId(), null, Profile.class, "systemProperties", event.getProfile().getSystemProperties());
+ persistenceService.update(event.getProfile(), null, Profile.class, "systemProperties", event.getProfile().getSystemProperties());
ST stringTemplate = new ST(template, '$', '$');
stringTemplate.add("profile", event.getProfile());
diff --git a/services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java b/services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java
index 80b0449..25db0b5 100644
--- a/services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java
+++ b/services/src/main/java/org/apache/unomi/services/impl/events/EventServiceImpl.java
@@ -152,7 +152,7 @@ public class EventServiceImpl implements EventService {
boolean saveSucceeded = true;
if (event.isPersistent()) {
- saveSucceeded = persistenceService.save(event);
+ saveSucceeded = persistenceService.save(event, null, true);
}
int changes;
diff --git a/services/src/main/java/org/apache/unomi/services/impl/rules/RulesServiceImpl.java b/services/src/main/java/org/apache/unomi/services/impl/rules/RulesServiceImpl.java
index d77173c..734d856 100644
--- a/services/src/main/java/org/apache/unomi/services/impl/rules/RulesServiceImpl.java
+++ b/services/src/main/java/org/apache/unomi/services/impl/rules/RulesServiceImpl.java
@@ -475,7 +475,7 @@ public class RulesServiceImpl implements RulesService, EventListenerService, Syn
}
allRuleStatistics.put(ruleStatistics.getItemId(), ruleStatistics);
if (mustPersist) {
- persistenceService.save(ruleStatistics);
+ persistenceService.save(ruleStatistics, null, true);
}
}
// now let's iterate over the rules coming from the persistence service, as we may have new ones.
diff --git a/services/src/main/java/org/apache/unomi/services/impl/segments/SegmentServiceImpl.java b/services/src/main/java/org/apache/unomi/services/impl/segments/SegmentServiceImpl.java
index 36a2a67..e971432 100644
--- a/services/src/main/java/org/apache/unomi/services/impl/segments/SegmentServiceImpl.java
+++ b/services/src/main/java/org/apache/unomi/services/impl/segments/SegmentServiceImpl.java
@@ -365,7 +365,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
sourceMap.put("segments", profileToRemove.getSegments());
profileToRemove.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profileToRemove.getSystemProperties());
- persistenceService.update(profileToRemove.getItemId(), null, Profile.class, sourceMap);
+ persistenceService.update(profileToRemove, null, Profile.class, sourceMap);
updatedProfileCount++;
}
logger.info("Removed segment from {} profiles in {} ms", updatedProfileCount, System.currentTimeMillis() - profileRemovalStartTime);
@@ -724,7 +724,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
// todo remove profile properties ?
persistenceService.remove(previousRule.getItemId(), Rule.class);
} else {
- persistenceService.update(previousRule.getItemId(), null, Rule.class, "linkedItems", previousRule.getLinkedItems());
+ persistenceService.update(previousRule, null, Rule.class, "linkedItems", previousRule.getLinkedItems());
}
}
}
@@ -862,7 +862,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
systemProperties.put("lastUpdated", new Date());
Profile profile = new Profile();
profile.setItemId(profileId);
- persistenceService.update(profile.getItemId(), null, Profile.class, "systemProperties", systemProperties);
+ persistenceService.update(profile, null, Profile.class, "systemProperties", systemProperties);
} catch (Exception e) {
logger.error("Error updating profile {} past event system properties", profileId, e);
}
@@ -930,7 +930,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
sourceMap.put("segments", profileToAdd.getSegments());
profileToAdd.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profileToAdd.getSystemProperties());
- persistenceService.update(profileToAdd.getItemId(), null, Profile.class, sourceMap);
+ persistenceService.update(profileToAdd, null, Profile.class, sourceMap);
Event profileUpdated = new Event("profileUpdated", null, profileToAdd, null, null, profileToAdd, new Date());
profileUpdated.setPersistent(false);
eventService.send(profileUpdated);
@@ -950,7 +950,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
sourceMap.put("segments", profileToRemove.getSegments());
profileToRemove.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profileToRemove.getSystemProperties());
- persistenceService.update(profileToRemove.getItemId(), null, Profile.class, sourceMap);
+ persistenceService.update(profileToRemove, null, Profile.class, sourceMap);
Event profileUpdated = new Event("profileUpdated", null, profileToRemove, null, null, profileToRemove, new Date());
profileUpdated.setPersistent(false);
eventService.send(profileUpdated);
@@ -973,7 +973,7 @@ public class SegmentServiceImpl extends AbstractServiceImpl implements SegmentSe
sourceMap.put("segments", profileToRemove.getSegments());
profileToRemove.setSystemProperty("lastUpdated", new Date());
sourceMap.put("systemProperties", profileToRemove.getSystemProperties());
- persistenceService.update(profileToRemove.getItemId(), null, Profile.class, sourceMap);
+ persistenceService.update(profileToRemove, null, Profile.class, sourceMap);
Event profileUpdated = new Event("profileUpdated", null, profileToRemove, null, null, profileToRemove, new Date());
profileUpdated.setPersistent(false);
eventService.send(profileUpdated);
[unomi] 15/15: Return error to the client when exception is thrown
from Elasticsearch impl (#202)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit db211807c305c1113a5329f3c606f1c160ff21f8
Author: liatiusim <62...@users.noreply.github.com>
AuthorDate: Thu Nov 19 09:20:29 2020 +0100
Return error to the client when exception is thrown from Elasticsearch impl (#202)
* Throw exception from elasticsearch impl and add option to return error to the client in this case
* read throwExcetions member from the elasticImpl class - classNotFpund exception
* merge glitch fix
(cherry picked from commit 9e346c97f839a8ed055d68c5b2e842d6852063a1)
---
.../org/apache/unomi/itests/ProfileServiceIT.java | 1 +
.../main/resources/etc/custom.system.properties | 2 +
.../ElasticSearchPersistenceServiceImpl.java | 4 +
.../apache/unomi/rest/RuntimeExceptionMapper.java | 38 ++
.../java/org/apache/unomi/web/ClientServlet.java | 38 +-
.../java/org/apache/unomi/web/ContextServlet.java | 389 +++++++++++----------
.../apache/unomi/web/EventsCollectorServlet.java | 24 +-
7 files changed, 283 insertions(+), 213 deletions(-)
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
index d75f608..3d9536b 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileServiceIT.java
@@ -22,6 +22,7 @@ import org.apache.unomi.api.services.ProfileService;
import org.apache.unomi.persistence.spi.PersistenceService;
import org.apache.unomi.api.services.DefinitionsService;
import org.apache.unomi.api.PartialList;
+import org.apache.unomi.persistence.elasticsearch.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.fail;
diff --git a/package/src/main/resources/etc/custom.system.properties b/package/src/main/resources/etc/custom.system.properties
index 4618ac0..a7b90f3 100644
--- a/package/src/main/resources/etc/custom.system.properties
+++ b/package/src/main/resources/etc/custom.system.properties
@@ -126,6 +126,8 @@ org.apache.unomi.elasticsearch.bulkProcessor.bulkActions=${env:UNOMI_ELASTICSEAR
org.apache.unomi.elasticsearch.bulkProcessor.bulkSize=${env:UNOMI_ELASTICSEARCH_BULK_SIZE:-5MB}
org.apache.unomi.elasticsearch.bulkProcessor.flushInterval=${env:UNOMI_ELASTICSEARCH_BULK_FLUSHINTERVAL:-5s}
org.apache.unomi.elasticsearch.bulkProcessor.backoffPolicy=${env:UNOMI_ELASTICSEARCH_BULK_BACKOFFPOLICY:-exponential}
+# Errors
+org.apache.unomi.elasticsearch.throwExceptions=${env:UNOMI_ELASTICSEARCH_THROW_EXCEPTIONS:-false}
# Authentication
org.apache.unomi.elasticsearch.username=${env:UNOMI_ELASTICSEARCH_USERNAME:-}
org.apache.unomi.elasticsearch.password=${env:UNOMI_ELASTICSEARCH_PASSWORD:-}
diff --git a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
index 5c4a251..8ae4701 100644
--- a/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
+++ b/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/ElasticSearchPersistenceServiceImpl.java
@@ -145,6 +145,7 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
public static final String PRIMARY_TERM = "primary_term";
private static final Logger logger = LoggerFactory.getLogger(ElasticSearchPersistenceServiceImpl.class.getName());
+ private static boolean throwExceptions = false;
private RestHighLevelClient client;
private BulkProcessor bulkProcessor;
private String elasticSearchAddresses;
@@ -2139,6 +2140,9 @@ public class ElasticSearchPersistenceServiceImpl implements PersistenceService,
if (logError) {
logger.error("Error while executing in class loader", t);
}
+ if (throwExceptions) {
+ throw new RuntimeException(t);
+ }
}
private void handleFatalStateError() {
diff --git a/rest/src/main/java/org/apache/unomi/rest/RuntimeExceptionMapper.java b/rest/src/main/java/org/apache/unomi/rest/RuntimeExceptionMapper.java
new file mode 100644
index 0000000..00e1732
--- /dev/null
+++ b/rest/src/main/java/org/apache/unomi/rest/RuntimeExceptionMapper.java
@@ -0,0 +1,38 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.rest;
+
+import org.osgi.service.component.annotations.Component;
+
+import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
+import javax.ws.rs.ext.ExceptionMapper;
+import javax.ws.rs.ext.Provider;
+
+import java.util.HashMap;
+
+@Provider
+@Component(service=ExceptionMapper.class)
+public class RuntimeExceptionMapper implements ExceptionMapper<RuntimeException> {
+
+ @Override
+ public Response toResponse(RuntimeException exception) {
+ HashMap<String, Object> body = new HashMap<>();
+ body.put("errorMessage", "internalServerError");
+ return Response.status(Response.Status.INTERNAL_SERVER_ERROR).header("Content-Type", MediaType.APPLICATION_JSON).entity(body).build();
+ }
+}
\ No newline at end of file
diff --git a/wab/src/main/java/org/apache/unomi/web/ClientServlet.java b/wab/src/main/java/org/apache/unomi/web/ClientServlet.java
index f385a53..59de0ce 100644
--- a/wab/src/main/java/org/apache/unomi/web/ClientServlet.java
+++ b/wab/src/main/java/org/apache/unomi/web/ClientServlet.java
@@ -67,26 +67,30 @@ public class ClientServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
- String[] pathInfo = req.getPathInfo().substring(1).split("\\.");
- if (pathInfo != null && pathInfo.length > 0) {
- String operation = pathInfo[0];
- String param = pathInfo[1];
- switch (operation) {
- case "myprofile":
- if (allowedProfileDownloadFormats.contains(param)) {
- donwloadCurrentProfile(req, resp, param);
- } else {
- resp.setStatus(HttpServletResponse.SC_BAD_REQUEST);
- }
- break;
- default:
- resp.setStatus(HttpServletResponse.SC_NOT_FOUND);
+ try {
+ String[] pathInfo = req.getPathInfo().substring(1).split("\\.");
+ if (pathInfo != null && pathInfo.length > 0) {
+ String operation = pathInfo[0];
+ String param = pathInfo[1];
+ switch (operation) {
+ case "myprofile":
+ if (allowedProfileDownloadFormats.contains(param)) {
+ donwloadCurrentProfile(req, resp, param);
+ } else {
+ resp.setStatus(HttpServletResponse.SC_BAD_REQUEST);
+ }
+ break;
+ default:
+ resp.setStatus(HttpServletResponse.SC_NOT_FOUND);
+ }
+ } else {
+ resp.setStatus(HttpServletResponse.SC_NOT_FOUND);
}
- } else {
- resp.setStatus(HttpServletResponse.SC_NOT_FOUND);
+ } catch (Throwable t) { // Here in order to return generic message instead of the whole stack trace in case of not caught exception
+ logger.error("ClientServlet failed to execute get", t);
+ resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Internal server error");
}
-
}
@Override
diff --git a/wab/src/main/java/org/apache/unomi/web/ContextServlet.java b/wab/src/main/java/org/apache/unomi/web/ContextServlet.java
index 033ef5c..f45a891 100644
--- a/wab/src/main/java/org/apache/unomi/web/ContextServlet.java
+++ b/wab/src/main/java/org/apache/unomi/web/ContextServlet.java
@@ -72,234 +72,239 @@ public class ContextServlet extends HttpServlet {
@Override
public void service(HttpServletRequest request, HttpServletResponse response) throws IOException {
- final Date timestamp = new Date();
- if (request.getParameter("timestamp") != null) {
- timestamp.setTime(Long.parseLong(request.getParameter("timestamp")));
- }
+ try {
+ final Date timestamp = new Date();
+ if (request.getParameter("timestamp") != null) {
+ timestamp.setTime(Long.parseLong(request.getParameter("timestamp")));
+ }
- // set up CORS headers as soon as possible so that errors are not misconstrued on the client for CORS errors
- HttpUtils.setupCORSHeaders(request, response);
+ // set up CORS headers as soon as possible so that errors are not misconstrued on the client for CORS errors
+ HttpUtils.setupCORSHeaders(request, response);
- // Handle OPTIONS request
- String httpMethod = request.getMethod();
- if ("options".equals(httpMethod.toLowerCase())) {
- response.flushBuffer();
- if (logger.isDebugEnabled()) {
- logger.debug("OPTIONS request received. No context will be returned.");
+ // Handle OPTIONS request
+ String httpMethod = request.getMethod();
+ if ("options".equals(httpMethod.toLowerCase())) {
+ response.flushBuffer();
+ if (logger.isDebugEnabled()) {
+ logger.debug("OPTIONS request received. No context will be returned.");
+ }
+ return;
}
- return;
- }
- // Handle persona
- Profile profile = null;
- Session session = null;
- String personaId = request.getParameter("personaId");
- if (personaId != null) {
- PersonaWithSessions personaWithSessions = profileService.loadPersonaWithSessions(personaId);
- if (personaWithSessions == null) {
- logger.error("Couldn't find persona with id=" + personaId);
- profile = null;
- } else {
- profile = personaWithSessions.getPersona();
- session = personaWithSessions.getLastSession();
+ // Handle persona
+ Profile profile = null;
+ Session session = null;
+ String personaId = request.getParameter("personaId");
+ if (personaId != null) {
+ PersonaWithSessions personaWithSessions = profileService.loadPersonaWithSessions(personaId);
+ if (personaWithSessions == null) {
+ logger.error("Couldn't find persona with id=" + personaId);
+ profile = null;
+ } else {
+ profile = personaWithSessions.getPersona();
+ session = personaWithSessions.getLastSession();
+ }
}
- }
- // Extract payload
- ContextRequest contextRequest = null;
- String scope = null;
- String sessionId = null;
- String profileId = null;
- String stringPayload = HttpUtils.getPayload(request);
- if (stringPayload != null) {
- ObjectMapper mapper = CustomObjectMapper.getObjectMapper();
- JsonFactory factory = mapper.getFactory();
- try {
- contextRequest = mapper.readValue(factory.createParser(stringPayload), ContextRequest.class);
- } catch (Exception e) {
- ((HttpServletResponse)response).sendError(HttpServletResponse.SC_BAD_REQUEST, "Check logs for more details");
- logger.error("Cannot read payload " + stringPayload, e);
- return;
- }
- if (contextRequest.getSource() != null) {
- scope = contextRequest.getSource().getScope();
+ // Extract payload
+ ContextRequest contextRequest = null;
+ String scope = null;
+ String sessionId = null;
+ String profileId = null;
+ String stringPayload = HttpUtils.getPayload(request);
+ if (stringPayload != null) {
+ ObjectMapper mapper = CustomObjectMapper.getObjectMapper();
+ JsonFactory factory = mapper.getFactory();
+ try {
+ contextRequest = mapper.readValue(factory.createParser(stringPayload), ContextRequest.class);
+ } catch (Exception e) {
+ ((HttpServletResponse)response).sendError(HttpServletResponse.SC_BAD_REQUEST, "Check logs for more details");
+ logger.error("Cannot read payload " + stringPayload, e);
+ return;
+ }
+ if (contextRequest.getSource() != null) {
+ scope = contextRequest.getSource().getScope();
+ }
+ sessionId = contextRequest.getSessionId();
+ profileId = contextRequest.getProfileId();
}
- sessionId = contextRequest.getSessionId();
- profileId = contextRequest.getProfileId();
- }
- if (sessionId == null) {
- sessionId = request.getParameter("sessionId");
- }
+ if (sessionId == null) {
+ sessionId = request.getParameter("sessionId");
+ }
- if (profileId == null) {
- // Get profile id from the cookie
- profileId = ServletCommon.getProfileIdCookieValue(request, profileIdCookieName);
- }
+ if (profileId == null) {
+ // Get profile id from the cookie
+ profileId = ServletCommon.getProfileIdCookieValue(request, profileIdCookieName);
+ }
- if (profileId == null && sessionId == null && personaId == null) {
- ((HttpServletResponse)response).sendError(HttpServletResponse.SC_BAD_REQUEST, "Check logs for more details");
- logger.error("Couldn't find profileId, sessionId or personaId in incoming request! Stopped processing request. See debug level for more information");
- if (logger.isDebugEnabled()) {
- logger.debug("Request dump: {}", HttpUtils.dumpRequestInfo(request));
+ if (profileId == null && sessionId == null && personaId == null) {
+ ((HttpServletResponse)response).sendError(HttpServletResponse.SC_BAD_REQUEST, "Check logs for more details");
+ logger.error("Couldn't find profileId, sessionId or personaId in incoming request! Stopped processing request. See debug level for more information");
+ if (logger.isDebugEnabled()) {
+ logger.debug("Request dump: {}", HttpUtils.dumpRequestInfo(request));
+ }
+ return;
}
- return;
- }
- int changes = EventService.NO_CHANGE;
- if (profile == null) {
- // Not a persona, resolve profile now
- boolean profileCreated = false;
-
- boolean invalidateProfile = request.getParameter("invalidateProfile") != null ?
- new Boolean(request.getParameter("invalidateProfile")) : false;
- if (profileId == null || invalidateProfile) {
- // no profileId cookie was found or the profile has to be invalidated, we generate a new one and create the profile in the profile service
- profile = createNewProfile(null, response, timestamp);
- profileCreated = true;
- } else {
- profile = profileService.load(profileId);
- if (profile == null) {
- // this can happen if we have an old cookie but have reset the server,
- // or if we merged the profiles and somehow this cookie didn't get updated.
- profile = createNewProfile(profileId, response, timestamp);
+ int changes = EventService.NO_CHANGE;
+ if (profile == null) {
+ // Not a persona, resolve profile now
+ boolean profileCreated = false;
+
+ boolean invalidateProfile = request.getParameter("invalidateProfile") != null ?
+ new Boolean(request.getParameter("invalidateProfile")) : false;
+ if (profileId == null || invalidateProfile) {
+ // no profileId cookie was found or the profile has to be invalidated, we generate a new one and create the profile in the profile service
+ profile = createNewProfile(null, response, timestamp);
profileCreated = true;
} else {
- Changes changesObject = checkMergedProfile(response, profile, session);
- changes |= changesObject.getChangeType();
- profile = changesObject.getProfile();
+ profile = profileService.load(profileId);
+ if (profile == null) {
+ // this can happen if we have an old cookie but have reset the server,
+ // or if we merged the profiles and somehow this cookie didn't get updated.
+ profile = createNewProfile(profileId, response, timestamp);
+ profileCreated = true;
+ } else {
+ Changes changesObject = checkMergedProfile(response, profile, session);
+ changes |= changesObject.getChangeType();
+ profile = changesObject.getProfile();
+ }
}
- }
- Profile sessionProfile;
- boolean invalidateSession = request.getParameter("invalidateSession") != null ?
- new Boolean(request.getParameter("invalidateSession")) : false;
- if (StringUtils.isNotBlank(sessionId) && !invalidateSession) {
- session = profileService.loadSession(sessionId, timestamp);
- if (session != null) {
- sessionProfile = session.getProfile();
-
- boolean anonymousSessionProfile = sessionProfile.isAnonymousProfile();
- if (!profile.isAnonymousProfile() && !anonymousSessionProfile && !profile.getItemId().equals(sessionProfile.getItemId())) {
- // Session user has been switched, profile id in cookie is not up to date
- // We must reload the profile with the session ID as some properties could be missing from the session profile
- // #personalIdentifier
- profile = profileService.load(sessionProfile.getItemId());
- if (profile != null) {
- HttpUtils.sendProfileCookie(profile, response, profileIdCookieName, profileIdCookieDomain, profileIdCookieMaxAgeInSeconds);
- } else {
- logger.warn("Couldn't load profile {} referenced in session {}", sessionProfile.getItemId(), session.getItemId());
+ Profile sessionProfile;
+ boolean invalidateSession = request.getParameter("invalidateSession") != null ?
+ new Boolean(request.getParameter("invalidateSession")) : false;
+ if (StringUtils.isNotBlank(sessionId) && !invalidateSession) {
+ session = profileService.loadSession(sessionId, timestamp);
+ if (session != null) {
+ sessionProfile = session.getProfile();
+
+ boolean anonymousSessionProfile = sessionProfile.isAnonymousProfile();
+ if (!profile.isAnonymousProfile() && !anonymousSessionProfile && !profile.getItemId().equals(sessionProfile.getItemId())) {
+ // Session user has been switched, profile id in cookie is not up to date
+ // We must reload the profile with the session ID as some properties could be missing from the session profile
+ // #personalIdentifier
+ profile = profileService.load(sessionProfile.getItemId());
+ if (profile != null) {
+ HttpUtils.sendProfileCookie(profile, response, profileIdCookieName, profileIdCookieDomain, profileIdCookieMaxAgeInSeconds);
+ } else {
+ logger.warn("Couldn't load profile {} referenced in session {}", sessionProfile.getItemId(), session.getItemId());
+ }
}
- }
- // Handle anonymous situation
- Boolean requireAnonymousBrowsing = privacyService.isRequireAnonymousBrowsing(profile);
- if (requireAnonymousBrowsing && anonymousSessionProfile) {
- // User wants to browse anonymously, anonymous profile is already set.
- } else if (requireAnonymousBrowsing && !anonymousSessionProfile) {
- // User wants to browse anonymously, update the sessionProfile to anonymous profile
- sessionProfile = privacyService.getAnonymousProfile(profile);
- session.setProfile(sessionProfile);
- changes |= EventService.SESSION_UPDATED;
- } else if (!requireAnonymousBrowsing && anonymousSessionProfile) {
- // User does not want to browse anonymously anymore, update the sessionProfile to real profile
- sessionProfile = profile;
- session.setProfile(sessionProfile);
- changes |= EventService.SESSION_UPDATED;
- } else if (!requireAnonymousBrowsing && !anonymousSessionProfile) {
- // User does not want to browse anonymously, use the real profile. Check that session contains the current profile.
- sessionProfile = profile;
- if (!session.getProfileId().equals(sessionProfile.getItemId())) {
+ // Handle anonymous situation
+ Boolean requireAnonymousBrowsing = privacyService.isRequireAnonymousBrowsing(profile);
+ if (requireAnonymousBrowsing && anonymousSessionProfile) {
+ // User wants to browse anonymously, anonymous profile is already set.
+ } else if (requireAnonymousBrowsing && !anonymousSessionProfile) {
+ // User wants to browse anonymously, update the sessionProfile to anonymous profile
+ sessionProfile = privacyService.getAnonymousProfile(profile);
+ session.setProfile(sessionProfile);
+ changes |= EventService.SESSION_UPDATED;
+ } else if (!requireAnonymousBrowsing && anonymousSessionProfile) {
+ // User does not want to browse anonymously anymore, update the sessionProfile to real profile
+ sessionProfile = profile;
+ session.setProfile(sessionProfile);
changes |= EventService.SESSION_UPDATED;
+ } else if (!requireAnonymousBrowsing && !anonymousSessionProfile) {
+ // User does not want to browse anonymously, use the real profile. Check that session contains the current profile.
+ sessionProfile = profile;
+ if (!session.getProfileId().equals(sessionProfile.getItemId())) {
+ changes |= EventService.SESSION_UPDATED;
+ }
+ session.setProfile(sessionProfile);
}
- session.setProfile(sessionProfile);
}
}
- }
- if (session == null || invalidateSession) {
- sessionProfile = privacyService.isRequireAnonymousBrowsing(profile) ? privacyService.getAnonymousProfile(profile) : profile;
-
- if (StringUtils.isNotBlank(sessionId)) {
- // Only save session and send event if a session id was provided, otherwise keep transient session
- session = new Session(sessionId, sessionProfile, timestamp, scope);
- changes |= EventService.SESSION_UPDATED;
- Event event = new Event("sessionCreated", session, profile, scope, null, session, timestamp);
- if (sessionProfile.isAnonymousProfile()) {
- // Do not keep track of profile in event
- event.setProfileId(null);
- }
- event.getAttributes().put(Event.HTTP_REQUEST_ATTRIBUTE, request);
- event.getAttributes().put(Event.HTTP_RESPONSE_ATTRIBUTE, response);
- if (logger.isDebugEnabled()) {
- logger.debug("Received event {} for profile={} session={} target={} timestamp={}",
- event.getEventType(), profile.getItemId(), session.getItemId(), event.getTarget(), timestamp);
+ if (session == null || invalidateSession) {
+ sessionProfile = privacyService.isRequireAnonymousBrowsing(profile) ? privacyService.getAnonymousProfile(profile) : profile;
+
+ if (StringUtils.isNotBlank(sessionId)) {
+ // Only save session and send event if a session id was provided, otherwise keep transient session
+ session = new Session(sessionId, sessionProfile, timestamp, scope);
+ changes |= EventService.SESSION_UPDATED;
+ Event event = new Event("sessionCreated", session, profile, scope, null, session, timestamp);
+ if (sessionProfile.isAnonymousProfile()) {
+ // Do not keep track of profile in event
+ event.setProfileId(null);
+ }
+ event.getAttributes().put(Event.HTTP_REQUEST_ATTRIBUTE, request);
+ event.getAttributes().put(Event.HTTP_RESPONSE_ATTRIBUTE, response);
+ if (logger.isDebugEnabled()) {
+ logger.debug("Received event {} for profile={} session={} target={} timestamp={}",
+ event.getEventType(), profile.getItemId(), session.getItemId(), event.getTarget(), timestamp);
+ }
+ changes |= eventService.send(event);
}
- changes |= eventService.send(event);
}
- }
- if (profileCreated) {
- changes |= EventService.PROFILE_UPDATED;
+ if (profileCreated) {
+ changes |= EventService.PROFILE_UPDATED;
- Event profileUpdated = new Event("profileUpdated", session, profile, scope, null, profile, timestamp);
- profileUpdated.setPersistent(false);
- profileUpdated.getAttributes().put(Event.HTTP_REQUEST_ATTRIBUTE, request);
- profileUpdated.getAttributes().put(Event.HTTP_RESPONSE_ATTRIBUTE, response);
+ Event profileUpdated = new Event("profileUpdated", session, profile, scope, null, profile, timestamp);
+ profileUpdated.setPersistent(false);
+ profileUpdated.getAttributes().put(Event.HTTP_REQUEST_ATTRIBUTE, request);
+ profileUpdated.getAttributes().put(Event.HTTP_RESPONSE_ATTRIBUTE, response);
- if (logger.isDebugEnabled()) {
- logger.debug("Received event {} for profile={} {} target={} timestamp={}", profileUpdated.getEventType(), profile.getItemId(),
- " session=" + (session != null ? session.getItemId() : null), profileUpdated.getTarget(), timestamp);
+ if (logger.isDebugEnabled()) {
+ logger.debug("Received event {} for profile={} {} target={} timestamp={}", profileUpdated.getEventType(), profile.getItemId(),
+ " session=" + (session != null ? session.getItemId() : null), profileUpdated.getTarget(), timestamp);
+ }
+ changes |= eventService.send(profileUpdated);
}
- changes |= eventService.send(profileUpdated);
}
- }
- ContextResponse contextResponse = new ContextResponse();
- contextResponse.setProfileId(profile.getItemId());
- if (session != null) {
- contextResponse.setSessionId(session.getItemId());
- } else if (sessionId != null) {
- contextResponse.setSessionId(sessionId);
- }
+ ContextResponse contextResponse = new ContextResponse();
+ contextResponse.setProfileId(profile.getItemId());
+ if (session != null) {
+ contextResponse.setSessionId(session.getItemId());
+ } else if (sessionId != null) {
+ contextResponse.setSessionId(sessionId);
+ }
- if (contextRequest != null) {
- Changes changesObject = handleRequest(contextRequest, session, profile, contextResponse, request, response, timestamp);
- changes |= changesObject.getChangeType();
- profile = changesObject.getProfile();
- }
+ if (contextRequest != null) {
+ Changes changesObject = handleRequest(contextRequest, session, profile, contextResponse, request, response, timestamp);
+ changes |= changesObject.getChangeType();
+ profile = changesObject.getProfile();
+ }
- if ((changes & EventService.PROFILE_UPDATED) == EventService.PROFILE_UPDATED) {
- profileService.save(profile);
- contextResponse.setProfileId(profile.getItemId());
- }
- if ((changes & EventService.SESSION_UPDATED) == EventService.SESSION_UPDATED && session != null) {
- profileService.saveSession(session);
- contextResponse.setSessionId(session.getItemId());
- }
+ if ((changes & EventService.PROFILE_UPDATED) == EventService.PROFILE_UPDATED) {
+ profileService.save(profile);
+ contextResponse.setProfileId(profile.getItemId());
+ }
+ if ((changes & EventService.SESSION_UPDATED) == EventService.SESSION_UPDATED && session != null) {
+ profileService.saveSession(session);
+ contextResponse.setSessionId(session.getItemId());
+ }
- if ((changes & EventService.ERROR) == EventService.ERROR) {
- response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
- }
+ if ((changes & EventService.ERROR) == EventService.ERROR) {
+ response.setStatus(HttpServletResponse.SC_INTERNAL_SERVER_ERROR);
+ }
- String extension = request.getRequestURI().substring(request.getRequestURI().lastIndexOf(".") + 1);
- boolean noScript = "json".equals(extension);
- String contextAsJSONString = CustomObjectMapper.getObjectMapper().writeValueAsString(contextResponse);
- Writer responseWriter;
- response.setCharacterEncoding("UTF-8");
- if (noScript) {
- responseWriter = response.getWriter();
- response.setContentType("application/json");
- IOUtils.write(contextAsJSONString, responseWriter);
- } else {
- responseWriter = response.getWriter();
- responseWriter.append("window.digitalData = window.digitalData || {};\n")
- .append("var cxs = ")
- .append(contextAsJSONString)
- .append(";\n");
- }
+ String extension = request.getRequestURI().substring(request.getRequestURI().lastIndexOf(".") + 1);
+ boolean noScript = "json".equals(extension);
+ String contextAsJSONString = CustomObjectMapper.getObjectMapper().writeValueAsString(contextResponse);
+ Writer responseWriter;
+ response.setCharacterEncoding("UTF-8");
+ if (noScript) {
+ responseWriter = response.getWriter();
+ response.setContentType("application/json");
+ IOUtils.write(contextAsJSONString, responseWriter);
+ } else {
+ responseWriter = response.getWriter();
+ responseWriter.append("window.digitalData = window.digitalData || {};\n")
+ .append("var cxs = ")
+ .append(contextAsJSONString)
+ .append(";\n");
+ }
- responseWriter.flush();
+ responseWriter.flush();
+ } catch (Throwable t) { // Here in order to return generic message instead of the whole stack trace in case of not caught exception
+ logger.error("ContextServlet failed to execute request", t);
+ response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Internal server error");
+ }
}
private Changes checkMergedProfile(ServletResponse response, Profile profile, Session session) {
diff --git a/wab/src/main/java/org/apache/unomi/web/EventsCollectorServlet.java b/wab/src/main/java/org/apache/unomi/web/EventsCollectorServlet.java
index 2c34659..313b25b 100644
--- a/wab/src/main/java/org/apache/unomi/web/EventsCollectorServlet.java
+++ b/wab/src/main/java/org/apache/unomi/web/EventsCollectorServlet.java
@@ -62,19 +62,35 @@ public class EventsCollectorServlet extends HttpServlet {
@Override
protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
- doEvent(req, resp);
+ try {
+ doEvent(req, resp);
+ } catch (Throwable t) { // Here in order to return generic message instead of the whole stack trace in case of not caught exception
+ logger.error("EventsCollectorServlet failed to execute get", t);
+ resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Internal server error");
+ }
}
@Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
- doEvent(req, resp);
+ try {
+ doEvent(req, resp);
+ } catch (Throwable t) { // Here in order to return generic message instead of the whole stack trace in case of not caught exception
+ logger.error("EventsCollectorServlet failed to execute post", t);
+ resp.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Internal server error");
+ }
}
@Override
protected void doOptions(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
// logger.debug(HttpUtils.dumpRequestInfo(request));
- HttpUtils.setupCORSHeaders(request, response);
- response.flushBuffer();
+ try {
+ HttpUtils.setupCORSHeaders(request, response);
+ response.flushBuffer();
+ }
+ catch (Throwable t) { // Here in order to return generic message instead of the whole stack trace in case of not caught exception
+ logger.error("EventsCollectorServlet failed to execute doOptions request", t);
+ response.sendError(HttpServletResponse.SC_INTERNAL_SERVER_ERROR, "Internal server error");
+ }
}
private void doEvent(HttpServletRequest request, HttpServletResponse response) throws IOException {
[unomi] 02/15: Setup code analysis
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit d9372b0d03a304fe9bef6eef657d492380da1d5d
Author: Serge Huber <sh...@jahia.com>
AuthorDate: Wed Dec 9 15:53:50 2020 +0100
Setup code analysis
(cherry picked from commit a507540601211e85d41448b2a93f0c8aa8368c10)
---
.github/workflows/codeql-analysis.yml | 67 +++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
new file mode 100644
index 0000000..f631368
--- /dev/null
+++ b/.github/workflows/codeql-analysis.yml
@@ -0,0 +1,67 @@
+# For most projects, this workflow file will not need changing; you simply need
+# to commit it to your repository.
+#
+# You may wish to alter this file to override the set of languages analyzed,
+# or to provide custom queries or build logic.
+#
+# ******** NOTE ********
+# We have attempted to detect the languages in your repository. Please check
+# the `language` matrix defined below to confirm you have the correct set of
+# supported CodeQL languages.
+#
+name: "CodeQL"
+
+on:
+ push:
+ branches: [ master ]
+ pull_request:
+ # The branches below must be a subset of the branches above
+ branches: [ master ]
+ schedule:
+ - cron: '38 1 * * 0'
+
+jobs:
+ analyze:
+ name: Analyze
+ runs-on: ubuntu-latest
+
+ strategy:
+ fail-fast: false
+ matrix:
+ language: [ 'java', 'javascript' ]
+ # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
+ # Learn more:
+ # https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
+
+ steps:
+ - name: Checkout repository
+ uses: actions/checkout@v2
+
+ # Initializes the CodeQL tools for scanning.
+ - name: Initialize CodeQL
+ uses: github/codeql-action/init@v1
+ with:
+ languages: ${{ matrix.language }}
+ # If you wish to specify custom queries, you can do so here or in a config file.
+ # By default, queries listed here will override any specified in a config file.
+ # Prefix the list here with "+" to use these queries and those in the config file.
+ # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+ # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
+ # If this step fails, then you should remove it and run the build manually (see below)
+ - name: Autobuild
+ uses: github/codeql-action/autobuild@v1
+
+ # âšī¸ Command-line programs to run using the OS shell.
+ # đ https://git.io/JvXDl
+
+ # âī¸ If the Autobuild fails above, remove it and uncomment the following three lines
+ # and modify them (or add more) to build your code if your project
+ # uses a compiled language
+
+ #- run: |
+ # make bootstrap
+ # make release
+
+ - name: Perform CodeQL Analysis
+ uses: github/codeql-action/analyze@v1
[unomi] 06/15: Skip tests in jenkins deploy stage (#227)
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit 3503efce0ab6fab3346b8dbeaa3d1d2865820f07
Author: MT BENTERKI <be...@gmail.com>
AuthorDate: Tue Jan 5 14:33:59 2021 +0100
Skip tests in jenkins deploy stage (#227)
(cherry picked from commit 0d6417faa5bddc57857d2ea4183403c74e9adb28)
---
Jenkinsfile | 4 ++--
Jenkinsfile-jdk11 | 4 ++--
2 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/Jenkinsfile b/Jenkinsfile
index 37620f4..e777517 100644
--- a/Jenkinsfile
+++ b/Jenkinsfile
@@ -116,7 +116,7 @@ pipeline {
}
steps {
echo 'Deploying'
- sh 'mvn deploy'
+ sh 'mvn deploy -DskipTests'
}
}
}
@@ -178,4 +178,4 @@ Is back to normal.
}
}
-}
\ No newline at end of file
+}
diff --git a/Jenkinsfile-jdk11 b/Jenkinsfile-jdk11
index aaaab69..dd3f5d2 100644
--- a/Jenkinsfile-jdk11
+++ b/Jenkinsfile-jdk11
@@ -116,7 +116,7 @@ pipeline {
}
steps {
echo 'Deploying'
- sh 'mvn deploy'
+ sh 'mvn deploy -DskipTests'
}
}
}
@@ -178,4 +178,4 @@ Is back to normal.
}
}
-}
\ No newline at end of file
+}
[unomi] 03/15: Fix build broken after committing github
configuration file.
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch unomi-1.5.x
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit f16e47368fa8b72067253eaddc9ca1d371c802f3
Author: Serge Huber <sh...@jahia.com>
AuthorDate: Fri Dec 11 11:10:33 2020 +0100
Fix build broken after committing github configuration file.
(cherry picked from commit 6cb3eb642ab71c62ef321b068bf47cba354ebcf5)
---
pom.xml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/pom.xml b/pom.xml
index c71ed9d..8ed77d1 100644
--- a/pom.xml
+++ b/pom.xml
@@ -481,6 +481,8 @@
<exclude>**/NOTICE-generated</exclude>
<exclude>**/DEPENDENCIES</exclude>
<exclude>**/target/**/*</exclude>
+ <!-- Github configuration files -->
+ <exclude>**/.github/**/*</exclude>
<!-- GIT files -->
<exclude>**/.git/**/*</exclude>
<!-- IDEA files -->