ElasticSearch 2.x upgrade and migration

[Serge Huber <sh...@jahia.com>]

Hi guys, 

After a LOT more work than I expected, I have finally been able to commit the upgrade to ElasticSearch 2.3.4 here : 
https://github.com/apache/incubator-unomi/pull/7

As you can see from the pull request, there were quite a few changes.

What is NOT in there is a migration procedure because we (currently) need to do the following:

- Moved the data from the data/contextElasticSearch directory to elasticsearch/data . I did this because I needed a top directory for the modules since we need the modules that ES 2 provides (Groovy, Expression). 
- More tricky : we will need to remove all the fields that contain “.” characters in them. As you can see in this ticket : 

https://issues.apache.org/jira/browse/UNOMI-28

We have a few fields like this. I’ve built a class to help parsing and escaping the dots in field names, you can find it here : 

https://github.com/apache/incubator-unomi/blob/UNOMI-28-ES-2-X-UPGRADE/persistence-elasticsearch/core/src/main/java/org/apache/unomi/persistence/elasticsearch/FieldDotEscaper.java

It is even capable of parsing JSON (built my own parser because I couldn’t figure out a way to do this “cleanly” with Jackson without modifying the JSON layout). I’ve added some unit tests for this new class since it is currently used low-level in the persistence implementation to automatically replace “.” characters when submitted. Maybe we could do this at a higher level but since the specification doesn’t forbid these we have to allow them for the moment at least.

Now all we need to do is combine this with the Karaf 4 upgrade, upgrade CXF and we’ll have a fully updated Apache Unomi.

For the migration I’m not too sure how this is best done, I’m thinking re-indexing on the old instance before moving the data over to the new one, but if anybody has some better ideas I’m *very* interested.

cheers,
  Serge…