You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by sh...@apache.org on 2020/06/05 14:55:48 UTC
[unomi] branch UNOMI-341-refactoring-graphql-servicemanager
updated: Reorganize documentation - Extract configuration into seperate
section - Add new migration section - Done for master & 1.5.x documentation
This is an automated email from the ASF dual-hosted git repository.
shuber pushed a commit to branch UNOMI-341-refactoring-graphql-servicemanager
in repository https://gitbox.apache.org/repos/asf/unomi.git
The following commit(s) were added to refs/heads/UNOMI-341-refactoring-graphql-servicemanager by this push:
new 4025b38 Reorganize documentation - Extract configuration into seperate section - Add new migration section - Done for master & 1.5.x documentation
4025b38 is described below
commit 4025b384934f4a969b907bc2e89fa6e6e3280e27
Author: Serge Huber <sh...@apache.org>
AuthorDate: Fri Jun 5 16:55:19 2020 +0200
Reorganize documentation
- Extract configuration into seperate section
- Add new migration section
- Done for master & 1.5.x documentation
---
.../src/archives/1.5/asciidoc/configuration.adoc | 34 +++++++--------
manual/src/archives/1.5/asciidoc/index.adoc | 14 +++---
.../asciidoc/migrations/migrate-1.4-to-1.5.adoc} | 51 ++++++++++++++--------
.../1.5/asciidoc/migrations/migrations.adoc | 20 +++++++++
manual/src/main/asciidoc/configuration.adoc | 34 +++++++--------
manual/src/main/asciidoc/index.adoc | 14 +++---
.../asciidoc/migrations/migrate-1.4-to-1.5.adoc} | 51 ++++++++++++++--------
.../src/main/asciidoc/migrations/migrations.adoc | 20 +++++++++
8 files changed, 154 insertions(+), 84 deletions(-)
diff --git a/manual/src/archives/1.5/asciidoc/configuration.adoc b/manual/src/archives/1.5/asciidoc/configuration.adoc
index 688d8ad..ee753d0 100644
--- a/manual/src/archives/1.5/asciidoc/configuration.adoc
+++ b/manual/src/archives/1.5/asciidoc/configuration.adoc
@@ -11,9 +11,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//
-=== Configuration
+== Configuration
-==== Centralized configuration
+=== Centralized configuration
Apache Unomi uses a centralized configuration file that contains both system properties and configuration properties.
These settings are then fed to the OSGi and other configuration files using placeholder that look something like this:
@@ -32,7 +32,7 @@ this file directly, as an override mechanism is available. Simply create a file
and put your own property values in their to override the defaults OR you can use environment variables to also override
the values in the `$MY_KARAF_HOME/etc/custom.system.properties`. See the next section for more information about that.
-==== Changing the default configuration using environment variables (i.e. Docker configuration)
+=== Changing the default configuration using environment variables (i.e. Docker configuration)
You might want to use environment variables to change the default system configuration, especially if you intend to run
Apache Unomi inside a Docker container. You can find the list of all the environment variable names in the following file:
@@ -45,7 +45,7 @@ Docker Compose you can put the environment variables in the docker-compose.yml f
If you want to "save" the environment values in a file, you can use the `bin/setenv(.bat)` to setup the environment
variables you want to use.
-==== Changing the default configuration using property files
+=== Changing the default configuration using property files
If you want to change the default configuration using property files instead of environment variables, you can perform
any modification you want in the `$MY_KARAF_HOME/etc/unomi.custom.system.properties` file.
@@ -84,7 +84,7 @@ org.apache.unomi.elasticsearch.cluster.name=contextElasticSearch
org.apache.unomi.elasticsearch.addresses=localhost:9200
----
-==== Secured events configuration
+=== Secured events configuration
Apache Unomi secures some events by default. It comes out of the box with a default configuration that you can adjust
by using the centralized configuration file override in `$MY_KARAF_HOME/etc/unomi.custom.system.properties`
@@ -146,7 +146,7 @@ thirdparty.provider1.allowedEvents=login,updateProperties
----
-==== Installing the MaxMind GeoIPLite2 IP lookup database
+=== Installing the MaxMind GeoIPLite2 IP lookup database
Apache Unomi requires an IP database in order to resolve IP addresses to user location.
The GeoLite2 database can be downloaded from MaxMind here :
@@ -154,7 +154,7 @@ http://dev.maxmind.com/geoip/geoip2/geolite2/[http://dev.maxmind.com/geoip/geoip
Simply download the GeoLite2-City.mmdb file into the "etc" directory.
-==== Installing Geonames database
+=== Installing Geonames database
Apache Unomi includes a geocoding service based on the geonames database ( http://www.geonames.org/[http://www.geonames.org/] ). It can be
used to create conditions on countries or cities.
@@ -168,7 +168,7 @@ import should start right away.
Otherwise, import should start at the next startup. Import runs in background, but can take about 15 minutes.
At the end, you should have about 4 million entries in the geonames index.
-==== REST API Security
+=== REST API Security
The Apache Unomi Context Server REST API is protected using JAAS authentication and using Basic or Digest HTTP auth.
By default, the login/password for the REST API full administrative access is "karaf/karaf".
@@ -193,7 +193,7 @@ You should now have SSL setup on Karaf with your certificate, and you can test i
Changing the default Karaf password can be done by modifying the `org.apache.unomi.security.root.password` in the
`$MY_KARAF_HOME/etc/unomi.custom.system.properties` file
-==== Scripting security
+=== Scripting security
By default, scripting (using in conditions, segments and rules) is controlled by a custom classloader that is quite
restrictive and using a white-list/black list system. It is controlled through the following property in the
@@ -211,7 +211,7 @@ restrictive configuration.
If you need, for example when adding a custom item type, to adjust these, please be careful as scripts may be called
directly from the context.json personalization conditions and therefore should be kept minimal.
-==== Automatic profile merging
+=== Automatic profile merging
Apache Unomi is capable of merging profiles based on a common property value. In order to use this, you must
add the MergeProfileOnPropertyAction to a rule (such as a login rule for example), and configure it with the name
@@ -226,7 +226,7 @@ will use the merged profile.
To test, simply configure the action in the "login" or "facebookLogin" rules and set it up on the "email" property.
Upon sending one of the events, all matching profiles will be merged.
-==== Securing a production environment
+=== Securing a production environment
Before going live with a project, you should _absolutely_ read the following section that will help you setup a proper
secure environment for running your context server.
@@ -293,7 +293,7 @@ Step 4 : Setup a proxy in front of the context server
As an alternative to an application-level firewall, you could also route all traffic to the context server through
a proxy, and use it to filter any communication.
-==== Integrating with an Apache HTTP web server
+=== Integrating with an Apache HTTP web server
If you want to setup an Apache HTTP web server in from of Apache Unomi, here is an example configuration using
mod_proxy.
@@ -383,7 +383,7 @@ ProxyPass / http://localhost:8181/ connectiontimeout=20 timeout=300 ttl=120
ProxyPassReverse / http://localhost:8181/
----
-==== Changing the default tracking location
+=== Changing the default tracking location
When performing localhost requests to Apache Unomi, a default location will be used to insert values into the session
to make the location-based personalization still work. You can modify the default location settings using the
@@ -407,7 +407,7 @@ org.apache.unomi.ip.default.longitude=${env:UNOMI_IP_DEFAULT_LONGITUDE:-6.128250
You might want to change these for testing or for demonstration purposes.
-==== Apache Karaf SSH Console
+=== Apache Karaf SSH Console
The Apache Karaf SSH console is available inside Apache Unomi, but the port has been changed from the default value of
8101 to 8102 to avoid conflicts with other Karaf-based products. So to connect to the SSH console you should use:
@@ -420,13 +420,13 @@ ssh -p 8102 karaf@localhost
or the user/password you have setup to protect the system if you have changed it. You can find the list of Apache Unomi
shell commands in the "Shell commands" section of the documentation.
-==== ElasticSearch authentication and security
+=== ElasticSearch authentication and security
With ElasticSearch 7, it's possible to secure the access to your data. (https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html])
Depending on your ElasticSearch license you may need to install Kibana and enable xpack security: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html]
-===== User authentication !
+==== User authentication !
If your ElasticSearch have been configured to be only accessible by authenticated users (https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html])
@@ -438,7 +438,7 @@ username=USER
password=PASSWORD
----
-===== SSL communication
+==== SSL communication
By default Unomi will communicate with ElasticSearch using `http`
but you can configure your ElasticSearch server(s) to allow encrypted request using `https`.
diff --git a/manual/src/archives/1.5/asciidoc/index.adoc b/manual/src/archives/1.5/asciidoc/index.adoc
index acb2e63..0d487fc 100644
--- a/manual/src/archives/1.5/asciidoc/index.adoc
+++ b/manual/src/archives/1.5/asciidoc/index.adoc
@@ -40,11 +40,7 @@ include::web-tracker.adoc[]
include::configuration.adoc[]
-include::useful-unomi-urls.adoc[]
-
-include::how-profile-tracking-works.adoc[]
-
-include::context-request-flow.adoc[]
+include::migrations/migrations.adoc[]
== Queries and aggregations
@@ -68,9 +64,13 @@ include::clustering.adoc[]
== Reference
-include::datamodel.adoc[]
+include::useful-unomi-urls.adoc[]
+
+include::how-profile-tracking-works.adoc[]
-include::new-data-model.adoc[]
+include::context-request-flow.adoc[]
+
+include::datamodel.adoc[]
include::builtin-event-types.adoc[]
diff --git a/manual/src/main/asciidoc/new-data-model.adoc b/manual/src/archives/1.5/asciidoc/migrations/migrate-1.4-to-1.5.adoc
similarity index 72%
rename from manual/src/main/asciidoc/new-data-model.adoc
rename to manual/src/archives/1.5/asciidoc/migrations/migrate-1.4-to-1.5.adoc
index 0b98dfd..0bc1914 100644
--- a/manual/src/main/asciidoc/new-data-model.adoc
+++ b/manual/src/archives/1.5/asciidoc/migrations/migrate-1.4-to-1.5.adoc
@@ -12,8 +12,6 @@
// limitations under the License.
//
-=== Data Model changes for Apache Unomi 1.5.0
-
==== Data model and ElasticSearch 7
Since Apache Unomi version 1.5.0 we decided to upgrade the supported ElasticSearch version to the latest 7.4.2.
@@ -51,26 +49,30 @@ Because of this changes the geonames DB index name is now respecting the index n
Previously named: `geonames` is now using the index name `context-geonameentry`
(see: link:#_installing_geonames_database[Documentation about geonames extension]).
-===== Migration
+==== Migration steps
In order to migrate the data from ElasticSearch 5 to 7, Unomi provides a migration tool that is directly integrated.
-In these migration steps the following is assumed:
+In this migration the following is assumed:
+
+- the ElasticSearch 5 cluster installation is referred to as the `source`
+- the ElasticSearch 7 cluster installation is referred to as the `target`
+- the Unomi 1.4 cluster installation is completely stopped
+- the Unomi 1.5 cluster installation has never been started (just uncompressed)
+- the Unomi 1.5 cluster installation has been configured to connect to the `target` (ElasticSearch 7) cluster
-- the ElasticSearch 5 installation is referred to as the `source`
-- the ElasticSearch 7 installation is referred to as the `target`
-- the Unomi 1.4 installation is completely stopped
-- the Unomi 1.5 installation has never been started (just uncompressed)
-- the Unomi 1.5 installation has been configured to connect to the `target` (ElasticSearch 7) cluster
+It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the `source` clusters (including ElasticSearch and
+Unomi clusters), and ideally to perform the migration on a restored snapshot of the `source` cluster. For more information
+on ElasticSearch 5 snapshots and restore you can find it here:
-It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the `source` cluster, and ideally to perform the
-migration on a restore of the `source` cluster. For more information on ElasticSearch 5 snapshots and restore you can
-find it here: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
+ https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
-Note that it is possible to do the migration procedure on a single machine, but you will need to change the ports on one
-of the ElasticSearch cluster. In the following example we have changed the ports on the `source` cluster, which requires
-a cluster restart to take effect. It is also possible to change the ports on the `target` cluster but those would have
-to be then changed again to your final setting.
+The way the migration works is that both ElasticSearch 5 AND an ElasticSearch 7 clusters (or just single nodes) will
+be started at the same time, and data will be migrated from the ES 5 to the ES 7 cluster. Note that it is possible to use
+a single node for both the `source` and the `target` clusters to - for example - perform the migration on a single
+machine. If you choose to do that you will have to adjust port numbers on either the `source` or `target` cluster node.
+Changing ports requires a restart of the ES cluster you are modifying. In this example we will illustrate how to migrate
+by modifying the `source` cluster node ports.
So in the `source` 's ElasticSearch 5 `config/elasticsearch.yml` file we have modified the default ports to:
@@ -99,6 +101,10 @@ Once in the console launch the migration using the following command:
migrate 1.4.0
+Note: the 1.4.0 version is the starting version. If you are starting from a different version (for example a fork), make
+sure that you know what official version of Apache Unomi it corresponds to and you can use the official version number
+as a start version for the migration.
+
Follow the instructions and answer the prompts. If you used the above configuration as an example you can simply use the
default values.
@@ -108,5 +114,14 @@ ES 5 one.
Note that it is also possible to change the index prefix to be different from the default `context` value
so that you could host multiple Apache Unomi instances on the same ElasticSearch cluster.
-Important note: only the data that Apache Unomi manages will be migrate. If you have any other data (for example Kibana
-or ElasticSearch monitoring indices) they will not be migrated by this migration tool.
\ No newline at end of file
+Important note: only the data that Apache Unomi manages will be migrated. If you have any other data (for example Kibana
+or ElasticSearch monitoring indices) they will not be migrated by this migration tool.
+
+Once the migration has completed, you can start the new Unomi instance using:
+
+ unomi:start
+
+You should then validate that all the data has been properly migrated. For example you could issue a command to list
+the profiles:
+
+ profile-list
diff --git a/manual/src/archives/1.5/asciidoc/migrations/migrations.adoc b/manual/src/archives/1.5/asciidoc/migrations/migrations.adoc
new file mode 100644
index 0000000..03f18fc
--- /dev/null
+++ b/manual/src/archives/1.5/asciidoc/migrations/migrations.adoc
@@ -0,0 +1,20 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+== Migrations
+
+This section contains information and steps to migrate between major Unomi versions.
+
+=== From version 1.4 to 1.5
+
+include::migrate-1.4-to-1.5.adoc[]
diff --git a/manual/src/main/asciidoc/configuration.adoc b/manual/src/main/asciidoc/configuration.adoc
index 688d8ad..ee753d0 100644
--- a/manual/src/main/asciidoc/configuration.adoc
+++ b/manual/src/main/asciidoc/configuration.adoc
@@ -11,9 +11,9 @@
// See the License for the specific language governing permissions and
// limitations under the License.
//
-=== Configuration
+== Configuration
-==== Centralized configuration
+=== Centralized configuration
Apache Unomi uses a centralized configuration file that contains both system properties and configuration properties.
These settings are then fed to the OSGi and other configuration files using placeholder that look something like this:
@@ -32,7 +32,7 @@ this file directly, as an override mechanism is available. Simply create a file
and put your own property values in their to override the defaults OR you can use environment variables to also override
the values in the `$MY_KARAF_HOME/etc/custom.system.properties`. See the next section for more information about that.
-==== Changing the default configuration using environment variables (i.e. Docker configuration)
+=== Changing the default configuration using environment variables (i.e. Docker configuration)
You might want to use environment variables to change the default system configuration, especially if you intend to run
Apache Unomi inside a Docker container. You can find the list of all the environment variable names in the following file:
@@ -45,7 +45,7 @@ Docker Compose you can put the environment variables in the docker-compose.yml f
If you want to "save" the environment values in a file, you can use the `bin/setenv(.bat)` to setup the environment
variables you want to use.
-==== Changing the default configuration using property files
+=== Changing the default configuration using property files
If you want to change the default configuration using property files instead of environment variables, you can perform
any modification you want in the `$MY_KARAF_HOME/etc/unomi.custom.system.properties` file.
@@ -84,7 +84,7 @@ org.apache.unomi.elasticsearch.cluster.name=contextElasticSearch
org.apache.unomi.elasticsearch.addresses=localhost:9200
----
-==== Secured events configuration
+=== Secured events configuration
Apache Unomi secures some events by default. It comes out of the box with a default configuration that you can adjust
by using the centralized configuration file override in `$MY_KARAF_HOME/etc/unomi.custom.system.properties`
@@ -146,7 +146,7 @@ thirdparty.provider1.allowedEvents=login,updateProperties
----
-==== Installing the MaxMind GeoIPLite2 IP lookup database
+=== Installing the MaxMind GeoIPLite2 IP lookup database
Apache Unomi requires an IP database in order to resolve IP addresses to user location.
The GeoLite2 database can be downloaded from MaxMind here :
@@ -154,7 +154,7 @@ http://dev.maxmind.com/geoip/geoip2/geolite2/[http://dev.maxmind.com/geoip/geoip
Simply download the GeoLite2-City.mmdb file into the "etc" directory.
-==== Installing Geonames database
+=== Installing Geonames database
Apache Unomi includes a geocoding service based on the geonames database ( http://www.geonames.org/[http://www.geonames.org/] ). It can be
used to create conditions on countries or cities.
@@ -168,7 +168,7 @@ import should start right away.
Otherwise, import should start at the next startup. Import runs in background, but can take about 15 minutes.
At the end, you should have about 4 million entries in the geonames index.
-==== REST API Security
+=== REST API Security
The Apache Unomi Context Server REST API is protected using JAAS authentication and using Basic or Digest HTTP auth.
By default, the login/password for the REST API full administrative access is "karaf/karaf".
@@ -193,7 +193,7 @@ You should now have SSL setup on Karaf with your certificate, and you can test i
Changing the default Karaf password can be done by modifying the `org.apache.unomi.security.root.password` in the
`$MY_KARAF_HOME/etc/unomi.custom.system.properties` file
-==== Scripting security
+=== Scripting security
By default, scripting (using in conditions, segments and rules) is controlled by a custom classloader that is quite
restrictive and using a white-list/black list system. It is controlled through the following property in the
@@ -211,7 +211,7 @@ restrictive configuration.
If you need, for example when adding a custom item type, to adjust these, please be careful as scripts may be called
directly from the context.json personalization conditions and therefore should be kept minimal.
-==== Automatic profile merging
+=== Automatic profile merging
Apache Unomi is capable of merging profiles based on a common property value. In order to use this, you must
add the MergeProfileOnPropertyAction to a rule (such as a login rule for example), and configure it with the name
@@ -226,7 +226,7 @@ will use the merged profile.
To test, simply configure the action in the "login" or "facebookLogin" rules and set it up on the "email" property.
Upon sending one of the events, all matching profiles will be merged.
-==== Securing a production environment
+=== Securing a production environment
Before going live with a project, you should _absolutely_ read the following section that will help you setup a proper
secure environment for running your context server.
@@ -293,7 +293,7 @@ Step 4 : Setup a proxy in front of the context server
As an alternative to an application-level firewall, you could also route all traffic to the context server through
a proxy, and use it to filter any communication.
-==== Integrating with an Apache HTTP web server
+=== Integrating with an Apache HTTP web server
If you want to setup an Apache HTTP web server in from of Apache Unomi, here is an example configuration using
mod_proxy.
@@ -383,7 +383,7 @@ ProxyPass / http://localhost:8181/ connectiontimeout=20 timeout=300 ttl=120
ProxyPassReverse / http://localhost:8181/
----
-==== Changing the default tracking location
+=== Changing the default tracking location
When performing localhost requests to Apache Unomi, a default location will be used to insert values into the session
to make the location-based personalization still work. You can modify the default location settings using the
@@ -407,7 +407,7 @@ org.apache.unomi.ip.default.longitude=${env:UNOMI_IP_DEFAULT_LONGITUDE:-6.128250
You might want to change these for testing or for demonstration purposes.
-==== Apache Karaf SSH Console
+=== Apache Karaf SSH Console
The Apache Karaf SSH console is available inside Apache Unomi, but the port has been changed from the default value of
8101 to 8102 to avoid conflicts with other Karaf-based products. So to connect to the SSH console you should use:
@@ -420,13 +420,13 @@ ssh -p 8102 karaf@localhost
or the user/password you have setup to protect the system if you have changed it. You can find the list of Apache Unomi
shell commands in the "Shell commands" section of the documentation.
-==== ElasticSearch authentication and security
+=== ElasticSearch authentication and security
With ElasticSearch 7, it's possible to secure the access to your data. (https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/secure-cluster.html])
Depending on your ElasticSearch license you may need to install Kibana and enable xpack security: https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/configuring-security.html]
-===== User authentication !
+==== User authentication !
If your ElasticSearch have been configured to be only accessible by authenticated users (https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html[https://www.elastic.co/guide/en/elasticsearch/reference/7.5/setting-up-authentication.html])
@@ -438,7 +438,7 @@ username=USER
password=PASSWORD
----
-===== SSL communication
+==== SSL communication
By default Unomi will communicate with ElasticSearch using `http`
but you can configure your ElasticSearch server(s) to allow encrypted request using `https`.
diff --git a/manual/src/main/asciidoc/index.adoc b/manual/src/main/asciidoc/index.adoc
index acb2e63..0d487fc 100644
--- a/manual/src/main/asciidoc/index.adoc
+++ b/manual/src/main/asciidoc/index.adoc
@@ -40,11 +40,7 @@ include::web-tracker.adoc[]
include::configuration.adoc[]
-include::useful-unomi-urls.adoc[]
-
-include::how-profile-tracking-works.adoc[]
-
-include::context-request-flow.adoc[]
+include::migrations/migrations.adoc[]
== Queries and aggregations
@@ -68,9 +64,13 @@ include::clustering.adoc[]
== Reference
-include::datamodel.adoc[]
+include::useful-unomi-urls.adoc[]
+
+include::how-profile-tracking-works.adoc[]
-include::new-data-model.adoc[]
+include::context-request-flow.adoc[]
+
+include::datamodel.adoc[]
include::builtin-event-types.adoc[]
diff --git a/manual/src/archives/1.5/asciidoc/new-data-model.adoc b/manual/src/main/asciidoc/migrations/migrate-1.4-to-1.5.adoc
similarity index 72%
rename from manual/src/archives/1.5/asciidoc/new-data-model.adoc
rename to manual/src/main/asciidoc/migrations/migrate-1.4-to-1.5.adoc
index 0b98dfd..0bc1914 100644
--- a/manual/src/archives/1.5/asciidoc/new-data-model.adoc
+++ b/manual/src/main/asciidoc/migrations/migrate-1.4-to-1.5.adoc
@@ -12,8 +12,6 @@
// limitations under the License.
//
-=== Data Model changes for Apache Unomi 1.5.0
-
==== Data model and ElasticSearch 7
Since Apache Unomi version 1.5.0 we decided to upgrade the supported ElasticSearch version to the latest 7.4.2.
@@ -51,26 +49,30 @@ Because of this changes the geonames DB index name is now respecting the index n
Previously named: `geonames` is now using the index name `context-geonameentry`
(see: link:#_installing_geonames_database[Documentation about geonames extension]).
-===== Migration
+==== Migration steps
In order to migrate the data from ElasticSearch 5 to 7, Unomi provides a migration tool that is directly integrated.
-In these migration steps the following is assumed:
+In this migration the following is assumed:
+
+- the ElasticSearch 5 cluster installation is referred to as the `source`
+- the ElasticSearch 7 cluster installation is referred to as the `target`
+- the Unomi 1.4 cluster installation is completely stopped
+- the Unomi 1.5 cluster installation has never been started (just uncompressed)
+- the Unomi 1.5 cluster installation has been configured to connect to the `target` (ElasticSearch 7) cluster
-- the ElasticSearch 5 installation is referred to as the `source`
-- the ElasticSearch 7 installation is referred to as the `target`
-- the Unomi 1.4 installation is completely stopped
-- the Unomi 1.5 installation has never been started (just uncompressed)
-- the Unomi 1.5 installation has been configured to connect to the `target` (ElasticSearch 7) cluster
+It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the `source` clusters (including ElasticSearch and
+Unomi clusters), and ideally to perform the migration on a restored snapshot of the `source` cluster. For more information
+on ElasticSearch 5 snapshots and restore you can find it here:
-It is HIGHLY RECOMMENDED to perform a full cluster backup/snapshot of the `source` cluster, and ideally to perform the
-migration on a restore of the `source` cluster. For more information on ElasticSearch 5 snapshots and restore you can
-find it here: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
+ https://www.elastic.co/guide/en/elasticsearch/reference/5.6/modules-snapshots.html
-Note that it is possible to do the migration procedure on a single machine, but you will need to change the ports on one
-of the ElasticSearch cluster. In the following example we have changed the ports on the `source` cluster, which requires
-a cluster restart to take effect. It is also possible to change the ports on the `target` cluster but those would have
-to be then changed again to your final setting.
+The way the migration works is that both ElasticSearch 5 AND an ElasticSearch 7 clusters (or just single nodes) will
+be started at the same time, and data will be migrated from the ES 5 to the ES 7 cluster. Note that it is possible to use
+a single node for both the `source` and the `target` clusters to - for example - perform the migration on a single
+machine. If you choose to do that you will have to adjust port numbers on either the `source` or `target` cluster node.
+Changing ports requires a restart of the ES cluster you are modifying. In this example we will illustrate how to migrate
+by modifying the `source` cluster node ports.
So in the `source` 's ElasticSearch 5 `config/elasticsearch.yml` file we have modified the default ports to:
@@ -99,6 +101,10 @@ Once in the console launch the migration using the following command:
migrate 1.4.0
+Note: the 1.4.0 version is the starting version. If you are starting from a different version (for example a fork), make
+sure that you know what official version of Apache Unomi it corresponds to and you can use the official version number
+as a start version for the migration.
+
Follow the instructions and answer the prompts. If you used the above configuration as an example you can simply use the
default values.
@@ -108,5 +114,14 @@ ES 5 one.
Note that it is also possible to change the index prefix to be different from the default `context` value
so that you could host multiple Apache Unomi instances on the same ElasticSearch cluster.
-Important note: only the data that Apache Unomi manages will be migrate. If you have any other data (for example Kibana
-or ElasticSearch monitoring indices) they will not be migrated by this migration tool.
\ No newline at end of file
+Important note: only the data that Apache Unomi manages will be migrated. If you have any other data (for example Kibana
+or ElasticSearch monitoring indices) they will not be migrated by this migration tool.
+
+Once the migration has completed, you can start the new Unomi instance using:
+
+ unomi:start
+
+You should then validate that all the data has been properly migrated. For example you could issue a command to list
+the profiles:
+
+ profile-list
diff --git a/manual/src/main/asciidoc/migrations/migrations.adoc b/manual/src/main/asciidoc/migrations/migrations.adoc
new file mode 100644
index 0000000..03f18fc
--- /dev/null
+++ b/manual/src/main/asciidoc/migrations/migrations.adoc
@@ -0,0 +1,20 @@
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+//
+== Migrations
+
+This section contains information and steps to migrate between major Unomi versions.
+
+=== From version 1.4 to 1.5
+
+include::migrate-1.4-to-1.5.adoc[]