You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by "sergehuber (via GitHub)" <gi...@apache.org> on 2026/05/18 13:05:57 UTC
[PR] UNOMI-880 (split A0): migrate Elasticsearch integration tests to Docker [unomi]
sergehuber opened a new pull request, #759:
URL: https://github.com/apache/unomi/pull/759
## Summary
First commit of the PR #757 stack split — see [`docs/PR-757-stack-extraction-tracker.md`](../blob/UNOMI-139-UNOMI-880-multitenancy/docs/PR-757-stack-extraction-tracker.md) on the monolith branch.
This A0 PR migrates the `elasticsearch` profile of `itests` from the legacy `com.github.alexcojocaru:elasticsearch-maven-plugin` (binary download + forked JVM) to `io.fabric8:docker-maven-plugin`, so the ES integration tests now run against `docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}` — matching how the `opensearch` profile already runs OpenSearch in Docker.
The change is intentionally minimal so it can be reviewed and merged independently of the rest of #757. Once merged, `git town sync --stack` propagates it through #754 → #756, and the remaining peeled phases (P / V / S / T / D) sit on top of it.
## Changes
### `itests/pom.xml` (elasticsearch profile only)
- Add `<elasticsearch.port>9400</elasticsearch.port>` property, surface it to failsafe `systemPropertyVariables`
- Replace the `elasticsearch-maven-plugin` plugin block with a `docker-maven-plugin` block (start/stop ES container, bind mount `target/snapshots_repository → /tmp/snapshots_repository`, wait on HTTP port)
- Add a `chmod -R ugo+rwx target/snapshots_repository` antrun step (Linux CI hits `access_denied` because the ES container runs as UID 1000)
### `pom.xml` (root)
- Add `<docker-maven-plugin.version>0.48.0</docker-maven-plugin.version>` property
- Add `io.fabric8:docker-maven-plugin` entry in `<pluginManagement>` so the elasticsearch profile inherits the version
## Test plan
- [ ] CI: \`Execute integration tests (elasticsearch, 9400)\` job runs ES via Docker and stays green
- [ ] CI: opensearch and unit-test jobs unaffected
- [ ] Local: `mvn -B verify -P elasticsearch -pl itests` runs the suite against a Docker ES container
- [ ] Local: `mvn -B verify -P opensearch -pl itests` still passes (no OS profile changes)
## Stack context
Part of the PR #757 split — see the tracker. Subsequent PRs in the stack rebuild from the same `backup/pre-split-757` tag (`7d5b6a43f`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests [unomi]
Posted by "Copilot (via GitHub)" <gi...@apache.org>.
Copilot commented on code in PR #759:
URL: https://github.com/apache/unomi/pull/759#discussion_r3261502842
##########
itests/pom.xml:
##########
@@ -247,43 +259,64 @@
</executions>
</plugin>
<plugin>
- <groupId>com.github.alexcojocaru</groupId>
- <artifactId>elasticsearch-maven-plugin</artifactId>
- <!-- REPLACE THE FOLLOWING WITH THE PLUGIN VERSION YOU NEED -->
- <version>6.29</version>
+ <groupId>io.fabric8</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
<configuration>
- <!-- <downloadUrl>https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.2.1-darwin-aarch64.tar
- .gz</downloadUrl> -->
- <clusterName>contextElasticSearchITests</clusterName>
- <transportPort>9500</transportPort>
- <httpPort>9400</httpPort>
- <version>${elasticsearch.test.version}</version>
- <autoCreateIndex>true</autoCreateIndex>
- <instanceStartupTimeout>120</instanceStartupTimeout>
- <environmentVariables>
- <ES_JAVA_OPTS>-Xms4g -Xmx4g</ES_JAVA_OPTS>
- </environmentVariables>
- <instanceSettings>
- <properties>
- <xpack.ml.enabled>false</xpack.ml.enabled>
- <path.repo>${project.build.directory}/snapshots_repository</path.repo>
- <cluster.routing.allocation.disk.threshold_enabled>false</cluster.routing.allocation.disk.threshold_enabled>
- <http.cors.allow-methods>OPTIONS,HEAD,GET,POST,PUT,DELETE</http.cors.allow-methods>
- <http.cors.allow-headers>Authorization,X-Requested-With,X-Auth-Token,Content-Type,Content-Length</http.cors.allow-headers>
- </properties>
- </instanceSettings>
+ <containerNamePattern>itests-elasticsearch</containerNamePattern>
+ <images>
+ <image>
+ <name>docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}</name>
+ <alias>elasticsearch</alias>
+ <run>
+ <ports>
+ <port>${elasticsearch.port}:9200</port>
+ </ports>
+ <env>
+ <discovery.type>single-node</discovery.type>
+ <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Review Comment:
The Elasticsearch container heap is hardcoded to `-Xms8g -Xmx8g`, while the OpenSearch profile uses 4g. 8g can exceed RAM limits on common CI runners and developer machines and cause the container to fail to start or the build to OOM. Consider making the heap size configurable via a Maven property (with a safer default, e.g., 4g) and/or keeping Elasticsearch and OpenSearch aligned.
##########
itests/pom.xml:
##########
@@ -247,43 +259,64 @@
</executions>
</plugin>
<plugin>
- <groupId>com.github.alexcojocaru</groupId>
- <artifactId>elasticsearch-maven-plugin</artifactId>
- <!-- REPLACE THE FOLLOWING WITH THE PLUGIN VERSION YOU NEED -->
- <version>6.29</version>
+ <groupId>io.fabric8</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
<configuration>
- <!-- <downloadUrl>https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.2.1-darwin-aarch64.tar
- .gz</downloadUrl> -->
- <clusterName>contextElasticSearchITests</clusterName>
- <transportPort>9500</transportPort>
- <httpPort>9400</httpPort>
- <version>${elasticsearch.test.version}</version>
- <autoCreateIndex>true</autoCreateIndex>
- <instanceStartupTimeout>120</instanceStartupTimeout>
- <environmentVariables>
- <ES_JAVA_OPTS>-Xms4g -Xmx4g</ES_JAVA_OPTS>
- </environmentVariables>
- <instanceSettings>
- <properties>
- <xpack.ml.enabled>false</xpack.ml.enabled>
- <path.repo>${project.build.directory}/snapshots_repository</path.repo>
- <cluster.routing.allocation.disk.threshold_enabled>false</cluster.routing.allocation.disk.threshold_enabled>
- <http.cors.allow-methods>OPTIONS,HEAD,GET,POST,PUT,DELETE</http.cors.allow-methods>
- <http.cors.allow-headers>Authorization,X-Requested-With,X-Auth-Token,Content-Type,Content-Length</http.cors.allow-headers>
- </properties>
- </instanceSettings>
+ <containerNamePattern>itests-elasticsearch</containerNamePattern>
+ <images>
+ <image>
+ <name>docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}</name>
+ <alias>elasticsearch</alias>
+ <run>
+ <ports>
+ <port>${elasticsearch.port}:9200</port>
+ </ports>
+ <env>
+ <discovery.type>single-node</discovery.type>
+ <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Review Comment:
`ES_JAVA_OPTS` is currently setting `-Dcluster.default.index.settings.number_of_replicas=0`. `-D...` only defines a JVM system property and won’t be applied as an Elasticsearch setting, so the cluster default replicas likely remain unchanged. Consider moving `cluster.default.index.settings.number_of_replicas=0` into its own `<env>` entry (like the other ES settings) or passing it as an Elasticsearch `-E` setting instead of a JVM `-D` property.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests [unomi]
Posted by "sergehuber (via GitHub)" <gi...@apache.org>.
sergehuber commented on code in PR #759:
URL: https://github.com/apache/unomi/pull/759#discussion_r3323403272
##########
itests/pom.xml:
##########
@@ -247,43 +259,64 @@
</executions>
</plugin>
<plugin>
- <groupId>com.github.alexcojocaru</groupId>
- <artifactId>elasticsearch-maven-plugin</artifactId>
- <!-- REPLACE THE FOLLOWING WITH THE PLUGIN VERSION YOU NEED -->
- <version>6.29</version>
+ <groupId>io.fabric8</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
<configuration>
- <!-- <downloadUrl>https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.2.1-darwin-aarch64.tar
- .gz</downloadUrl> -->
- <clusterName>contextElasticSearchITests</clusterName>
- <transportPort>9500</transportPort>
- <httpPort>9400</httpPort>
- <version>${elasticsearch.test.version}</version>
- <autoCreateIndex>true</autoCreateIndex>
- <instanceStartupTimeout>120</instanceStartupTimeout>
- <environmentVariables>
- <ES_JAVA_OPTS>-Xms4g -Xmx4g</ES_JAVA_OPTS>
- </environmentVariables>
- <instanceSettings>
- <properties>
- <xpack.ml.enabled>false</xpack.ml.enabled>
- <path.repo>${project.build.directory}/snapshots_repository</path.repo>
- <cluster.routing.allocation.disk.threshold_enabled>false</cluster.routing.allocation.disk.threshold_enabled>
- <http.cors.allow-methods>OPTIONS,HEAD,GET,POST,PUT,DELETE</http.cors.allow-methods>
- <http.cors.allow-headers>Authorization,X-Requested-With,X-Auth-Token,Content-Type,Content-Length</http.cors.allow-headers>
- </properties>
- </instanceSettings>
+ <containerNamePattern>itests-elasticsearch</containerNamePattern>
+ <images>
+ <image>
+ <name>docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}</name>
+ <alias>elasticsearch</alias>
+ <run>
+ <ports>
+ <port>${elasticsearch.port}:9200</port>
+ </ports>
+ <env>
+ <discovery.type>single-node</discovery.type>
+ <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Review Comment:
Good catch, you're absolutely right. I've removed it from `ES_JAVA_OPTS` entirely — rather than moving it to a separate `<env>` entry, I chose to drop it because Unomi creates its own indices with explicit settings so a cluster-level default isn't needed here. The same mistake was also present in the OpenSearch `OPENSEARCH_JAVA_OPTS`, so I cleaned that one up too.
##########
itests/pom.xml:
##########
@@ -247,43 +259,64 @@
</executions>
</plugin>
<plugin>
- <groupId>com.github.alexcojocaru</groupId>
- <artifactId>elasticsearch-maven-plugin</artifactId>
- <!-- REPLACE THE FOLLOWING WITH THE PLUGIN VERSION YOU NEED -->
- <version>6.29</version>
+ <groupId>io.fabric8</groupId>
+ <artifactId>docker-maven-plugin</artifactId>
<configuration>
- <!-- <downloadUrl>https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-9.2.1-darwin-aarch64.tar
- .gz</downloadUrl> -->
- <clusterName>contextElasticSearchITests</clusterName>
- <transportPort>9500</transportPort>
- <httpPort>9400</httpPort>
- <version>${elasticsearch.test.version}</version>
- <autoCreateIndex>true</autoCreateIndex>
- <instanceStartupTimeout>120</instanceStartupTimeout>
- <environmentVariables>
- <ES_JAVA_OPTS>-Xms4g -Xmx4g</ES_JAVA_OPTS>
- </environmentVariables>
- <instanceSettings>
- <properties>
- <xpack.ml.enabled>false</xpack.ml.enabled>
- <path.repo>${project.build.directory}/snapshots_repository</path.repo>
- <cluster.routing.allocation.disk.threshold_enabled>false</cluster.routing.allocation.disk.threshold_enabled>
- <http.cors.allow-methods>OPTIONS,HEAD,GET,POST,PUT,DELETE</http.cors.allow-methods>
- <http.cors.allow-headers>Authorization,X-Requested-With,X-Auth-Token,Content-Type,Content-Length</http.cors.allow-headers>
- </properties>
- </instanceSettings>
+ <containerNamePattern>itests-elasticsearch</containerNamePattern>
+ <images>
+ <image>
+ <name>docker.elastic.co/elasticsearch/elasticsearch:${elasticsearch.test.version}</name>
+ <alias>elasticsearch</alias>
+ <run>
+ <ports>
+ <port>${elasticsearch.port}:9200</port>
+ </ports>
+ <env>
+ <discovery.type>single-node</discovery.type>
+ <ES_JAVA_OPTS>-Xms8g -Xmx8g -Dcluster.default.index.settings.number_of_replicas=0</ES_JAVA_OPTS>
Review Comment:
Fixed — the heap is now controlled by a `${elasticsearch.heap}` Maven property (default: `4g`), which aligns it with the OpenSearch profile and makes it easy to override per-run with `-Delasticsearch.heap=8g` if needed. Thanks for pointing that out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests [unomi]
Posted by "sergehuber (via GitHub)" <gi...@apache.org>.
sergehuber commented on PR #759:
URL: https://github.com/apache/unomi/pull/759#issuecomment-4573212008
Following an additional local review pass, a few more things have been cleaned up in the latest commit:
- **`stop-opensearch` execution added** — the OpenSearch profile was missing a `post-integration-test` stop, so the container would keep running after a test run. Both profiles now stop their container symmetrically after tests.
- **`--keep-container` flag** — to make up for losing the "container stays around after a failure" behaviour, both profiles now support `it.keepContainer=true` (Maven) / `--keep-container` (build.sh). When set, the post-test stop is skipped so you can inspect logs or run queries manually. The pre-test cleanup still runs unconditionally on the next run.
- **`ignoreRunningContainers` on the pre-start cleanup** — added to both profiles so the `stop`/`remove` at the start of a run doesn't fail when no leftover container exists (e.g. first-ever run or after a manual cleanup).
- **`-Dcluster.default.index.settings.number_of_replicas=0` removed from OpenSearch too** — the same JVM-property mistake noted by the Copilot review was also present in `OPENSEARCH_JAVA_OPTS`; cleaned up there as well.
- **`GraphQLListIT` polling resilience** — the catch block and null-guard in the `keepTrying` supplier were removed in an earlier commit; restored them to prevent a transient HTTP error from aborting the poll loop immediately instead of retrying.
- **`docker-maven-plugin` version pin** — centralised in root `pluginManagement`; the Elasticsearch profile inherits it from there. (The OpenSearch profile still has an explicit `<version>` — leaving that as a follow-up.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests [unomi]
Posted by "sergehuber (via GitHub)" <gi...@apache.org>.
sergehuber closed pull request #759: UNOMI-921: Replace elasticsearch-maven-plugin with Docker-based Elasticsearch in integration tests
URL: https://github.com/apache/unomi/pull/759
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@unomi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org