You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@unomi.apache.org by sh...@apache.org on 2026/10/26 16:42:56 UTC
(unomi-tracker) 02/02: UNOMI-1023, UNOMI-1024: Check compatibility with 1.5.0 and Unomi 2.3 in the server tests
This is an automated email from the ASF dual-hosted git repository.
sergehuber pushed a commit to branch UNOMI-1023-UNOMI-1024-compat-fixes
in repository https://gitbox.apache.org/repos/asf/unomi-tracker.git
commit 6a86de038ae4fe3a6000e71afe596531213860e6
Author: Serge Huber <sh...@apache.org>
AuthorDate: Mon Oct 26 17:42:15 2026 +0100
UNOMI-1023, UNOMI-1024: Check compatibility with 1.5.0 and Unomi 2.3 in the server tests
- compare-1.5.0: loads the published 1.5.0 and the current tracker on
the same page and compares their requests; they must be identical,
except for the empty page "attributes" no longer sent
- unomi2-legacy: Unomi 2.3.0, the last version that accepts an empty
page "attributes"
- The test page no longer adds a Buffer shim (only for 1.5.0, whose
users always have a polyfill), and the two known-failure tests are now
normal tests
Co-Authored-By: Claude Opus 5.5 <no...@anthropic.com>
---
.gitignore | 1 +
test/servers/README.md | 35 ++++++-----
test/servers/compare-1.5.0.spec.js | 97 +++++++++++++++++++++++++++++++
test/servers/docker-compose.yml | 14 ++++-
test/servers/page/index.html | 20 ++++---
test/servers/playwright.compare.config.js | 28 +++++++++
test/servers/run.js | 31 ++++++++--
test/servers/serve.js | 4 +-
test/servers/tracker.spec.js | 27 +++++----
test/servers/unomi.js | 2 +
10 files changed, 219 insertions(+), 40 deletions(-)
diff --git a/.gitignore b/.gitignore
index cf71b0f..d11e8e9 100644
--- a/.gitignore
+++ b/.gitignore
@@ -275,3 +275,4 @@ yalc.lock
# Tracker server tests (test/servers)
test/servers/test-results/
playwright-report/
+test/servers/.cache/
diff --git a/test/servers/README.md b/test/servers/README.md
index dcf4ef5..23d3f9d 100644
--- a/test/servers/README.md
+++ b/test/servers/README.md
@@ -33,25 +33,28 @@ Requirements
Running
-------
```shell
-yarn test:servers # every server, one after the other
+yarn test:servers # the comparison with 1.5.0, then every server
yarn test:servers unomi2 unomi4 # only these servers
+yarn test:servers compare-1.5.0 # only the comparison with 1.5.0 (no Docker needed)
yarn test:servers --keep unomi4 # leave the server running afterwards
```
-The command builds the tracker, then for each server: starts it, sets it up, runs the tests and
-removes it with its data. It exits with an error if any server fails.
+The command builds the tracker and compares its requests with those of the published 1.5.0. Then,
+for each server, it starts the server, sets it up, runs the tests and removes the server with its
+data. It exits with an error if anything fails.
Servers
-------
| Name | Server | Search engine | Setup before the tests |
|---|---|---|---|
+| `unomi2-legacy` | `apache/unomi:2.3.0`, the last version that accepts an empty page `attributes` | Elasticsearch 7.17 | Scope `tracker-test` |
| `unomi2` | `apache/unomi:2.7.0` | Elasticsearch 7.17 | Scope `tracker-test` |
| `unomi3` | `apache/unomi:3.0.0` | Elasticsearch 9.1 | Scope `tracker-test` |
| `unomi4` | `apache/unomi:4.0.0-SNAPSHOT`, multi-tenant | Elasticsearch 9.4 | Tenant `tracker-test`, new public and private keys, scope `tracker-test` in the tenant |
| `unomi4-compat` | `apache/unomi:4.0.0-SNAPSHOT`, single-tenant compatibility mode | Elasticsearch 9.4 | Tenant `default`, scope `tracker-test` in it |
-Unomi listens on http://localhost:8181. Override the images with `UNOMI2_VERSION`, `UNOMI3_VERSION`
-and `UNOMI4_VERSION`, for example `UNOMI4_VERSION=4.0.1-SNAPSHOT yarn test:servers unomi4`.
+Unomi listens on http://localhost:8181. Override the images with `UNOMI2_LEGACY_VERSION`,
+`UNOMI2_VERSION`, `UNOMI3_VERSION` and `UNOMI4_VERSION`, for example `UNOMI4_VERSION=4.0.1-SNAPSHOT yarn test:servers unomi4`.
The Unomi 4 servers use the administrator password `tracker-test-root` (`UNOMI_ROOT_PASSWORD`).
With `--keep`, the setup log shows the tenant public key, so you can try the server by hand.
@@ -61,9 +64,15 @@ cookies (`UNOMI_PUBLIC_CORS_ALLOWED_ORIGINS`) and does not list http://localhost
What is tested
--------------
+Compatibility with existing sites (`compare-1.5.0`): the current tracker and the published 1.5.0 are
+loaded on the same page with the same configuration, and their requests are compared (URLs, headers
+and bodies). They must be identical, except that an empty page `attributes` is no longer sent.
+1.5.0 gets a `Buffer` polyfill, as sites using it always have one.
+
For each server:
- The context loads and a click event is stored, with the configuration a site uses for that
server (`apiKey` on Unomi 4 multi-tenant only), and in the expected tenant on Unomi 4.
+- The browser build runs without a `Buffer` polyfill.
- Setting `apiKey` works on every server, and the `X-Unomi-Api-Key` header is only sent when set.
- Unomi 4 multi-tenant answers `401` without a key.
- With cookies, the profile is kept across page loads.
@@ -71,20 +80,16 @@ For each server:
Unomi 2.x and 3.x keep the profile through the session.
- From an origin the server does not list, Unomi 4 blocks requests with cookies, and accepts them
without cookies.
-
-Two tests are marked as known failures (`test.fail()`), for bugs that are not fixed yet. Playwright
-reports them as passed while the bug is there, and fails them once it is fixed, so that the marker
-gets removed:
-- The UMD build throws `Buffer is not defined` in a browser: es6-crawler-detect calls
- `Buffer.byteLength`. The test page adds a small shim so that the other tests can run.
-- Events from a page whose `digitalData.page` has `attributes` (as the Unomi manual tutorial
- showed) are dropped by the server: the page schema does not allow that property since
- UNOMI-790 (Unomi 2.4.0).
+- A page with empty `attributes` (as the Unomi manual tutorial showed) gets its events stored.
+ Non-empty `attributes` are sent as before; no default page schema accepts them, so the server
+ drops the event, as with 1.5.0. Unomi 2.0 to 2.3 accept an empty one; 2.4 and later reject it
+ too (UNOMI-790).
Files
-----
- `docker-compose.yml`: the servers, one Compose profile each
- `run.js`: starts each server and runs the tests (`yarn test:servers`)
- `unomi.js`: server setup (tenant, keys, scope) and event queries
-- `tracker.spec.js`: the tests
+- `tracker.spec.js`: the tests against each server
+- `compare-1.5.0.spec.js`: the comparison with the published 1.5.0
- `page/index.html`, `serve.js`: the test page, configured through its query string
diff --git a/test/servers/compare-1.5.0.spec.js b/test/servers/compare-1.5.0.spec.js
new file mode 100644
index 0000000..1f49cdc
--- /dev/null
+++ b/test/servers/compare-1.5.0.spec.js
@@ -0,0 +1,97 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Compares the requests of the current tracker with those of the published 1.5.0, for the same
+// page and configuration, so that existing sites keep sending the same data. The test answers
+// the requests itself: no Unomi server is needed. Run through `yarn test:servers`.
+const { test, expect } = require('@playwright/test');
+
+const UNOMI_URL = 'http://localhost:8181';
+const ORIGIN = 'http://localhost:8000';
+
+async function captureRequests(page, tracker, config) {
+ const requests = [];
+ await page.route(UNOMI_URL + '/**', async route => {
+ const request = route.request();
+ const headers = await request.allHeaders();
+ requests.push({
+ method: request.method(),
+ url: request.url(),
+ contentType: headers['content-type'],
+ accept: headers['accept'],
+ apiKey: headers['x-unomi-api-key'],
+ body: request.postData() ? JSON.parse(request.postData()) : undefined
+ });
+ await route.fulfill({
+ status: 200,
+ headers: { 'Access-Control-Allow-Origin': ORIGIN, 'Access-Control-Allow-Credentials': 'true', 'Content-Type': 'application/json' },
+ body: JSON.stringify({ profileId: 'compare-profile', sessionId: 'compare-session' })
+ });
+ });
+ if (tracker) {
+ await page.context().addCookies([{ name: 'tracker', value: tracker, url: ORIGIN }]);
+ }
+ const params = new URLSearchParams(Object.assign({ unomi: UNOMI_URL, sessionId: 'compare-session' }, config));
+ await page.goto(ORIGIN + '/?' + params);
+ const context = await page.evaluate(() => window.trackerTest.context);
+ expect(context.loaded).toBe(true);
+ const click = await page.evaluate(() => window.trackerTest.sendClick('compared-click'));
+ expect(click.sent).toBe(true);
+ return requests;
+}
+
+// Removes an empty "attributes" from the page items, the only change the current tracker makes
+function withoutEmptyPageAttributes(value) {
+ if (Array.isArray(value)) {
+ return value.map(withoutEmptyPageAttributes);
+ }
+ if (value && typeof value === 'object') {
+ const copy = {};
+ for (const key of Object.keys(value)) {
+ copy[key] = withoutEmptyPageAttributes(value[key]);
+ }
+ if (copy.itemType === 'page' && copy.properties && copy.properties.attributes
+ && typeof copy.properties.attributes === 'object' && Object.keys(copy.properties.attributes).length === 0) {
+ delete copy.properties.attributes;
+ }
+ return copy;
+ }
+ return value;
+}
+
+const cases = [
+ { name: 'a page without attributes', config: {}, same: true },
+ { name: 'a page with non-empty attributes', config: { pageAttributes: '{"color":"blue"}' }, same: true },
+ { name: 'a page with empty attributes', config: { pageAttributes: '{}' }, same: false }
+];
+
+for (const { name, config, same } of cases) {
+ test('sends the same requests as 1.5.0 for ' + name, async ({ browser }) => {
+ const published = await browser.newPage();
+ const current = await browser.newPage();
+ const before = await captureRequests(published, '1.5.0', config);
+ const after = await captureRequests(current, undefined, config);
+ expect(before.map(request => request.url)).toEqual([UNOMI_URL + '/context.json', UNOMI_URL + '/eventcollector']);
+ expect(after).toEqual(same ? before : withoutEmptyPageAttributes(before));
+ if (!same) {
+ // The difference must be real: 1.5.0 sends the empty attributes, the current tracker does not
+ expect(after).not.toEqual(before);
+ }
+ await published.close();
+ await current.close();
+ });
+}
diff --git a/test/servers/docker-compose.yml b/test/servers/docker-compose.yml
index 1fc23b4..b50a138 100644
--- a/test/servers/docker-compose.yml
+++ b/test/servers/docker-compose.yml
@@ -36,7 +36,7 @@ services:
elasticsearch7:
<<: *elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:${ES7_VERSION:-7.17.28}
- profiles: ["unomi2"]
+ profiles: ["unomi2-legacy", "unomi2"]
elasticsearch9-1:
<<: *elasticsearch
@@ -48,6 +48,18 @@ services:
image: docker.elastic.co/elasticsearch/elasticsearch:${ES9_4_VERSION:-9.4.3}
profiles: ["unomi4", "unomi4-compat"]
+ # Unomi 2.3 is the last version whose page schema allows an empty "attributes" (UNOMI-790)
+ unomi2-legacy:
+ image: apache/unomi:${UNOMI2_LEGACY_VERSION:-2.3.0}
+ profiles: ["unomi2-legacy"]
+ environment:
+ - UNOMI_ELASTICSEARCH_ADDRESSES=elasticsearch7:9200
+ ports:
+ - "${UNOMI_PORT:-8181}:8181"
+ depends_on:
+ elasticsearch7:
+ condition: service_healthy
+
unomi2:
image: apache/unomi:${UNOMI2_VERSION:-2.7.0}
profiles: ["unomi2"]
diff --git a/test/servers/page/index.html b/test/servers/page/index.html
index 69f07c3..04f35fe 100644
--- a/test/servers/page/index.html
+++ b/test/servers/page/index.html
@@ -20,17 +20,20 @@
<meta charset="utf-8">
<title>Unomi tracker test page</title>
<script>
- // The crawler detection library calls Buffer.byteLength, which browsers do not have.
- // ?noBufferShim loads the tracker without this shim, to check the bug is still there.
- if (!new URLSearchParams(document.location.search).has('noBufferShim') && typeof window.Buffer === 'undefined') {
+ // A "tracker=1.5.0" cookie loads the published 1.5.0 tracker instead of the current build (a cookie,
+ // because the tracker sends the URL parameters). Sites using 1.5.0 always have a Buffer polyfill
+ // (it fails without one), so add one for it only.
+ if (document.cookie.split('; ').indexOf('tracker=1.5.0') !== -1) {
window.Buffer = {
byteLength: function (value) {
return new TextEncoder().encode(value).length;
}
};
+ document.write('<script src="/tracker-1.5.0.js"><\/script>');
+ } else {
+ document.write('<script src="/tracker.js"><\/script>');
}
</script>
- <script src="/tracker.js"></script>
</head>
<body>
<script>
@@ -39,7 +42,8 @@
// apiKey wemInitConfig.apiKey (optional)
// sendCredentials wemInitConfig.sendCredentials, "true" or "false" (optional)
// noBufferShim load the tracker without the Buffer shim (optional)
- // pageAttributes add "attributes": {} to digitalData.page, as the Unomi manual does (optional)
+ // pageAttributes digitalData.page.attributes as JSON, for example {} (optional)
+ // sessionId session ID to use instead of a new one (optional)
(function () {
const params = new URLSearchParams(document.location.search);
const wemInitConfig = {
@@ -76,11 +80,13 @@
};
if (params.has('pageAttributes')) {
- digitalData.page.attributes = {};
+ digitalData.page.attributes = JSON.parse(params.get('pageAttributes'));
}
const tracker = window['apache-unomi-tracker'].useTracker();
- if (tracker.getCookie(wemInitConfig.trackerSessionIdCookieName) == null) {
+ if (params.has('sessionId')) {
+ tracker.setCookie(wemInitConfig.trackerSessionIdCookieName, params.get('sessionId'), 1);
+ } else if (tracker.getCookie(wemInitConfig.trackerSessionIdCookieName) == null) {
tracker.setCookie(wemInitConfig.trackerSessionIdCookieName, tracker.generateGuid(), 1);
}
diff --git a/test/servers/playwright.compare.config.js b/test/servers/playwright.compare.config.js
new file mode 100644
index 0000000..f29a32f
--- /dev/null
+++ b/test/servers/playwright.compare.config.js
@@ -0,0 +1,28 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+// Comparison with the published 1.5.0 tracker (compare-1.5.0.spec.js). No Unomi server needed.
+// Run through `yarn test:servers`, which downloads 1.5.0 first.
+const path = require('path');
+const { defineConfig } = require('@playwright/test');
+const base = require('./playwright.config');
+
+module.exports = defineConfig(Object.assign({}, base, {
+ testMatch: 'compare-1.5.0.spec.js',
+ outputDir: path.join(__dirname, 'test-results', 'compare-1.5.0'),
+ globalSetup: undefined
+}));
diff --git a/test/servers/run.js b/test/servers/run.js
index 9979cdd..8d41235 100644
--- a/test/servers/run.js
+++ b/test/servers/run.js
@@ -17,12 +17,14 @@
// Runs the browser tests against each Unomi server version, one after the other.
//
-// yarn test:servers all servers
+// yarn test:servers the comparison with 1.5.0, then all servers
// yarn test:servers unomi2 unomi4 only these servers
+// yarn test:servers compare-1.5.0 only the comparison with 1.5.0 (no Docker needed)
// yarn test:servers --keep unomi4 leave the server running afterwards
//
// Not part of CI: it needs Docker and takes several minutes.
const { spawnSync } = require('child_process');
+const fs = require('fs');
const path = require('path');
const { TARGETS } = require('./unomi');
@@ -30,12 +32,13 @@ const root = path.join(__dirname, '..', '..');
const composeFile = path.join(__dirname, 'docker-compose.yml');
const args = process.argv.slice(2);
const keep = args.includes('--keep');
+const COMPARE = 'compare-1.5.0';
const names = args.filter(arg => !arg.startsWith('--'));
-const selected = names.length > 0 ? names : Object.keys(TARGETS);
+const selected = names.length > 0 ? names : [COMPARE].concat(Object.keys(TARGETS));
for (const name of selected) {
- if (!TARGETS[name]) {
- console.error('Unknown server "' + name + '", expected one of: ' + Object.keys(TARGETS).join(', '));
+ if (name !== COMPARE && !TARGETS[name]) {
+ console.error('Unknown name "' + name + '", expected one of: ' + [COMPARE].concat(Object.keys(TARGETS)).join(', '));
process.exit(2);
}
}
@@ -54,8 +57,26 @@ if (!run('yarn', ['build'])) {
process.exit(1);
}
+// The published 1.5.0 tracker, for the comparison
+function downloadPublishedTracker() {
+ const directory = path.join(__dirname, '.cache', 'apache-unomi-tracker-1.5.0');
+ if (fs.existsSync(path.join(directory, 'package', 'dist', 'apache-unomi-tracker.umd.js'))) {
+ return true;
+ }
+ fs.mkdirSync(directory, { recursive: true });
+ return run('npm', ['pack', 'apache-unomi-tracker@1.5.0', '--pack-destination', directory])
+ && run('tar', ['xzf', path.join(directory, 'apache-unomi-tracker-1.5.0.tgz'), '-C', directory]);
+}
+
const results = [];
-for (const name of selected) {
+if (selected.includes(COMPARE)) {
+ console.log('\n=== ' + COMPARE);
+ const passed = downloadPublishedTracker()
+ && run('npx', ['playwright', 'test', '-c', path.join(__dirname, 'playwright.compare.config.js')]);
+ results.push({ name: COMPARE, passed });
+}
+
+for (const name of selected.filter(name => name !== COMPARE)) {
console.log('\n=== ' + name);
let passed = false;
try {
diff --git a/test/servers/serve.js b/test/servers/serve.js
index c97f476..95f8591 100644
--- a/test/servers/serve.js
+++ b/test/servers/serve.js
@@ -25,7 +25,9 @@ const path = require('path');
const PORTS = [8000, 8001];
const files = {
'/': { file: path.join(__dirname, 'page', 'index.html'), type: 'text/html; charset=utf-8' },
- '/tracker.js': { file: path.join(__dirname, '..', '..', 'dist', 'apache-unomi-tracker.umd.js'), type: 'text/javascript; charset=utf-8' }
+ '/tracker.js': { file: path.join(__dirname, '..', '..', 'dist', 'apache-unomi-tracker.umd.js'), type: 'text/javascript; charset=utf-8' },
+ // Downloaded by run.js, for the comparison with the published 1.5.0
+ '/tracker-1.5.0.js': { file: path.join(__dirname, '.cache', 'apache-unomi-tracker-1.5.0', 'package', 'dist', 'apache-unomi-tracker.umd.js'), type: 'text/javascript; charset=utf-8' }
};
function handle(request, response) {
diff --git a/test/servers/tracker.spec.js b/test/servers/tracker.spec.js
index a57a55d..d340cd4 100644
--- a/test/servers/tracker.spec.js
+++ b/test/servers/tracker.spec.js
@@ -61,23 +61,28 @@ async function expectStoredClick(click) {
}
test.describe(server.name, () => {
- test('the browser build runs without Node polyfills', async ({ page }) => {
- // Known bug: es6-crawler-detect calls Buffer.byteLength, so the UMD build throws in a browser.
- // Remove test.fail() once the build works without the shim.
- test.fail();
+ test('the browser build runs without a Buffer polyfill', async ({ page }) => {
const errors = [];
page.on('pageerror', error => errors.push(error.message));
- const { context } = await openPage(page, LISTED_ORIGIN, Object.assign({ noBufferShim: 'true' }, siteConfig), 10000);
+ const { context } = await openPage(page, LISTED_ORIGIN, siteConfig);
expect(errors).toEqual([]);
expect(context.loaded).toBe(true);
});
- test('collects events from a page with "attributes", as the Unomi manual shows', async ({ page }) => {
- // Known bug: the tracker sends digitalData.page as the page properties, and the page schema
- // does not allow "attributes", so Unomi drops the events. Remove test.fail() once fixed.
- test.fail();
- await openPage(page, LISTED_ORIGIN, Object.assign({ pageAttributes: 'true' }, siteConfig));
- await expectStoredClick(await sendClick(page, 'page-attributes'));
+ test('collects events from a page with empty "attributes", as the Unomi manual showed', async ({ page }) => {
+ const { context } = await openPage(page, LISTED_ORIGIN, Object.assign({ pageAttributes: '{}' }, siteConfig));
+ expect(context.loaded).toBe(true);
+ await expectStoredClick(await sendClick(page, 'empty-page-attributes'));
+ });
+
+ test('sends non-empty page "attributes" as before', async ({ page }) => {
+ // No default page schema accepts attributes with content, so the server drops the event, as with
+ // 1.5.0 (compare-1.5.0 checks that the request is the same). Unomi 2.0 to 2.3 accept an empty one.
+ await openPage(page, LISTED_ORIGIN, Object.assign({ pageAttributes: '{"color":"blue"}' }, siteConfig));
+ const click = await sendClick(page, 'page-attributes');
+ expect(click.sent).toBe(true);
+ const events = await findClickEvents(process.env.UNOMI_READ_CREDENTIALS, click.targetId, 5000);
+ expect(events).toHaveLength(0);
});
test('loads the context and collects an event with the site configuration', async ({ page }) => {
diff --git a/test/servers/unomi.js b/test/servers/unomi.js
index 7b63654..6a9d288 100644
--- a/test/servers/unomi.js
+++ b/test/servers/unomi.js
@@ -23,7 +23,9 @@ const SCOPE = 'tracker-test';
// Unomi 2.x and 3.x have no tenants and accept cookies from any origin.
// Unomi 4 only accepts cookies from listed origins (here http://localhost:8000, not http://localhost:8001).
+// Unomi 2.0 to 2.3 accept an empty page "attributes"; 2.4 removed it from the page schema (UNOMI-790).
const TARGETS = {
+ 'unomi2-legacy': { major: 2, multiTenant: false, admin: 'karaf:karaf' },
'unomi2': { major: 2, multiTenant: false, admin: 'karaf:karaf' },
'unomi3': { major: 3, multiTenant: false, admin: 'karaf:karaf' },
'unomi4': { major: 4, multiTenant: true, admin: 'karaf:' + ROOT_PASSWORD },