You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@unomi.apache.org by sh...@apache.org on 2026/09/27 08:16:11 UTC
(unomi) branch UNOMI-973-tenant-file-endpoints created (now ecc0385b8)
This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a change to branch UNOMI-973-tenant-file-endpoints
in repository https://gitbox.apache.org/repos/asf/unomi.git
at ecc0385b8 UNOMI-973: Confine each tenant's import and export files to its own directory
This branch includes the following new commits:
new ecc0385b8 UNOMI-973: Confine each tenant's import and export files to its own directory
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
(unomi) 01/01: UNOMI-973: Confine each tenant's import and export files to its own directory
Posted by sh...@apache.org.
This is an automated email from the ASF dual-hosted git repository.
asf-gitbox-commits pushed a commit to branch UNOMI-973-tenant-file-endpoints
in repository https://gitbox.apache.org/repos/asf/unomi.git
commit ecc0385b800f1e83a449ec5b86f26865edd8e5e1
Author: Serge Huber <sh...@apache.org>
AuthorDate: Thu Sep 3 09:53:17 2026 +0200
UNOMI-973: Confine each tenant's import and export files to its own directory
Up-port of #849. A recurrent file endpoint must resolve under the deployment's base directory and under that tenant's subdirectory, so one tenant cannot read or write another's files. A missing tenant is refused rather than judged against the shared parent.
Co-authored-by: Jerome Blanchard <jb...@jahia.com>
Co-authored-by: Jonathan SINOVASSIN-NAIK <58...@users.noreply.github.com>
---
.../apache/unomi/router/api/EndpointValidator.java | 535 +++++++++++++++
.../apache/unomi/router/api/RouterConstants.java | 11 +
extensions/router/router-core/pom.xml | 7 +
.../router/core/context/RouterCamelContext.java | 20 +
.../route/ProfileExportCollectRouteBuilder.java | 26 +-
.../route/ProfileImportFromSourceRouteBuilder.java | 37 +-
.../core/route/RouterAbstractRouteBuilder.java | 85 ++-
.../resources/OSGI-INF/blueprint/blueprint.xml | 4 +
.../src/main/resources/org.apache.unomi.router.cfg | 8 +
.../core/route/FileEndpointContainmentTest.java | 760 +++++++++++++++++++++
.../core/route/RefusedConfigurationStatusTest.java | 344 ++++++++++
extensions/router/router-rest/pom.xml | 11 +
.../rest/AbstractConfigurationServiceEndpoint.java | 67 ++
.../rest/ExportConfigurationServiceEndPoint.java | 20 +-
.../rest/ImportConfigurationServiceEndPoint.java | 14 +-
.../rest/ConfigurationEndpointValidationTest.java | 385 +++++++++++
.../test/java/org/apache/unomi/itests/AllITs.java | 1 +
.../test/java/org/apache/unomi/itests/BaseIT.java | 18 +-
.../org/apache/unomi/itests/ProfileExportIT.java | 4 +-
.../apache/unomi/itests/ProfileImportActorsIT.java | 2 +-
.../itests/ProfileImportExportContainmentIT.java | 260 +++++++
.../unomi/itests/ProfileImportRankingIT.java | 2 +-
.../unomi/itests/ProfileImportSurfersIT.java | 2 +-
.../src/test/resources/org.apache.unomi.router.cfg | 8 +
.../src/main/asciidoc/profile-import-export.adoc | 18 +-
.../main/resources/etc/custom.system.properties | 8 +
26 files changed, 2610 insertions(+), 47 deletions(-)
diff --git a/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/EndpointValidator.java b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/EndpointValidator.java
new file mode 100644
index 000000000..04ef984e7
--- /dev/null
+++ b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/EndpointValidator.java
@@ -0,0 +1,535 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.router.api;
+
+import java.io.ByteArrayOutputStream;
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.InvalidPathException;
+import java.nio.file.LinkOption;
+import java.nio.file.Path;
+import java.nio.file.Paths;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import java.util.regex.Pattern;
+
+/**
+ * Decides whether the endpoint URI carried by an import or export configuration may be used.
+ *
+ * <p>Two rules apply. The scheme must belong to the configured allow-list. And a {@code file}
+ * endpoint must resolve inside one of the base directories the deployment permits — the directory
+ * the URI names, and every path-bearing option it carries, since validating only the directory would
+ * leave {@code file:///permitted/?fileName=../../elsewhere} open.
+ *
+ * <p>Containment is recursive: any depth under a permitted base directory is accepted, whether or
+ * not the directory exists yet. It is decided on canonical paths — percent-encoding decoded, parent
+ * segments resolved, symbolic links followed — and compared component by component, so a sibling
+ * that merely shares a textual prefix with a permitted directory is not mistaken for one of its
+ * children.
+ *
+ * <p>What is validated is what Camel will use, so the URI is read the way Camel reads it: option
+ * names are percent-decoded before they are matched, a {@code RAW()} value keeps its ampersands
+ * instead of being cut in two, and the File Language expressions a path-bearing option may carry are
+ * accounted for rather than taken literally.
+ *
+ * <p>A path the file system cannot make sense of, and a path whose existing part cannot be resolved,
+ * are both refusals: nothing is thrown out of this class, because one malformed endpoint must not
+ * cost a deployment the routes of every other configuration.
+ *
+ * <p>A scheme other than {@code file} addresses a remote server, so its directory and its
+ * path-bearing options are remote and none of this applies to them — with one exception.
+ * {@code localWorkDirectory} names a <em>local</em> directory, which the remote components stage
+ * their downloads in, so it is confined whatever the scheme.
+ *
+ * <p>On a multi-tenant deployment those base directories are shared, so {@link #validateForTenant}
+ * confines a file endpoint one directory further, to {@code {baseDir}/{tenantId}}. A configuration
+ * with no tenant, or a tenant id that is not a single path segment, is refused rather than judged
+ * against the shared parent.
+ */
+public final class EndpointValidator {
+
+ public static final String FILE_SCHEME = "file";
+
+ /**
+ * The Camel file endpoint options whose value is, or contains, a path. Compared in lower case.
+ *
+ * <p>Selection options ({@code include}, {@code exclude}, {@code antInclude}, {@code antExclude})
+ * are deliberately absent: they are patterns matched against the files the endpoint directory
+ * already offers, not paths Camel resolves, so they cannot direct a read or a write anywhere else.
+ * Holding them to containment would only refuse legitimate patterns — a regular expression is not
+ * a valid path on every file system.
+ */
+ private static final Set<String> PATH_BEARING_OPTIONS = Collections.unmodifiableSet(new HashSet<>(Arrays.asList(
+ "filename", "tempfilename", "tempprefix", "move", "movefailed", "moveexisting", "premove",
+ "donefilename")));
+
+ /**
+ * Camel evaluates a path-bearing option as a File Language expression, so the value that reaches
+ * the file system is not the one configured. These tokens expand to a name, or to a name and the
+ * subdirectories the file already sits in: whatever they hold, they cannot hold a parent segment,
+ * so containment can be decided with them replaced by a placeholder.
+ */
+ private static final Set<String> NAME_TOKENS = Collections.unmodifiableSet(new HashSet<>(Arrays.asList(
+ "file:name", "file:name.noext", "file:name.ext", "file:onlyname", "file:onlyname.noext",
+ "file:ext", "file:size", "file:modified", "exchangeid")));
+
+ /** Expands to a date, which cannot hold a parent segment either. A prefix, not a whole token. */
+ private static final String DATE_TOKEN_PREFIX = "date:";
+
+ /** Expands to the directory the file sits in, which is the endpoint directory or one below it. */
+ private static final String PARENT_TOKEN = "file:parent";
+
+ /**
+ * The one option that names a local directory whatever the scheme: the remote components write the
+ * content they download into files there, under the name the remote server announces.
+ *
+ * <p>It is kept out of {@link #PATH_BEARING_OPTIONS} because it is not resolved the same way. Those
+ * options are resolved against the directory the endpoint names; this one Camel resolves on its
+ * own — {@code new File(FileUtil.normalizePath(value))} — so it is held to the permitted
+ * directories directly, and it carries no File Language expression to account for.
+ */
+ private static final String LOCAL_WORK_DIRECTORY_OPTION = "localworkdirectory";
+
+ /** Stands in for a token that expands to a name: one path component, never a parent segment. */
+ private static final String NAME_PLACEHOLDER = "_";
+
+ /**
+ * A tenant id used as a directory name. The same shape the one-shot import accepts, so a tenant
+ * cannot be {@code ..} or a path of its own and leave the directory this class confines it to.
+ */
+ private static final Pattern TENANT_ID = Pattern.compile("[A-Za-z0-9_-]+");
+
+ /** The path segments the walk interprets rather than resolves against the file system. */
+ private static final String CURRENT_DIRECTORY = ".";
+ private static final String PARENT_DIRECTORY = "..";
+
+ private EndpointValidator() {
+ }
+
+ /**
+ * Validates the endpoint URI of an import or export configuration.
+ *
+ * @param endpointUri the endpoint URI, as configured
+ * @param allowedSchemes the comma-separated list of allowed schemes
+ * @param permittedBaseDirs the comma-separated list of base directories a {@code file} endpoint may
+ * resolve into
+ * @return {@code null} when the endpoint may be used, otherwise the reason it is refused
+ */
+ public static String validate(String endpointUri, String allowedSchemes, String permittedBaseDirs) {
+ if (isBlank(endpointUri)) {
+ return "no endpoint is configured";
+ }
+
+ int schemeSeparator = endpointUri.indexOf(':');
+ if (schemeSeparator <= 0) {
+ return "endpoint '" + endpointUri + "' has no scheme";
+ }
+
+ String scheme = endpointUri.substring(0, schemeSeparator);
+ if (!containsIgnoreCase(split(allowedSchemes), scheme)) {
+ return "endpoint scheme '" + scheme + "' is not allowed";
+ }
+
+ try {
+ String refusal = validateLocalWorkDirectory(endpointUri, permittedBaseDirs);
+ if (refusal != null || !FILE_SCHEME.equalsIgnoreCase(scheme)) {
+ return refusal;
+ }
+ return validateContainment(endpointUri, permittedBaseDirs);
+ } catch (Refusal refusal) {
+ return refusal.getMessage();
+ } catch (InvalidPathException e) {
+ return "endpoint '" + endpointUri + "' does not denote a path this file system can use: "
+ + e.getMessage();
+ }
+ }
+
+ /**
+ * Validates an endpoint the way {@link #validate(String, String, String)} does, after confining a
+ * {@code file} path to {@code {baseDir}/{tenantId}} for every configured base directory.
+ *
+ * @param endpointUri the endpoint URI, as configured
+ * @param allowedSchemes the comma-separated list of allowed schemes
+ * @param permittedBaseDirs the comma-separated list of base directories the deployment permits
+ * @param tenantId the tenant the configuration belongs to
+ * @return {@code null} when the endpoint may be used, otherwise the reason it is refused
+ */
+ public static String validateForTenant(String endpointUri, String allowedSchemes, String permittedBaseDirs, String tenantId) {
+ if (tenantId == null || !TENANT_ID.matcher(tenantId).matches()) {
+ return "no tenant is set for this configuration";
+ }
+ return validate(endpointUri, allowedSchemes, scopeToTenant(permittedBaseDirs, tenantId));
+ }
+
+ private static String scopeToTenant(String permittedBaseDirs, String tenantId) {
+ List<String> scoped = new ArrayList<>();
+ for (String baseDir : split(permittedBaseDirs)) {
+ scoped.add(Paths.get(baseDir).resolve(tenantId).toString());
+ }
+ return String.join(",", scoped);
+ }
+
+ /**
+ * Confines {@code localWorkDirectory}, the local directory a remote endpoint may be told to stage
+ * its downloads in. Left alone, it is a write outside the permitted directories that the scheme
+ * allow-list never sees: {@code ftp://host/x?localWorkDirectory=/opt/unomi/deploy} stages what the
+ * remote server sends, under the name the remote server chooses.
+ */
+ private static String validateLocalWorkDirectory(String endpointUri, String permittedBaseDirs) throws Refusal {
+ int querySeparator = endpointUri.indexOf('?');
+ if (querySeparator < 0) {
+ return null;
+ }
+ for (String[] parameter : parseQuery(endpointUri.substring(querySeparator + 1))) {
+ if (!LOCAL_WORK_DIRECTORY_OPTION.equals(decode(parameter[0]).toLowerCase(Locale.ROOT))) {
+ continue;
+ }
+ String value = stripRaw(decode(parameter[1]));
+ if (value.isEmpty()) {
+ continue;
+ }
+ if (!isContained(Paths.get(value), baseDirectories(permittedBaseDirs))) {
+ return "option '" + parameter[0] + "' points outside the permitted directories";
+ }
+ }
+ return null;
+ }
+
+ private static List<Path> baseDirectories(String permittedBaseDirs) throws Refusal {
+ List<Path> baseDirs = new ArrayList<>();
+ for (String baseDir : split(permittedBaseDirs)) {
+ baseDirs.add(canonicalize(Paths.get(baseDir)));
+ }
+ if (baseDirs.isEmpty()) {
+ throw new Refusal("no permitted base directory is configured for file endpoints");
+ }
+ return baseDirs;
+ }
+
+ private static String validateContainment(String endpointUri, String permittedBaseDirs) throws Refusal {
+ List<Path> baseDirs = baseDirectories(permittedBaseDirs);
+
+ int querySeparator = endpointUri.indexOf('?');
+ String head = querySeparator < 0 ? endpointUri : endpointUri.substring(0, querySeparator);
+ String query = querySeparator < 0 ? "" : endpointUri.substring(querySeparator + 1);
+
+ Path directory = Paths.get(decode(stripScheme(head)));
+ if (!isContained(directory, baseDirs)) {
+ return "directory '" + directory + "' is outside the permitted directories";
+ }
+
+ for (String[] parameter : parseQuery(query)) {
+ String option = parameter[0];
+ if (!PATH_BEARING_OPTIONS.contains(decode(option).toLowerCase(Locale.ROOT))) {
+ continue;
+ }
+ String value = stripRaw(decode(parameter[1]));
+ if (value.isEmpty()) {
+ continue;
+ }
+ if (!isContained(directory.resolve(withoutExpressions(option, value)), baseDirs)) {
+ return "option '" + option + "' points outside the permitted directories";
+ }
+ }
+
+ return null;
+ }
+
+ /**
+ * Removes the scheme, and the authority separator Camel tolerates in any of its forms
+ * ({@code file:dir}, {@code file://dir}, {@code file:///dir}).
+ */
+ private static String stripScheme(String head) {
+ String path = head.substring(head.indexOf(':') + 1);
+ return path.startsWith("//") ? path.substring(2) : path;
+ }
+
+ private static boolean isContained(Path path, List<Path> baseDirs) throws Refusal {
+ Path candidate = canonicalize(path);
+ for (Path baseDir : baseDirs) {
+ if (candidate.startsWith(baseDir)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ /**
+ * Resolves a path to the one the file system would actually use, walking it one component at a
+ * time from the root the way the file system does: a symbolic link is expanded where it stands,
+ * and a parent segment is applied to what the walk has resolved so far.
+ *
+ * <p>The order is what makes this correct. Collapsing parent segments first — {@code normalize()}
+ * on the whole path — erases the component they cancel, symbolic link included, so
+ * {@code /base/link/..} reads as {@code /base} while the file system resolves it to the parent of
+ * the link's target. Only a walk sees the link before the segment that cancels it.
+ *
+ * <p>A path that does not exist yet is resolved as far as it exists and kept as it stands from
+ * there — an export destination is created on first write, and must be decided on before it
+ * exists.
+ *
+ * <p>A path whose existing part cannot be resolved is refused rather than accepted as it stands: a
+ * dangling symbolic link inside a permitted directory would otherwise be taken for a child of it,
+ * and would leave it as soon as its target is created.
+ */
+ private static Path canonicalize(Path path) throws Refusal {
+ Path absolute = path.toAbsolutePath();
+ Path resolved = absolute.getRoot();
+ if (resolved == null) {
+ // an absolute path always has a root; without one there is nothing to decide on
+ throw new Refusal("path '" + path + "' cannot be made absolute");
+ }
+ // once a component is missing, nothing below it can exist: the rest is kept as written
+ boolean belowWhatExists = false;
+ for (Path component : absolute) {
+ String name = component.toString();
+ if (CURRENT_DIRECTORY.equals(name)) {
+ continue;
+ }
+ if (PARENT_DIRECTORY.equals(name)) {
+ Path parent = resolved.getParent();
+ if (parent != null) {
+ resolved = parent;
+ }
+ continue;
+ }
+ Path candidate = resolved.resolve(component);
+ if (belowWhatExists || !Files.exists(candidate, LinkOption.NOFOLLOW_LINKS)) {
+ belowWhatExists = true;
+ resolved = candidate;
+ } else if (Files.isSymbolicLink(candidate)) {
+ try {
+ resolved = candidate.toRealPath();
+ } catch (IOException e) {
+ throw new Refusal("path '" + candidate + "' cannot be resolved on the file system: " + e);
+ }
+ } else {
+ resolved = candidate;
+ }
+ }
+ return resolved;
+ }
+
+ /**
+ * Replaces the File Language expressions of a path-bearing option by what they can contribute to a
+ * path, so that containment is decided on the value Camel will resolve instead of on the
+ * placeholder as it is written: {@code move=${file:parent}/../elsewhere} normalizes to a child of
+ * the endpoint directory while Camel sends it to a sibling.
+ *
+ * <p>{@code ${file:parent}} becomes the endpoint directory itself, which is the shallowest
+ * directory it can expand to, so parent segments are measured against the worst case. A token that
+ * expands to a name becomes a placeholder. Every other expression is refused: a header, a property
+ * or the body can hold any path at all, and there is nothing left to validate.
+ */
+ private static String withoutExpressions(String option, String value) throws Refusal {
+ if (value.indexOf('$') < 0) {
+ return value;
+ }
+ StringBuilder substituted = new StringBuilder(value.length());
+ int position = 0;
+ while (position < value.length()) {
+ int start = expressionStart(value, position);
+ if (start < 0) {
+ substituted.append(value, position, value.length());
+ break;
+ }
+ substituted.append(value, position, start);
+ int open = value.indexOf('{', start);
+ int close = value.indexOf('}', open);
+ if (close < 0) {
+ throw new Refusal("option '" + option + "' carries an expression that is never closed");
+ }
+ substituted.append(contributionOf(option, value.substring(open + 1, close)));
+ position = close + 1;
+ }
+ return substituted.toString();
+ }
+
+ /**
+ * The start of the next expression, in either of the forms Camel accepts ({@code ${...}} and
+ * {@code $simple{...}}), or {@code -1}. A dollar sign that starts neither is a plain character.
+ */
+ private static int expressionStart(String value, int from) {
+ for (int i = value.indexOf('$', from); i >= 0; i = value.indexOf('$', i + 1)) {
+ if (value.startsWith("${", i) || value.startsWith("$simple{", i)) {
+ return i;
+ }
+ }
+ return -1;
+ }
+
+ private static String contributionOf(String option, String expression) throws Refusal {
+ String token = expression.trim().toLowerCase(Locale.ROOT);
+ if (PARENT_TOKEN.equals(token)) {
+ return ".";
+ }
+ if (NAME_TOKENS.contains(token) || token.startsWith(DATE_TOKEN_PREFIX)) {
+ return NAME_PLACEHOLDER;
+ }
+ throw new Refusal("option '" + option + "' uses the expression '${" + expression
+ + "}', whose value cannot be held inside the permitted directories");
+ }
+
+ /**
+ * {@code RAW(...)} and {@code RAW{...}} tell Camel not to decode a value; the path it wraps is used
+ * as it stands.
+ */
+ private static String stripRaw(String value) {
+ if (value.startsWith("RAW(") && value.endsWith(")")) {
+ return value.substring(4, value.length() - 1);
+ }
+ if (value.startsWith("RAW{") && value.endsWith("}")) {
+ return value.substring(4, value.length() - 1);
+ }
+ return value;
+ }
+
+ /**
+ * Splits the query into its parameters the way Camel does. A {@code RAW()} value ends at the marker
+ * that closes it, not at the first ampersand, so an ampersand it contains does not start a new
+ * parameter: splitting on every ampersand would leave the rest of that value unvalidated, which is
+ * enough to carry {@code fileName=RAW(profiles.csv&../../elsewhere)} through.
+ */
+ private static List<String[]> parseQuery(String query) {
+ List<String[]> parameters = new ArrayList<>();
+ int position = 0;
+ while (position < query.length()) {
+ int end = endOfParameter(query, position);
+ String parameter = query.substring(position, end);
+ int separator = parameter.indexOf('=');
+ if (separator > 0) {
+ parameters.add(new String[]{parameter.substring(0, separator), parameter.substring(separator + 1)});
+ }
+ position = end + 1;
+ }
+ return parameters;
+ }
+
+ /** Where the parameter that starts at {@code from} ends: exclusive, on its closing ampersand. */
+ private static int endOfParameter(String query, int from) {
+ int ampersand = query.indexOf('&', from);
+ int valueStart = query.indexOf('=', from);
+ char closing = 0;
+ if (valueStart >= 0 && (ampersand < 0 || valueStart < ampersand)) {
+ closing = rawClosingMarker(query.substring(valueStart + 1));
+ }
+ if (closing == 0) {
+ return ampersand < 0 ? query.length() : ampersand;
+ }
+ for (int i = valueStart + 1; i < query.length(); i++) {
+ if (query.charAt(i) == closing && (i + 1 == query.length() || query.charAt(i + 1) == '&')) {
+ return i + 1;
+ }
+ }
+ return query.length();
+ }
+
+ private static char rawClosingMarker(String value) {
+ if (value.startsWith("RAW(")) {
+ return ')';
+ }
+ if (value.startsWith("RAW{")) {
+ return '}';
+ }
+ return 0;
+ }
+
+ /**
+ * Decodes the percent-encoding of a URI, so that containment is decided on the path the file system
+ * will see. Unlike form decoding, {@code +} is left alone: it is a valid character in a file name.
+ * Characters that are not escaped keep their own encoding, so a path that mixes an escape with a
+ * non-ASCII name is not corrupted into a different path.
+ */
+ private static String decode(String value) {
+ if (value.indexOf('%') < 0) {
+ return value;
+ }
+ ByteArrayOutputStream decoded = new ByteArrayOutputStream(value.length());
+ StringBuilder verbatim = new StringBuilder();
+ for (int i = 0; i < value.length(); i++) {
+ char character = value.charAt(i);
+ if (character == '%' && i + 2 < value.length()) {
+ int high = Character.digit(value.charAt(i + 1), 16);
+ int low = Character.digit(value.charAt(i + 2), 16);
+ if (high >= 0 && low >= 0) {
+ writeUtf8(verbatim, decoded);
+ decoded.write((high << 4) + low);
+ i += 2;
+ continue;
+ }
+ }
+ verbatim.append(character);
+ }
+ writeUtf8(verbatim, decoded);
+ return new String(decoded.toByteArray(), StandardCharsets.UTF_8);
+ }
+
+ private static void writeUtf8(StringBuilder verbatim, ByteArrayOutputStream decoded) {
+ if (verbatim.length() == 0) {
+ return;
+ }
+ byte[] bytes = verbatim.toString().getBytes(StandardCharsets.UTF_8);
+ decoded.write(bytes, 0, bytes.length);
+ verbatim.setLength(0);
+ }
+
+ private static List<String> split(String commaSeparated) {
+ List<String> values = new ArrayList<>();
+ if (commaSeparated == null) {
+ return values;
+ }
+ for (String value : commaSeparated.split(",")) {
+ String trimmed = value.trim();
+ if (!trimmed.isEmpty()) {
+ values.add(trimmed);
+ }
+ }
+ return values;
+ }
+
+ private static boolean containsIgnoreCase(List<String> values, String searched) {
+ for (String value : values) {
+ if (value.equalsIgnoreCase(searched)) {
+ return true;
+ }
+ }
+ return false;
+ }
+
+ private static boolean isBlank(String value) {
+ return value == null || value.trim().isEmpty();
+ }
+
+ /**
+ * A reason an endpoint cannot be validated, carried back to {@link #validate} to be answered as a
+ * refusal. Nothing is thrown out of this class.
+ */
+ private static final class Refusal extends Exception {
+
+ private static final long serialVersionUID = 1L;
+
+ Refusal(String reason) {
+ super(reason);
+ }
+ }
+}
diff --git a/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/RouterConstants.java b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/RouterConstants.java
index 373b8e41f..5e3f9fb69 100644
--- a/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/RouterConstants.java
+++ b/extensions/router/router-api/src/main/java/org/apache/unomi/router/api/RouterConstants.java
@@ -38,6 +38,13 @@ public interface RouterConstants {
String CONFIG_STATUS_COMPLETE_SUCCESS = "SUCCESS";
String CONFIG_STATUS_COMPLETE_WITH_ERRORS = "WITH_ERRORS";
String CONFIG_STATUS_ROUTE_CREATION_FAILED = "ROUTE_CREATION_FAILED";
+ /**
+ * The configuration names an endpoint that cannot be honoured, so no route carries it. Kept apart
+ * from the execution statuses above: those report on a run that happened, this one says no run can.
+ * It is set and cleared by the route builders alone, so that restoring the deployment's permitted
+ * directories brings the configuration back on its own.
+ */
+ String CONFIG_STATUS_INVALID_ENDPOINT = "INVALID_ENDPOINT";
String IMPORT_EXPORT_CONFIG_TYPE_RECURRENT = "recurrent";
String IMPORT_EXPORT_CONFIG_TYPE_ONESHOT = "oneshot";
@@ -58,6 +65,10 @@ public interface RouterConstants {
String IMPORT_ONESHOT_ROUTE_ID = "ONE_SHOT_ROUTE";
String IMPORT_ONESHOT_UPLOAD_DIR = "oneshotImportUploadDir";
+ String CONFIG_ALLOWED_ENDPOINTS = "routerAllowedEndpoints";
+ String CONFIG_IMPORT_BASE_DIRS = "routerImportBaseDirs";
+ String CONFIG_EXPORT_BASE_DIRS = "routerExportBaseDirs";
+
String DEFAULT_FILE_COLUMN_SEPARATOR = ",";
String DEFAULT_FILE_LINE_SEPARATOR = "\n";
String KEY_HISTORY_SIZE = "historySize";
diff --git a/extensions/router/router-core/pom.xml b/extensions/router/router-core/pom.xml
index 54f03113f..a5750c7e1 100644
--- a/extensions/router/router-core/pom.xml
+++ b/extensions/router/router-core/pom.xml
@@ -154,6 +154,13 @@
<artifactId>slf4j-api</artifactId>
<scope>provided</scope>
</dependency>
+
+ <!-- tests -->
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
</dependencies>
<build>
diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
index 711b88915..fcfebb6d8 100644
--- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
+++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/context/RouterCamelContext.java
@@ -89,6 +89,8 @@ public class RouterCamelContext implements IRouterCamelContext {
private Map<String, String> kafkaProps;
private String configType;
private String allowedEndpoints;
+ private String permittedImportBaseDirs;
+ private String permittedExportBaseDirs;
private BundleContext bundleContext;
private ConfigSharingService configSharingService;
private ExecutionContextManager contextManager;
@@ -139,6 +141,10 @@ public class RouterCamelContext implements IRouterCamelContext {
LOGGER.info("Initialize Camel Context...");
configSharingService.setProperty(RouterConstants.IMPORT_ONESHOT_UPLOAD_DIR, uploadDir);
+ // shared with router-rest, which validates a configuration's endpoint before it is stored
+ configSharingService.setProperty(RouterConstants.CONFIG_ALLOWED_ENDPOINTS, allowedEndpoints);
+ configSharingService.setProperty(RouterConstants.CONFIG_IMPORT_BASE_DIRS, permittedImportBaseDirs);
+ configSharingService.setProperty(RouterConstants.CONFIG_EXPORT_BASE_DIRS, permittedExportBaseDirs);
configSharingService.setProperty(RouterConstants.KEY_HISTORY_SIZE, execHistorySize);
initCamel();
@@ -281,6 +287,7 @@ public class RouterCamelContext implements IRouterCamelContext {
builderReader.setImportConfigurationService(importConfigurationService);
builderReader.setJacksonDataFormat(jacksonDataFormat);
builderReader.setAllowedEndpoints(allowedEndpoints);
+ builderReader.setPermittedImportBaseDirs(permittedImportBaseDirs);
builderReader.setContext(camelContext);
builderReader.setExecutionContextManager(contextManager);
builderReader.setSecurityService(securityService);
@@ -308,8 +315,10 @@ public class RouterCamelContext implements IRouterCamelContext {
//Profiles collect
ProfileExportCollectRouteBuilder profileExportCollectRouteBuilder = new ProfileExportCollectRouteBuilder(kafkaProps, configType);
profileExportCollectRouteBuilder.setExportConfigurationList(exportConfigurationService.getAll());
+ profileExportCollectRouteBuilder.setExportConfigurationService(exportConfigurationService);
profileExportCollectRouteBuilder.setPersistenceService(persistenceService);
profileExportCollectRouteBuilder.setAllowedEndpoints(allowedEndpoints);
+ profileExportCollectRouteBuilder.setPermittedExportBaseDirs(permittedExportBaseDirs);
profileExportCollectRouteBuilder.setJacksonDataFormat(jacksonDataFormat);
profileExportCollectRouteBuilder.setContext(camelContext);
profileExportCollectRouteBuilder.setExecutionContextManager(contextManager);
@@ -375,6 +384,7 @@ public class RouterCamelContext implements IRouterCamelContext {
builder.setImportConfigurationService(importConfigurationService);
builder.setProfileService(profileService);
builder.setAllowedEndpoints(allowedEndpoints);
+ builder.setPermittedImportBaseDirs(permittedImportBaseDirs);
builder.setJacksonDataFormat(jacksonDataFormat);
builder.setContext(camelContext);
builder.setExecutionContextManager(contextManager);
@@ -396,9 +406,11 @@ public class RouterCamelContext implements IRouterCamelContext {
if (RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT.equals(exportConfiguration.getConfigType())) {
ProfileExportCollectRouteBuilder profileExportCollectRouteBuilder = new ProfileExportCollectRouteBuilder(kafkaProps, configType);
profileExportCollectRouteBuilder.setExportConfigurationList(Collections.singletonList(exportConfiguration));
+ profileExportCollectRouteBuilder.setExportConfigurationService(exportConfigurationService);
profileExportCollectRouteBuilder.setPersistenceService(persistenceService);
profileExportCollectRouteBuilder.setExecutionContextManager(contextManager);
profileExportCollectRouteBuilder.setAllowedEndpoints(allowedEndpoints);
+ profileExportCollectRouteBuilder.setPermittedExportBaseDirs(permittedExportBaseDirs);
profileExportCollectRouteBuilder.setJacksonDataFormat(jacksonDataFormat);
profileExportCollectRouteBuilder.setContext(camelContext);
camelContext.addRoutes(profileExportCollectRouteBuilder);
@@ -477,4 +489,12 @@ public class RouterCamelContext implements IRouterCamelContext {
public void setSecurityService(SecurityService securityService) {
this.securityService = securityService;
}
+
+ public void setPermittedImportBaseDirs(String permittedImportBaseDirs) {
+ this.permittedImportBaseDirs = permittedImportBaseDirs;
+ }
+
+ public void setPermittedExportBaseDirs(String permittedExportBaseDirs) {
+ this.permittedExportBaseDirs = permittedExportBaseDirs;
+ }
}
diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileExportCollectRouteBuilder.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileExportCollectRouteBuilder.java
index 48fb967f0..27480bbea 100644
--- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileExportCollectRouteBuilder.java
+++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileExportCollectRouteBuilder.java
@@ -19,11 +19,12 @@ package org.apache.unomi.router.core.route;
import org.apache.camel.LoggingLevel;
import org.apache.camel.component.kafka.KafkaEndpoint;
import org.apache.camel.model.ProcessorDefinition;
-import org.apache.commons.lang3.StringUtils;
import org.apache.unomi.api.services.ExecutionContextManager;
import org.apache.unomi.persistence.spi.PersistenceService;
+import org.apache.unomi.router.api.EndpointValidator;
import org.apache.unomi.router.api.ExportConfiguration;
import org.apache.unomi.router.api.RouterConstants;
+import org.apache.unomi.router.api.services.ImportExportConfigurationService;
import org.apache.unomi.router.core.bean.CollectProfileBean;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -55,6 +56,8 @@ public class ProfileExportCollectRouteBuilder extends RouterAbstractRouteBuilder
/** List of export configurations to process */
private List<ExportConfiguration> exportConfigurationList;
+ private ImportExportConfigurationService<ExportConfiguration> exportConfigurationService;
+
/** Service for persisting and retrieving data */
private PersistenceService persistenceService;
@@ -103,7 +106,10 @@ public class ProfileExportCollectRouteBuilder extends RouterAbstractRouteBuilder
exportConfiguration.getProperties() != null && exportConfiguration.getProperties().size() > 0) {
if ((Map<String, String>) exportConfiguration.getProperties().get("mapping") != null) {
String destinationEndpoint = (String) exportConfiguration.getProperties().get("destination");
- if (StringUtils.isNotBlank(destinationEndpoint) && allowedEndpoints.contains(destinationEndpoint.substring(0, destinationEndpoint.indexOf(':')))) {
+ String refusal = EndpointValidator.validateForTenant(destinationEndpoint, allowedEndpoints, permittedBaseDirs,
+ exportConfiguration.getTenantId());
+ recordEndpointOutcome(exportConfiguration, exportConfigurationService, refusal, executionContextManager);
+ if (refusal == null) {
String timerString = "timer://collectProfile?fixedRate=true&period=" + (String) exportConfiguration.getProperties().get("period");
if ((String) exportConfiguration.getProperties().get("delay") != null) {
timerString += "&delay=" + (String) exportConfiguration.getProperties().get("delay");
@@ -124,7 +130,7 @@ public class ProfileExportCollectRouteBuilder extends RouterAbstractRouteBuilder
prDef.to((String) getEndpointURI(RouterConstants.DIRECTION_FROM, RouterConstants.DIRECT_EXPORT_DEPOSIT_BUFFER));
}
} else {
- LOGGER.error("Endpoint scheme {} is not allowed, route {} will be skipped.", destinationEndpoint.substring(0, destinationEndpoint.indexOf(':')), exportConfiguration.getItemId());
+ LOGGER.error("Destination endpoint is refused ({}), route {} will be skipped.", refusal, exportConfiguration.getItemId());
}
} else {
LOGGER.warn("Mapping is null in export configuration, route {} will be skipped!", exportConfiguration.getItemId());
@@ -135,6 +141,20 @@ public class ProfileExportCollectRouteBuilder extends RouterAbstractRouteBuilder
}
}
+ /**
+ * Sets the comma-separated list of base directories an export {@code file} endpoint may resolve into.
+ * Each tenant is then confined to {@code {baseDir}/{tenantId}}.
+ *
+ * @param permittedExportBaseDirs the permitted base directories
+ */
+ public void setPermittedExportBaseDirs(String permittedExportBaseDirs) {
+ this.permittedBaseDirs = permittedExportBaseDirs;
+ }
+
+ public void setExportConfigurationService(ImportExportConfigurationService<ExportConfiguration> exportConfigurationService) {
+ this.exportConfigurationService = exportConfigurationService;
+ }
+
/**
* Sets the list of export configurations to process.
*
diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportFromSourceRouteBuilder.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportFromSourceRouteBuilder.java
index f7906f3f5..c31713256 100644
--- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportFromSourceRouteBuilder.java
+++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/ProfileImportFromSourceRouteBuilder.java
@@ -23,8 +23,9 @@ import org.apache.camel.ShutdownRunningTask;
import org.apache.camel.component.kafka.KafkaEndpoint;
import org.apache.camel.model.ProcessorDefinition;
import org.apache.commons.lang3.StringUtils;
-import org.apache.unomi.api.services.ExecutionContextManager;
import org.apache.unomi.api.security.SecurityService;
+import org.apache.unomi.api.services.ExecutionContextManager;
+import org.apache.unomi.router.api.EndpointValidator;
import org.apache.unomi.router.api.ImportConfiguration;
import org.apache.unomi.router.api.RouterConstants;
import org.apache.unomi.router.api.services.ImportExportConfigurationService;
@@ -135,21 +136,17 @@ public class ProfileImportFromSourceRouteBuilder extends RouterAbstractRouteBuil
lineSplitProcessor.setProfilePropertyTypes(profileService.getTargetPropertyTypes("profiles"));
String endpoint = (String) importConfiguration.getProperties().get("source");
- if (StringUtils.isBlank(endpoint)) {
- LOGGER.error("No source endpoint configured, route {} will be skipped.", importConfiguration.getItemId());
- continue;
+ if (StringUtils.isNotBlank(endpoint)) {
+ // The separator depends on whether the source already carries a query: appending
+ // '&' to a source that has none makes the option part of the directory name.
+ // Poll immediately rather than waiting Camel's default 1-second initialDelay.
+ endpoint += (endpoint.indexOf('?') < 0 ? "?" : "&") + "initialDelay=0&moveFailed=.error";
}
- // Poll immediately when the route starts rather than waiting the default 1-second initialDelay.
- endpoint += "&initialDelay=0&moveFailed=.error";
- int schemeSeparatorIndex = endpoint.indexOf(':');
- if (schemeSeparatorIndex < 0) {
- LOGGER.error("Endpoint {} has no scheme, route {} will be skipped.", endpoint, importConfiguration.getItemId());
- continue;
- }
- String endpointScheme = endpoint.substring(0, schemeSeparatorIndex);
-
- if (allowedEndpoints.contains(endpointScheme)) {
+ String refusal = EndpointValidator.validateForTenant(endpoint, allowedEndpoints, permittedBaseDirs,
+ importConfiguration.getTenantId());
+ recordEndpointOutcome(importConfiguration, importConfigurationService, refusal, executionContextManager);
+ if (refusal == null) {
ProcessorDefinition prDef = from(endpoint)
.routeId(importConfiguration.getItemId())// This allow identification of the route for manual start/stop
.autoStartup(importConfiguration.isActive())// Auto-start if the import configuration is set active
@@ -186,12 +183,22 @@ public class ProfileImportFromSourceRouteBuilder extends RouterAbstractRouteBuil
prDef.to((String) getEndpointURI(RouterConstants.DIRECTION_FROM, RouterConstants.DIRECT_IMPORT_DEPOSIT_BUFFER));
}
} else {
- LOGGER.error("Endpoint scheme {} is not allowed, route {} will be skipped.", endpointScheme, importConfiguration.getItemId());
+ LOGGER.error("Source endpoint is refused ({}), route {} will be skipped.", refusal, importConfiguration.getItemId());
}
}
}
}
+ /**
+ * Sets the comma-separated list of base directories an import {@code file} endpoint may resolve into.
+ * Each tenant is then confined to {@code {baseDir}/{tenantId}}.
+ *
+ * @param permittedImportBaseDirs the permitted base directories
+ */
+ public void setPermittedImportBaseDirs(String permittedImportBaseDirs) {
+ this.permittedBaseDirs = permittedImportBaseDirs;
+ }
+
/**
* Sets the list of import configurations to process.
*
diff --git a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/RouterAbstractRouteBuilder.java b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/RouterAbstractRouteBuilder.java
index 720274ec0..125c83419 100644
--- a/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/RouterAbstractRouteBuilder.java
+++ b/extensions/router/router-core/src/main/java/org/apache/unomi/router/core/route/RouterAbstractRouteBuilder.java
@@ -22,8 +22,13 @@ import org.apache.camel.component.kafka.KafkaComponent;
import org.apache.camel.component.kafka.KafkaConfiguration;
import org.apache.camel.component.kafka.KafkaEndpoint;
import org.apache.commons.lang3.StringUtils;
+import org.apache.unomi.api.services.ExecutionContextManager;
import org.apache.unomi.api.services.ProfileService;
+import org.apache.unomi.router.api.ImportExportConfiguration;
import org.apache.unomi.router.api.RouterConstants;
+import org.apache.unomi.router.api.services.ImportExportConfigurationService;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
import java.util.Map;
@@ -46,6 +51,8 @@ import java.util.Map;
*/
public abstract class RouterAbstractRouteBuilder extends RouteBuilder {
+ private static final Logger LOGGER = LoggerFactory.getLogger(RouterAbstractRouteBuilder.class);
+
/** JSON data format configuration */
protected JacksonDataFormat jacksonDataFormat;
@@ -78,6 +85,7 @@ public abstract class RouterAbstractRouteBuilder extends RouteBuilder {
/** List of allowed endpoint schemes */
protected String allowedEndpoints;
+ protected String permittedBaseDirs;
/** Service for profile operations */
protected ProfileService profileService;
@@ -101,18 +109,79 @@ public abstract class RouterAbstractRouteBuilder extends RouteBuilder {
}
/**
- * Gets the appropriate endpoint URI based on configuration type and operation.
+ * Records, on the configuration itself, whether the endpoint it names can be honoured.
+ *
+ * <p>The permitted directories are an operational setting and the configurations are user data, so
+ * the two drift apart: a configuration that was legitimate when it was created can be refused after
+ * the deployment is reconfigured. Refusing it silently leaves the owner with a configuration that
+ * looks fine and does nothing, so the refusal is written where they will see it. It is theirs to
+ * correct or remove — nothing is deleted here.
+ *
+ * <p>The other way round matters just as much: restoring the permitted directories must bring the
+ * configuration back on its own, without anyone having to touch it. Only the status this method
+ * sets is cleared, so the record of a run that genuinely failed survives.
+ *
+ * <p>The configuration is saved without asking for its running route to be refreshed: the refresh
+ * would rebuild the route, refuse it again and save it again, without end.
+ *
+ * @param configuration the configuration whose endpoint was examined
+ * @param service the service holding that kind of configuration
+ * @param refusal the reason the endpoint was refused, or {@code null} if it can be honoured
+ */
+ protected <T extends ImportExportConfiguration> void recordEndpointOutcome(
+ T configuration, ImportExportConfigurationService<T> service, String refusal) {
+ if (refusal != null) {
+ configuration.setStatus(RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT);
+ saveQuietly(configuration, service);
+ } else if (RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT.equals(configuration.getStatus())) {
+ configuration.setStatus(null);
+ saveQuietly(configuration, service);
+ }
+ }
+
+ /**
+ * Saves the mark, and keeps a failure to itself.
+ *
+ * <p>This runs inside {@code configure()}, which builds the routes of every configuration of the
+ * batch. An exception thrown here would leave {@code addRoutes} and cost all of them their routes
+ * — the very failure this validation exists to prevent, over the report of a refusal rather than
+ * the refusal itself. The store may be unreachable at start-up; the mark is worth what it costs,
+ * and no more.
+ */
+ private <T extends ImportExportConfiguration> void saveQuietly(T configuration, ImportExportConfigurationService<T> service) {
+ try {
+ service.save(configuration, false);
+ } catch (RuntimeException e) {
+ LOGGER.error("Could not record the endpoint outcome on configuration {}; its route is built "
+ + "or skipped as decided, only the record of it is missing", configuration.getItemId(), e);
+ }
+ }
+
+ /**
+ * Records the outcome under the configuration's own tenant.
*
- * <p>This method:
- * <ul>
- * <li>Creates Kafka endpoints with appropriate configuration when using Kafka</li>
- * <li>Returns direct endpoint URIs when not using Kafka</li>
- * <li>Configures consumer properties for incoming endpoints</li>
- * </ul>
+ * <p>Startup builds every tenant's routes in one pass, and the configuration service refuses a save
+ * whose current tenant is not the configuration's. Writing the mark from the wrong tenant would
+ * either fail or store it in another tenant's index. When no context manager is available the mark
+ * is written as it stands, which is what the unit tests do.
+ */
+ protected <T extends ImportExportConfiguration> void recordEndpointOutcome(
+ T configuration, ImportExportConfigurationService<T> service, String refusal,
+ ExecutionContextManager executionContextManager) {
+ String tenantId = configuration.getTenantId();
+ if (executionContextManager != null && tenantId != null && !tenantId.isEmpty()) {
+ executionContextManager.executeAsTenant(tenantId, () -> recordEndpointOutcome(configuration, service, refusal));
+ } else {
+ recordEndpointOutcome(configuration, service, refusal);
+ }
+ }
+
+ /**
+ * Gets the appropriate endpoint URI based on configuration type and operation.
*
* @param direction the direction of the endpoint (to/from)
* @param operationDepositBuffer the operation buffer identifier
- * @return Object either a KafkaEndpoint or String depending on configuration
+ * @return either a KafkaEndpoint or a direct-endpoint URI, depending on configuration
*/
public Object getEndpointURI(String direction, String operationDepositBuffer) {
Object endpoint;
diff --git a/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml b/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
index e1084a5d6..e40ce7945 100644
--- a/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
+++ b/extensions/router/router-core/src/main/resources/OSGI-INF/blueprint/blueprint.xml
@@ -39,6 +39,8 @@
<cm:default-properties>
<cm:property name="router.config.type" value="nobroker"/>
<cm:property name="config.allowedEndpoints" value="file,ftp"/>
+ <cm:property name="config.import.baseDir" value="${karaf.data}/router/import/"/>
+ <cm:property name="config.export.baseDir" value="${karaf.data}/router/export/"/>
<cm:property name="kafka.host" value="localhost"/>
<cm:property name="kafka.port" value="9092"/>
<cm:property name="kafka.import.topic" value="import-deposit"/>
@@ -101,6 +103,8 @@
init-method="init" destroy-method="destroy">
<property name="configType" value="${router.config.type}"/>
<property name="allowedEndpoints" value="${config.allowedEndpoints}"/>
+ <property name="permittedImportBaseDirs" value="${config.import.baseDir}"/>
+ <property name="permittedExportBaseDirs" value="${config.export.baseDir}"/>
<property name="uploadDir" value="${import.oneshot.uploadDir}"/>
<property name="execHistorySize" value="${executionsHistory.size}"/>
<property name="execErrReportSize" value="${executions.error.report.size}"/>
diff --git a/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg b/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg
index 98dec513a..b61b3f2d4 100644
--- a/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg
+++ b/extensions/router/router-core/src/main/resources/org.apache.unomi.router.cfg
@@ -42,3 +42,11 @@ config.allowedEndpoints=${org.apache.unomi.router.config.allowedEndpoints:-file,
#Configs refresh interval
configs.refresh.interval=${org.apache.unomi.router.configs.refresh.interval:-1000}
+
+#Base directories a file endpoint may resolve into, comma-separated. A recurrent import source or
+#export destination using the file scheme is refused unless it resolves inside one of them, under
+#the directory named after its tenant ({baseDir}/{tenantId}), at any depth. Import and export are
+#kept apart so that an export cannot write into a directory an import route is polling; point them
+#at the same directory only if that is what you mean.
+config.import.baseDir=${org.apache.unomi.router.config.import.baseDir:-${karaf.data}/router/import/}
+config.export.baseDir=${org.apache.unomi.router.config.export.baseDir:-${karaf.data}/router/export/}
diff --git a/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/FileEndpointContainmentTest.java b/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/FileEndpointContainmentTest.java
new file mode 100644
index 000000000..cdc6a5ddc
--- /dev/null
+++ b/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/FileEndpointContainmentTest.java
@@ -0,0 +1,760 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.router.core.route;
+
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.apache.unomi.api.services.ProfileService;
+import org.apache.unomi.router.api.ExportConfiguration;
+import org.apache.unomi.router.api.ImportConfiguration;
+import org.apache.unomi.router.api.ProfileToImport;
+import org.apache.unomi.router.api.RouterConstants;
+import org.apache.unomi.router.api.services.ImportExportConfigurationService;
+import org.junit.After;
+import org.junit.Assume;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import java.io.File;
+import java.io.IOException;
+import java.lang.reflect.Proxy;
+import java.nio.file.Files;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+/**
+ * A recurrent import configuration names a {@code source}, and a recurrent export configuration
+ * names a {@code destination}. Both are used as Apache Camel endpoint URIs. When the URI uses the
+ * {@code file} scheme, it must resolve <em>inside</em> one of the base directories the deployment
+ * permits: a {@code file} endpoint that resolves anywhere else is refused, and its route is not
+ * built.
+ *
+ * <p>Containment covers the directory named by the URI <em>and</em> every path-bearing option it
+ * carries ({@code fileName}, {@code move}, {@code moveFailed}, {@code preMove}, {@code doneFileName},
+ * ...) — validating only the directory part would leave
+ * {@code file:///permitted/?fileName=../../elsewhere} open. It is decided on what Camel will use, so
+ * the URI is read the way Camel reads it: percent-encoded option names, {@code RAW()} values that
+ * carry an ampersand, and the File Language expressions a path-bearing option may hold.
+ *
+ * <p>Selection options ({@code include}, {@code antInclude}) are patterns matched against the files
+ * the directory already offers, not paths Camel resolves, and are not held to containment.
+ *
+ * <p>Remote schemes ({@code ftp}, {@code sftp}, {@code ftps}) address a remote server, so their
+ * directory and their path-bearing options are remote and are not subject to directory containment;
+ * the scheme allow-list keeps governing them. {@code localWorkDirectory} is the exception: it names a
+ * local directory, which the remote components stage their downloads in, and it is confined whatever
+ * the scheme.
+ *
+ * <p>These tests exercise route <em>construction</em>, which is where a configuration is turned into
+ * a live route: a configuration whose endpoint is refused must leave no route behind, whether it
+ * arrives through the REST API or was already persisted before the deployment was configured.
+ */
+public class FileEndpointContainmentTest {
+
+ /** The shipped default. The containment rules must hold while {@code file} is an allowed scheme. */
+ private static final String DEFAULT_ALLOWED_ENDPOINTS = "file,ftp,sftp,ftps";
+
+ private static final String TENANT = "acme";
+
+ private static final Map<String, String> NO_KAFKA = new HashMap<>();
+
+ @Rule
+ public TemporaryFolder tmp = new TemporaryFolder();
+
+ private DefaultCamelContext camelContext;
+
+ /** The deployment's import base directory. Each tenant is confined to a subdirectory of it. */
+ private File importRoot;
+
+ /** {@code importRoot/acme}, the directory tenant {@code acme} may use. */
+ private File permittedImportDir;
+
+ /** The deployment's export base directory. */
+ private File exportRoot;
+
+ /** {@code exportRoot/acme}. */
+ private File permittedExportDir;
+
+ /** A directory the deployment never permitted — the "arbitrary directory" of the defect. */
+ private File arbitraryDir;
+
+ /** A sibling whose path shares a textual prefix with the permitted import directory. */
+ private File permittedImportDirLookalike;
+
+ @Before
+ public void setUp() throws Exception {
+ importRoot = tmp.newFolder("permitted-import");
+ permittedImportDir = new File(importRoot, TENANT);
+ assertTrue(permittedImportDir.mkdirs());
+ exportRoot = tmp.newFolder("permitted-export");
+ permittedExportDir = new File(exportRoot, TENANT);
+ assertTrue(permittedExportDir.mkdirs());
+ arbitraryDir = tmp.newFolder("arbitrary");
+ permittedImportDirLookalike = tmp.newFolder("permitted-import-evil");
+ camelContext = new DefaultCamelContext();
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ camelContext.stop();
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Import — the configured source is read by Unomi
+ // ---------------------------------------------------------------------------------------------
+
+ @Test
+ public void importRouteIsBuiltWhenSourceIsInsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("in-bounds", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("in-bounds", "a source inside the permitted base directory is legitimate");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenSourceIsInAnExistingSubdirectoryOfPermittedBaseDir() throws Exception {
+ File dropDir = new File(permittedImportDir, "incoming");
+ assertTrue("could not prepare the test fixture", dropDir.mkdir());
+
+ addImportRoutes(recurrentImport("subdirectory", fileUri(dropDir, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("subdirectory", "containment is recursive — a source at any depth under the base directory is legitimate");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceIsASymlinkPointingOutsidePermittedBaseDir() throws Exception {
+ File link = new File(permittedImportDir, "elsewhere");
+ try {
+ Files.createSymbolicLink(link.toPath(), arbitraryDir.toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ addImportRoutes(recurrentImport("symlink", fileUri(link, "?fileName=profiles.csv")));
+
+ assertRouteRefused("symlink", "the source leaves the permitted base directory once symbolic links are resolved");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceLeavesPermittedBaseDirThroughASymlinkAndAParentSegment() throws Exception {
+ File outsideChild = new File(arbitraryDir, "child");
+ assertTrue("could not prepare the test fixture", outsideChild.mkdir());
+ File link = new File(permittedImportDir, "link");
+ try {
+ Files.createSymbolicLink(link.toPath(), outsideChild.toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ // the file system expands the link, then applies the parent segment to its target: the source
+ // is the arbitrary directory. Collapsing the segment first would read it as the base directory.
+ addImportRoutes(recurrentImport("symlink-parent", fileUri(new File(link, ".."), "?fileName=profiles.csv")));
+
+ assertRouteRefused("symlink-parent",
+ "a parent segment applies to the target of the link that precedes it, not to the link's own parent");
+ }
+
+ @Test
+ public void moveFailedIsAppendedAsTheFirstOptionOfASourceThatCarriesNone() throws Exception {
+ addImportRoutes(recurrentImport("first-option", fileUri(permittedImportDir, "")));
+
+ assertEquals("the option must open the query, or it becomes part of the directory name",
+ fileUri(permittedImportDir, "?initialDelay=0&moveFailed=.error"), builtEndpointUri("first-option"));
+ }
+
+ @Test
+ public void moveFailedIsAppendedToASourceThatAlreadyCarriesAQuery() throws Exception {
+ addImportRoutes(recurrentImport("next-option", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertEquals("an existing query takes the option as another parameter",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&initialDelay=0&moveFailed=.error"),
+ builtEndpointUri("next-option"));
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenSourceCarriesNoOption() throws Exception {
+ // the router appends moveFailed itself; a source with no query has no separator to append to,
+ // and the option would otherwise become part of the directory the endpoint names
+ addImportRoutes(recurrentImport("no-option", fileUri(permittedImportDir, "")));
+
+ assertRouteBuilt("no-option", "a source may name a directory and nothing else");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenSourceIsASymlinkPointingInsidePermittedBaseDir() throws Exception {
+ File target = new File(permittedImportDir, "incoming");
+ assertTrue("could not prepare the test fixture", target.mkdir());
+ File link = new File(permittedImportDir, "shortcut");
+ try {
+ Files.createSymbolicLink(link.toPath(), target.toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ addImportRoutes(recurrentImport("symlink-inside", fileUri(link, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("symlink-inside", "resolving a link must not refuse one that stays inside the base directory");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenThePermittedBaseDirIsItselfASymlink() throws Exception {
+ File link = new File(tmp.getRoot(), "permitted-import-link");
+ try {
+ Files.createSymbolicLink(link.toPath(), importRoot.toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ // the deployment names the link, the configuration names the directory it points at
+ addImportRoutesInto(link, recurrentImport("base-dir-link", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("base-dir-link",
+ "both sides are resolved, so naming the same directory by two paths is the same directory");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenAnOptionLeavesThroughASymlinkAndAParentSegment() throws Exception {
+ File outsideChild = new File(arbitraryDir, "child");
+ assertTrue("could not prepare the test fixture", outsideChild.mkdir());
+ File link = new File(permittedImportDir, "option-link");
+ try {
+ Files.createSymbolicLink(link.toPath(), outsideChild.toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ addImportRoutes(recurrentImport("option-symlink-parent",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=option-link/..")));
+
+ assertRouteRefused("option-symlink-parent",
+ "a path-bearing option is resolved the same way the endpoint directory is");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceClimbsAboveTheRoot() throws Exception {
+ addImportRoutes(recurrentImport("above-root", "file:///../../../../../../../../etc?fileName=profiles.csv"));
+
+ assertRouteRefused("above-root",
+ "more parent segments than the path has components is a refusal, not an exception");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceIsOutsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("arbitrary-dir", fileUri(arbitraryDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("arbitrary-dir", "the source directory is not one the deployment permits");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceEscapesPermittedBaseDirWithParentSegments() throws Exception {
+ String escaping = fileUri(permittedImportDir, "/../" + arbitraryDir.getName() + "?fileName=profiles.csv");
+
+ addImportRoutes(recurrentImport("dot-dot", escaping));
+
+ assertRouteRefused("dot-dot", "the source resolves outside the permitted base directory once normalized");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceEscapesPermittedBaseDirWithEncodedParentSegments() throws Exception {
+ String escaping = fileUri(permittedImportDir, "/%2e%2e/" + arbitraryDir.getName() + "?fileName=profiles.csv");
+
+ addImportRoutes(recurrentImport("encoded-dot-dot", escaping));
+
+ assertRouteRefused("encoded-dot-dot", "percent-encoded parent segments must be decoded before containment is decided");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceDirectoryOnlySharesAPrefixWithPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("lookalike", fileUri(permittedImportDirLookalike, "?fileName=profiles.csv")));
+
+ assertRouteRefused("lookalike", "containment compares path components, not string prefixes");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenFileNameOptionEscapesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("filename-escape",
+ fileUri(permittedImportDir, "?fileName=../" + arbitraryDir.getName() + "/profiles.csv")));
+
+ assertRouteRefused("filename-escape", "fileName carries a path and must be contained too");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenMoveOptionEscapesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("move-escape",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=../" + arbitraryDir.getName())));
+
+ assertRouteRefused("move-escape", "move carries a path and must be contained too");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenMoveFailedOptionEscapesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("movefailed-escape",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&moveFailed=../" + arbitraryDir.getName())));
+
+ assertRouteRefused("movefailed-escape",
+ "moveFailed carries a path, and the route builder appends one of its own — every occurrence must be contained");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenPreMoveOptionEscapesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("premove-escape",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&preMove=../" + arbitraryDir.getName())));
+
+ assertRouteRefused("premove-escape", "preMove carries a path and must be contained too");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenDoneFileNameOptionEscapesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("donefilename-escape",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&doneFileName=../" + arbitraryDir.getName() + "/done")));
+
+ assertRouteRefused("donefilename-escape", "doneFileName carries a path and must be contained too");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenPathBearingOptionIsWrappedInRaw() throws Exception {
+ addImportRoutes(recurrentImport("raw-escape",
+ fileUri(permittedImportDir, "?fileName=RAW(../" + arbitraryDir.getName() + "/profiles.csv)")));
+
+ assertRouteRefused("raw-escape", "RAW() only tells Camel not to decode the value — the path it carries is used as-is");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenMoveOptionIsRelativeAndStaysInsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("relative-move",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=.done")));
+
+ assertRouteBuilt("relative-move", "a relative move target inside the base directory is how the feature is normally used");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenPathBearingOptionNameIsPercentEncoded() throws Exception {
+ addImportRoutes(recurrentImport("encoded-option-name",
+ fileUri(permittedImportDir, "?file%4Eame=../" + arbitraryDir.getName() + "/profiles.csv")));
+
+ assertRouteRefused("encoded-option-name",
+ "Camel decodes an option name before it binds it, so 'file%4Eame' is the fileName option");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenRawOptionValueCarriesTheEscapeBehindAnAmpersand() throws Exception {
+ addImportRoutes(recurrentImport("raw-ampersand",
+ fileUri(permittedImportDir, "?fileName=RAW(x&/../../" + arbitraryDir.getName() + "/profiles.csv)")));
+
+ assertRouteRefused("raw-ampersand",
+ "a RAW value ends at the marker that closes it, so the ampersand it carries does not start a new option");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenMoveOptionUsesAnExpressionThatLeavesPermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("expression-escape",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=${file:parent}/../" + arbitraryDir.getName())));
+
+ assertRouteRefused("expression-escape",
+ "Camel evaluates move as an expression, and this one sends the consumed file to a sibling directory");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenMoveOptionUsesAnExpressionThatCannotBeValidated() throws Exception {
+ addImportRoutes(recurrentImport("opaque-expression",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=${header.destination}")));
+
+ assertRouteRefused("opaque-expression", "a header can hold any path at all, so there is nothing left to validate");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenMoveOptionUsesTheParentExpressionAndStaysInsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("parent-expression",
+ fileUri(permittedImportDir, "?fileName=profiles.csv&move=${file:parent}/.done/${file:onlyname}")));
+
+ assertRouteBuilt("parent-expression", "moving a consumed file next to itself is how the feature is normally used");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenSelectionPatternsAreNotPaths() throws Exception {
+ addImportRoutes(recurrentImport("selection-patterns",
+ fileUri(permittedImportDir, "?include=..&antInclude=**/*.csv&consumer.delay=10m")));
+
+ assertRouteBuilt("selection-patterns",
+ "a selection pattern is matched against the files the directory offers, so '..' asks for a two-character "
+ + "name — it is not a path Camel resolves, and holding it to containment only refuses patterns");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceIsADanglingSymlinkInsidePermittedBaseDir() throws Exception {
+ File link = new File(permittedImportDir, "dangling");
+ try {
+ Files.createSymbolicLink(link.toPath(), new File(tmp.getRoot(), "not-created-yet").toPath());
+ } catch (IOException | UnsupportedOperationException e) {
+ Assume.assumeNoException("this file system does not support symbolic links", e);
+ }
+
+ addImportRoutes(recurrentImport("dangling-symlink", fileUri(link, "?fileName=profiles.csv")));
+
+ assertRouteRefused("dangling-symlink",
+ "a link whose target cannot be resolved would leave the base directory as soon as its target is created");
+ }
+
+ @Test
+ public void unusablePathIsSkippedWithoutPreventingTheOtherRoutesFromBeingBuilt() throws Exception {
+ addImportRoutes(
+ recurrentImport("nul-character", fileUri(permittedImportDir, "?fileName=profiles%00.csv")),
+ recurrentImport("well-formed", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("nul-character", "the file system cannot use a path that holds a nul character");
+ assertRouteBuilt("well-formed",
+ "a path the file system rejects is a refusal, not an exception that costs the deployment its other routes");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceIsTheSharedBaseOrAnotherTenantsDirectory() throws Exception {
+ File otherTenant = new File(importRoot, "beta");
+ assertTrue(otherTenant.mkdirs());
+
+ ImportConfiguration shared = recurrentImport("shared-base", fileUri(importRoot, "?fileName=profiles.csv"));
+ ImportConfiguration foreign = recurrentImport("other-tenant", fileUri(otherTenant, "?fileName=profiles.csv"));
+ ImportConfiguration own = recurrentImport("own-tenant", fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ ImportConfiguration beta = recurrentImport("beta-tenant", fileUri(otherTenant, "?fileName=profiles.csv"));
+ beta.setTenantId("beta");
+
+ addImportRoutes(shared, foreign, own, beta);
+
+ assertRouteRefused("shared-base", "the shared base directory is not a tenant directory");
+ assertRouteRefused("other-tenant", "acme may not read beta's directory");
+ assertRouteBuilt("own-tenant", "acme may read its own directory");
+ assertRouteBuilt("beta-tenant", "beta may read its own directory");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenDestinationIsAnotherTenantsDirectory() throws Exception {
+ File otherTenant = new File(exportRoot, "beta");
+ assertTrue(otherTenant.mkdirs());
+
+ addExportRoutes(recurrentExport("other-tenant", fileUri(otherTenant, "?fileName=profiles.csv")));
+
+ assertRouteRefused("other-tenant", "acme may not write into beta's directory");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenPermittedBaseDirIsNonAsciiAndTheUriIsPartlyEncoded() throws Exception {
+ File nonAsciiBaseDir = tmp.newFolder("caf\u00e9-import");
+ Assume.assumeTrue("this file system does not keep non-ASCII directory names", nonAsciiBaseDir.isDirectory());
+ File tenantDir = new File(nonAsciiBaseDir, TENANT);
+ File dropDir = new File(tenantDir, "drop zone");
+ assertTrue("could not prepare the test fixture", dropDir.mkdirs());
+
+ addImportRoutesInto(nonAsciiBaseDir,
+ recurrentImport("non-ascii", fileUri(tenantDir, "/drop%20zone?fileName=profiles.csv")));
+
+ assertRouteBuilt("non-ascii",
+ "decoding an escape must not corrupt the characters around it, or containment is decided on another path");
+ }
+
+ @Test
+ public void remoteImportEndpointIsNotSubjectToDirectoryContainment() throws Exception {
+ addImportRoutes(recurrentImport("remote", "ftp://ftp.example.com/profiles?fileName=profiles.csv"));
+
+ assertRouteBuilt("remote", "ftp is an allowed scheme and carries no local path");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenARemoteSourceStagesItsDownloadsOutsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("remote-staging", "ftp://ftp.example.com/profiles"
+ + "?fileName=profiles.csv&localWorkDirectory=" + arbitraryDir.getAbsolutePath()));
+
+ assertRouteRefused("remote-staging",
+ "localWorkDirectory is a local directory: the remote server's content is written there, "
+ + "under the name the remote server chooses");
+ }
+
+ @Test
+ public void importRouteIsBuiltWhenARemoteSourceStagesItsDownloadsInsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("remote-staging-in-bounds", "ftp://ftp.example.com/profiles"
+ + "?fileName=profiles.csv&localWorkDirectory="
+ + new File(permittedImportDir, "staging").getAbsolutePath()));
+
+ assertRouteBuilt("remote-staging-in-bounds",
+ "staging downloads inside the permitted base directory is legitimate, and the directory "
+ + "need not exist yet");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSourceStagesItsDownloadsOutsidePermittedBaseDir() throws Exception {
+ addImportRoutes(recurrentImport("local-staging", fileUri(permittedImportDir,
+ "?fileName=profiles.csv&localWorkDirectory=" + arbitraryDir.getAbsolutePath())));
+
+ assertRouteRefused("local-staging",
+ "the option is resolved on its own, not against the endpoint directory, so a permitted "
+ + "directory does not cover it");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSchemeIsNotAllowed() throws Exception {
+ addImportRoutes("ftp,sftp,ftps", recurrentImport("scheme-denied", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("scheme-denied", "file is not in the configured scheme allow-list");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenSchemeIsOnlyASubstringOfAnAllowedScheme() throws Exception {
+ addImportRoutes(recurrentImport("substring-scheme", "fil://" + permittedImportDir.getAbsolutePath() + "?fileName=profiles.csv"));
+
+ assertRouteRefused("substring-scheme", "the allow-list is a set of schemes, not a string to search");
+ }
+
+ @Test
+ public void malformedSourceIsSkippedWithoutPreventingTheOtherRoutesFromBeingBuilt() throws Exception {
+ addImportRoutes(
+ recurrentImport("no-scheme", permittedImportDir.getAbsolutePath() + "/profiles.csv"),
+ recurrentImport("well-formed", fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("no-scheme", "an endpoint without a scheme cannot be honoured");
+ assertRouteBuilt("well-formed", "one malformed configuration must not cost the deployment its other routes");
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Export — the configured destination is written by Unomi
+ // ---------------------------------------------------------------------------------------------
+
+ @Test
+ public void exportRouteIsBuiltWhenDestinationIsInsidePermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("in-bounds", fileUri(permittedExportDir, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("in-bounds", "a destination inside the permitted base directory is legitimate");
+ }
+
+ @Test
+ public void exportRouteIsBuiltWhenDestinationIsInASubdirectoryThatDoesNotExistYet() throws Exception {
+ File notCreatedYet = new File(permittedExportDir, "2026-08");
+
+ addExportRoutes(recurrentExport("subdirectory", fileUri(notCreatedYet, "?fileName=profiles.csv")));
+
+ assertRouteBuilt("subdirectory", "an export destination is created on first write — containment must not require the directory to exist");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenDestinationIsOutsidePermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("arbitrary-dir", fileUri(arbitraryDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("arbitrary-dir", "the destination directory is not one the deployment permits");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenDestinationEscapesPermittedBaseDirWithParentSegments() throws Exception {
+ String escaping = fileUri(permittedExportDir, "/../" + arbitraryDir.getName() + "?fileName=profiles.csv");
+
+ addExportRoutes(recurrentExport("dot-dot", escaping));
+
+ assertRouteRefused("dot-dot", "the destination resolves outside the permitted base directory once normalized");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenFileNameOptionEscapesPermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("filename-escape",
+ fileUri(permittedExportDir, "?fileName=../" + arbitraryDir.getName() + "/profiles.csv")));
+
+ assertRouteRefused("filename-escape", "fileName carries a path and must be contained too");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenTempFileNameOptionEscapesPermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("tempfilename-escape",
+ fileUri(permittedExportDir, "?fileName=profiles.csv&tempFileName=../" + arbitraryDir.getName() + "/profiles.tmp")));
+
+ assertRouteRefused("tempfilename-escape", "tempFileName carries a path and must be contained too");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenDoneFileNameOptionEscapesPermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("donefilename-escape",
+ fileUri(permittedExportDir, "?fileName=profiles.csv&doneFileName=../" + arbitraryDir.getName() + "/done")));
+
+ assertRouteRefused("donefilename-escape", "doneFileName carries a path and must be contained too");
+ }
+
+ @Test
+ public void exportRouteIsBuiltWhenFileNameOptionUsesADateExpression() throws Exception {
+ addExportRoutes(recurrentExport("date-expression",
+ fileUri(permittedExportDir, "?fileName=profiles-export-${date:now:yyyyMMddHHmm}.csv")));
+
+ assertRouteBuilt("date-expression",
+ "a date cannot hold a parent segment, and naming an export after it is what the documentation shows");
+ }
+
+ @Test
+ public void remoteExportEndpointIsNotSubjectToDirectoryContainment() throws Exception {
+ addExportRoutes(recurrentExport("remote", "ftp://ftp.example.com/profiles?fileName=profiles.csv"));
+
+ assertRouteBuilt("remote", "ftp is an allowed scheme and carries no local path");
+ }
+
+ @Test
+ public void exportRouteIsRefusedWhenDestinationStagesItsDownloadsOutsidePermittedBaseDir() throws Exception {
+ addExportRoutes(recurrentExport("remote-staging", "ftp://ftp.example.com/profiles"
+ + "?fileName=profiles.csv&localWorkDirectory=" + arbitraryDir.getAbsolutePath()));
+
+ assertRouteRefused("remote-staging", "the option names a local directory in either direction");
+ }
+
+ @Test
+ public void importRouteIsRefusedWhenTheStagingOptionIsSpeltInAnotherCase() throws Exception {
+ addImportRoutes(recurrentImport("staging-case", "ftp://ftp.example.com/profiles"
+ + "?fileName=profiles.csv&LOCALWORKDIRECTORY=" + arbitraryDir.getAbsolutePath()));
+
+ assertRouteRefused("staging-case", "the option is matched on its name, not on how it is spelt");
+ }
+
+ @Test
+ public void malformedDestinationIsSkippedWithoutPreventingTheOtherRoutesFromBeingBuilt() throws Exception {
+ addExportRoutes(
+ recurrentExport("blank", ""),
+ recurrentExport("well-formed", fileUri(permittedExportDir, "?fileName=profiles.csv")));
+
+ assertRouteRefused("blank", "a blank destination cannot be honoured");
+ assertRouteBuilt("well-formed", "one malformed configuration must not cost the deployment its other routes");
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Fixtures
+ // ---------------------------------------------------------------------------------------------
+
+ private void assertRouteBuilt(String routeId, String why) {
+ assertNotNull("no route was built for configuration '" + routeId + "', although " + why,
+ camelContext.getRouteDefinition(routeId));
+ }
+
+ /** The endpoint URI the route was actually built on, which is not the one that was configured. */
+ private String builtEndpointUri(String routeId) {
+ assertRouteBuilt(routeId, "there is no endpoint to look at otherwise");
+ return camelContext.getRouteDefinition(routeId).getInputs().get(0).getUri();
+ }
+
+ private void assertRouteRefused(String routeId, String why) {
+ assertNull("a route was built for configuration '" + routeId + "', although " + why,
+ camelContext.getRouteDefinition(routeId));
+ }
+
+ private String fileUri(File directory, String suffix) {
+ return "file://" + directory.getAbsolutePath() + suffix;
+ }
+
+ private ImportConfiguration recurrentImport(String itemId, String source) {
+ ImportConfiguration configuration = new ImportConfiguration();
+ configuration.setItemId(itemId);
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setActive(true);
+ configuration.getProperties().put("source", source);
+ configuration.getProperties().put("mapping", Collections.singletonMap("0", 0));
+ return configuration;
+ }
+
+ private ExportConfiguration recurrentExport(String itemId, String destination) {
+ ExportConfiguration configuration = new ExportConfiguration();
+ configuration.setItemId(itemId);
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setActive(true);
+ configuration.getProperties().put("destination", destination);
+ configuration.getProperties().put("mapping", Collections.singletonMap("0", "firstName"));
+ configuration.getProperties().put("segment", "exportSegment");
+ configuration.getProperties().put("period", "1m");
+ return configuration;
+ }
+
+ private void addImportRoutes(ImportConfiguration... configurations) throws Exception {
+ addImportRoutes(DEFAULT_ALLOWED_ENDPOINTS, configurations);
+ }
+
+ private void addImportRoutes(String allowedEndpoints, ImportConfiguration... configurations) throws Exception {
+ ProfileImportFromSourceRouteBuilder builder = importRouteBuilder(allowedEndpoints, configurations);
+ builder.setPermittedImportBaseDirs(importRoot.getAbsolutePath());
+ builder.setContext(camelContext);
+ camelContext.addRoutes(builder);
+ }
+
+ private ProfileImportFromSourceRouteBuilder importRouteBuilder(String allowedEndpoints, ImportConfiguration... configurations) {
+ ProfileImportFromSourceRouteBuilder builder =
+ new ProfileImportFromSourceRouteBuilder(NO_KAFKA, RouterConstants.CONFIG_TYPE_NOBROKER);
+ builder.setImportConfigurationList(Arrays.asList(configurations));
+ builder.setImportConfigurationService(discardingConfigurationService());
+ builder.setProfileService(noOpProfileService());
+ builder.setJacksonDataFormat(new JacksonDataFormat(ProfileToImport.class));
+ builder.setAllowedEndpoints(allowedEndpoints);
+ return builder;
+ }
+
+ private void addImportRoutesInto(File permittedBaseDir, ImportConfiguration... configurations) throws Exception {
+ ProfileImportFromSourceRouteBuilder builder = importRouteBuilder(DEFAULT_ALLOWED_ENDPOINTS, configurations);
+ builder.setPermittedImportBaseDirs(permittedBaseDir.getAbsolutePath());
+ builder.setContext(camelContext);
+ camelContext.addRoutes(builder);
+ }
+
+ private void addExportRoutes(ExportConfiguration... configurations) throws Exception {
+ ProfileExportCollectRouteBuilder builder =
+ new ProfileExportCollectRouteBuilder(NO_KAFKA, RouterConstants.CONFIG_TYPE_NOBROKER);
+ builder.setExportConfigurationList(Arrays.asList(configurations));
+ builder.setExportConfigurationService(discardingConfigurationService());
+ builder.setJacksonDataFormat(new JacksonDataFormat(ProfileToImport.class));
+ builder.setAllowedEndpoints(DEFAULT_ALLOWED_ENDPOINTS);
+ builder.setPermittedExportBaseDirs(exportRoot.getAbsolutePath());
+ builder.setContext(camelContext);
+ camelContext.addRoutes(builder);
+ }
+
+ /**
+ * A refused configuration is recorded, and these tests are not about that: what they observe is
+ * whether the route was built.
+ */
+ @SuppressWarnings("unchecked")
+ private static <T> ImportExportConfigurationService<T> discardingConfigurationService() {
+ return (ImportExportConfigurationService<T>) Proxy.newProxyInstance(
+ ImportExportConfigurationService.class.getClassLoader(),
+ new Class<?>[]{ImportExportConfigurationService.class},
+ (proxy, method, args) -> "save".equals(method.getName()) ? args[0] : null);
+ }
+
+ /**
+ * The route builders ask the profile service for the profile property types while they build.
+ * Nothing in these tests depends on what it answers.
+ */
+ private static ProfileService noOpProfileService() {
+ return (ProfileService) Proxy.newProxyInstance(
+ ProfileService.class.getClassLoader(),
+ new Class<?>[]{ProfileService.class},
+ (proxy, method, args) -> {
+ if (Collection.class.isAssignableFrom(method.getReturnType())) {
+ return Collections.emptyList();
+ }
+ if (List.class.isAssignableFrom(method.getReturnType())) {
+ return Collections.emptyList();
+ }
+ return null;
+ });
+ }
+}
diff --git a/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/RefusedConfigurationStatusTest.java b/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/RefusedConfigurationStatusTest.java
new file mode 100644
index 000000000..e6a24b59c
--- /dev/null
+++ b/extensions/router/router-core/src/test/java/org/apache/unomi/router/core/route/RefusedConfigurationStatusTest.java
@@ -0,0 +1,344 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.router.core.route;
+
+import org.apache.camel.component.jackson.JacksonDataFormat;
+import org.apache.camel.impl.DefaultCamelContext;
+import org.apache.unomi.api.services.ProfileService;
+import org.apache.unomi.router.api.ExportConfiguration;
+import org.apache.unomi.router.api.ImportConfiguration;
+import org.apache.unomi.router.api.ProfileToImport;
+import org.apache.unomi.router.api.RouterConstants;
+import org.apache.unomi.router.api.services.ImportExportConfigurationService;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import java.io.File;
+import java.lang.reflect.Proxy;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNotNull;
+import static org.junit.Assert.assertNull;
+import static org.junit.Assert.assertTrue;
+
+/**
+ * The permitted base directories are an operational setting, and the configurations are user data:
+ * the two drift apart. A configuration that was legitimate when it was created can find itself
+ * outside the permitted directories after the deployment is reconfigured, and its route then stops
+ * being built.
+ *
+ * <p>Leaving that silent is what makes it painful — the screen keeps showing the configuration as
+ * running, and the only trace is a log line nobody reads. So a configuration whose endpoint is
+ * refused while its route is being built is recorded as failed, and stays in the store: it is for
+ * whoever owns it to correct it or remove it, and they can only do that if they can see it.
+ *
+ * <p>Recording it must not schedule the configuration for a route refresh — that would have the
+ * refresh rebuild the route, refuse it again, and save it again, indefinitely.
+ */
+public class RefusedConfigurationStatusTest {
+
+ private static final String DEFAULT_ALLOWED_ENDPOINTS = "file,ftp,sftp,ftps";
+
+ private static final Map<String, String> NO_KAFKA = new HashMap<>();
+
+ @Rule
+ public TemporaryFolder tmp = new TemporaryFolder();
+
+ private DefaultCamelContext camelContext;
+
+ private static final String TENANT = "acme";
+
+ private File importRoot;
+ private File exportRoot;
+ private File permittedImportDir;
+ private File permittedExportDir;
+ private File arbitraryDir;
+
+ private RecordingConfigurationService<ImportConfiguration> importConfigurations;
+ private RecordingConfigurationService<ExportConfiguration> exportConfigurations;
+
+ @Before
+ public void setUp() throws Exception {
+ importRoot = tmp.newFolder("permitted-import");
+ permittedImportDir = new File(importRoot, TENANT);
+ assertTrue(permittedImportDir.mkdirs());
+ exportRoot = tmp.newFolder("permitted-export");
+ permittedExportDir = new File(exportRoot, TENANT);
+ assertTrue(permittedExportDir.mkdirs());
+ arbitraryDir = tmp.newFolder("arbitrary");
+ camelContext = new DefaultCamelContext();
+ importConfigurations = new RecordingConfigurationService<>();
+ exportConfigurations = new RecordingConfigurationService<>();
+ }
+
+ @After
+ public void tearDown() throws Exception {
+ camelContext.stop();
+ }
+
+ @Test
+ public void aRefusedImportConfigurationIsRecordedAsFailed() throws Exception {
+ ImportConfiguration configuration = recurrentImport(fileUri(arbitraryDir, "?fileName=profiles.csv"));
+
+ addImportRoutes(configuration);
+
+ assertNull("no route should have been built", camelContext.getRouteDefinition("out-of-bounds"));
+ assertEquals("the configuration should be recorded as failed",
+ RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT, configuration.getStatus());
+ assertTrue("the configuration should have been saved so the failure is visible",
+ importConfigurations.contains("out-of-bounds"));
+ }
+
+ @Test
+ public void aRefusedExportConfigurationIsRecordedAsFailed() throws Exception {
+ ExportConfiguration configuration = recurrentExport(fileUri(arbitraryDir, "?fileName=profiles.csv"));
+
+ addExportRoutes(configuration);
+
+ assertNull("no route should have been built", camelContext.getRouteDefinition("out-of-bounds"));
+ assertEquals("the configuration should be recorded as failed",
+ RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT, configuration.getStatus());
+ assertTrue("the configuration should have been saved so the failure is visible",
+ exportConfigurations.contains("out-of-bounds"));
+ }
+
+ @Test
+ public void aStoreThatCannotRecordTheRefusalDoesNotCostTheBatchItsOtherRoutes() throws Exception {
+ importConfigurations.unwritable = true;
+
+ addImportRoutes(recurrentImport(fileUri(arbitraryDir, "?fileName=profiles.csv")),
+ inBoundsImport("in-bounds"));
+
+ assertNull("the refused configuration still gets no route", camelContext.getRouteDefinition("out-of-bounds"));
+ assertNotNull("failing to record the refusal must not cost the other configurations their routes",
+ camelContext.getRouteDefinition("in-bounds"));
+ }
+
+ @Test
+ public void recordingARefusedImportConfigurationDoesNotScheduleARouteRefresh() throws Exception {
+ addImportRoutes(recurrentImport(fileUri(arbitraryDir, "?fileName=profiles.csv")));
+
+ assertFalse("refreshing the route would refuse and save it again, without end",
+ importConfigurations.lastSaveAskedForARouteRefresh);
+ }
+
+ @Test
+ public void recordingARefusedExportConfigurationDoesNotScheduleARouteRefresh() throws Exception {
+ addExportRoutes(recurrentExport(fileUri(arbitraryDir, "?fileName=profiles.csv")));
+
+ assertFalse("refreshing the route would refuse and save it again, without end",
+ exportConfigurations.lastSaveAskedForARouteRefresh);
+ }
+
+ @Test
+ public void anImportConfigurationRecoversWhenItsEndpointBecomesAcceptableAgain() throws Exception {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setStatus(RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT);
+
+ addImportRoutes(configuration);
+
+ assertNull("restoring the permitted directories must bring the configuration back on its own",
+ configuration.getStatus());
+ assertTrue("the recovery must be persisted", importConfigurations.contains("out-of-bounds"));
+ }
+
+ @Test
+ public void anExportConfigurationRecoversWhenItsEndpointBecomesAcceptableAgain() throws Exception {
+ ExportConfiguration configuration = recurrentExport(fileUri(permittedExportDir, "?fileName=profiles.csv"));
+ configuration.setStatus(RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT);
+
+ addExportRoutes(configuration);
+
+ assertNull("restoring the permitted directories must bring the configuration back on its own",
+ configuration.getStatus());
+ assertTrue("the recovery must be persisted", exportConfigurations.contains("out-of-bounds"));
+ }
+
+ @Test
+ public void aConfigurationThatFailedItsLastRunKeepsThatStatusWhenItsRouteIsRebuilt() throws Exception {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setStatus(RouterConstants.CONFIG_STATUS_COMPLETE_ERRORS);
+
+ addImportRoutes(configuration);
+
+ assertEquals("a failed run is a different matter, and its record must survive",
+ RouterConstants.CONFIG_STATUS_COMPLETE_ERRORS, configuration.getStatus());
+ }
+
+ @Test
+ public void anAcceptedImportConfigurationIsLeftAlone() throws Exception {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setItemId("in-bounds");
+
+ addImportRoutes(configuration);
+
+ assertNull("an accepted configuration keeps the status it had", configuration.getStatus());
+ assertFalse("an accepted configuration is not saved while its route is built",
+ importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void anAcceptedExportConfigurationIsLeftAlone() throws Exception {
+ ExportConfiguration configuration = recurrentExport(fileUri(permittedExportDir, "?fileName=profiles.csv"));
+ configuration.setItemId("in-bounds");
+
+ addExportRoutes(configuration);
+
+ assertNull("an accepted configuration keeps the status it had", configuration.getStatus());
+ assertFalse("an accepted configuration is not saved while its route is built",
+ exportConfigurations.contains("in-bounds"));
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Fixtures
+ // ---------------------------------------------------------------------------------------------
+
+ private String fileUri(File directory, String suffix) {
+ return "file://" + directory.getAbsolutePath() + suffix;
+ }
+
+ private ImportConfiguration inBoundsImport(String itemId) {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setItemId(itemId);
+ return configuration;
+ }
+
+ private ImportConfiguration recurrentImport(String source) {
+ ImportConfiguration configuration = new ImportConfiguration();
+ configuration.setItemId("out-of-bounds");
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setActive(true);
+ configuration.getProperties().put("source", source);
+ configuration.getProperties().put("mapping", Collections.singletonMap("0", 0));
+ return configuration;
+ }
+
+ private ExportConfiguration recurrentExport(String destination) {
+ ExportConfiguration configuration = new ExportConfiguration();
+ configuration.setItemId("out-of-bounds");
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setActive(true);
+ configuration.getProperties().put("destination", destination);
+ configuration.getProperties().put("mapping", Collections.singletonMap("0", "firstName"));
+ configuration.getProperties().put("segment", "exportSegment");
+ configuration.getProperties().put("period", "1m");
+ return configuration;
+ }
+
+ private void addImportRoutes(ImportConfiguration... configurations) throws Exception {
+ ProfileImportFromSourceRouteBuilder builder =
+ new ProfileImportFromSourceRouteBuilder(NO_KAFKA, RouterConstants.CONFIG_TYPE_NOBROKER);
+ builder.setImportConfigurationList(java.util.Arrays.asList(configurations));
+ builder.setImportConfigurationService(importConfigurations);
+ builder.setProfileService(noOpProfileService());
+ builder.setJacksonDataFormat(new JacksonDataFormat(ProfileToImport.class));
+ builder.setAllowedEndpoints(DEFAULT_ALLOWED_ENDPOINTS);
+ builder.setPermittedImportBaseDirs(importRoot.getAbsolutePath());
+ builder.setContext(camelContext);
+ camelContext.addRoutes(builder);
+ }
+
+ private void addExportRoutes(ExportConfiguration... configurations) throws Exception {
+ ProfileExportCollectRouteBuilder builder =
+ new ProfileExportCollectRouteBuilder(NO_KAFKA, RouterConstants.CONFIG_TYPE_NOBROKER);
+ builder.setExportConfigurationList(java.util.Arrays.asList(configurations));
+ builder.setExportConfigurationService(exportConfigurations);
+ builder.setJacksonDataFormat(new JacksonDataFormat(ProfileToImport.class));
+ builder.setAllowedEndpoints(DEFAULT_ALLOWED_ENDPOINTS);
+ builder.setPermittedExportBaseDirs(exportRoot.getAbsolutePath());
+ builder.setContext(camelContext);
+ camelContext.addRoutes(builder);
+ }
+
+ private static ProfileService noOpProfileService() {
+ return (ProfileService) Proxy.newProxyInstance(
+ ProfileService.class.getClassLoader(),
+ new Class<?>[]{ProfileService.class},
+ (proxy, method, args) -> java.util.Collection.class.isAssignableFrom(method.getReturnType())
+ ? Collections.emptyList() : null);
+ }
+
+ /**
+ * Stores what it is given, and remembers whether the last save asked for the running route to be
+ * refreshed — a refused configuration must not, or the refresh loops.
+ */
+ private static final class RecordingConfigurationService<T> implements ImportExportConfigurationService<T> {
+
+ private final Map<String, T> stored = new LinkedHashMap<>();
+
+ private boolean lastSaveAskedForARouteRefresh;
+
+ /** Stands in for a store that cannot be written to -- Elasticsearch unreachable at start-up. */
+ private boolean unwritable;
+
+ boolean contains(String configId) {
+ return stored.containsKey(configId);
+ }
+
+ @Override
+ public List<T> getAll() {
+ return new ArrayList<>(stored.values());
+ }
+
+ @Override
+ public T load(String configId) {
+ return stored.get(configId);
+ }
+
+ @Override
+ public T save(T configuration, boolean updateRunningRoute) {
+ if (unwritable) {
+ throw new IllegalStateException("the store is unreachable");
+ }
+ lastSaveAskedForARouteRefresh = updateRunningRoute;
+ stored.put(itemIdOf(configuration), configuration);
+ return configuration;
+ }
+
+ @Override
+ public void delete(String configId) {
+ stored.remove(configId);
+ }
+
+ @Override
+ public Map<String, Map<String, RouterConstants.CONFIG_CAMEL_REFRESH>> consumeConfigsToBeRefresh() {
+ return Collections.emptyMap();
+ }
+
+ @Override
+ public void requeueForRefresh(String tenantId, String configId, RouterConstants.CONFIG_CAMEL_REFRESH refreshType) {
+ }
+
+ private String itemIdOf(T configuration) {
+ return configuration instanceof ImportConfiguration
+ ? ((ImportConfiguration) configuration).getItemId()
+ : ((ExportConfiguration) configuration).getItemId();
+ }
+ }
+}
diff --git a/extensions/router/router-rest/pom.xml b/extensions/router/router-rest/pom.xml
index ef2c4b848..5d9904848 100644
--- a/extensions/router/router-rest/pom.xml
+++ b/extensions/router/router-rest/pom.xml
@@ -118,6 +118,17 @@
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>junit</groupId>
+ <artifactId>junit</artifactId>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.junit.vintage</groupId>
+ <artifactId>junit-vintage-engine</artifactId>
+ <version>${junit-jupiter.version}</version>
+ <scope>test</scope>
+ </dependency>
</dependencies>
<build>
diff --git a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/AbstractConfigurationServiceEndpoint.java b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/AbstractConfigurationServiceEndpoint.java
index df966f634..172e33f44 100644
--- a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/AbstractConfigurationServiceEndpoint.java
+++ b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/AbstractConfigurationServiceEndpoint.java
@@ -16,10 +16,16 @@
*/
package org.apache.unomi.router.rest;
+import org.apache.unomi.api.ExecutionContext;
+import org.apache.unomi.api.services.ConfigSharingService;
+import org.apache.unomi.api.services.ExecutionContextManager;
+import org.apache.unomi.router.api.EndpointValidator;
+import org.apache.unomi.router.api.RouterConstants;
import org.apache.unomi.router.api.services.ImportExportConfigurationService;
import javax.ws.rs.*;
import javax.ws.rs.core.MediaType;
+import javax.ws.rs.core.Response;
import java.util.List;
/**
@@ -31,6 +37,67 @@ public abstract class AbstractConfigurationServiceEndpoint<T> {
protected ImportExportConfigurationService<T> configurationService;
+ protected ConfigSharingService configSharingService;
+
+ protected ExecutionContextManager executionContextManager;
+
+ /**
+ * Refuses the configuration when the endpoint it names cannot be honoured -- an unsupported scheme,
+ * or a file path outside the directories the deployment permits.
+ *
+ * <p>The route that would carry the configuration is built asynchronously, long after this call has
+ * answered, so a configuration refused there would be stored and answered {@code 200} with nothing
+ * but a log line to show for it. Refusing here gives the caller the reason while it can still act
+ * on it, and keeps the configuration out of the store.
+ *
+ * <p>Answers {@code 503} instead while the router has yet to publish its settings, since a
+ * configuration cannot be judged against settings that are not there yet.
+ *
+ * @param endpointUri the endpoint URI the configuration names
+ * @param permittedBaseDirsProperty the shared property holding the base directories for this direction
+ * @param tenantId the tenant the configuration will be stored for
+ */
+ protected void refuseIfEndpointCannotBeHonoured(String endpointUri, String permittedBaseDirsProperty, String tenantId) {
+ String allowedSchemes = (String) configSharingService.getProperty(RouterConstants.CONFIG_ALLOWED_ENDPOINTS);
+ String permittedBaseDirs = (String) configSharingService.getProperty(permittedBaseDirsProperty);
+ if (allowedSchemes == null || permittedBaseDirs == null) {
+ // The router's Camel context publishes both on start-up, and this endpoint answers before
+ // it has. An absent setting is not an empty allow-list: reading it as one would refuse a
+ // legitimate configuration, and blame its scheme for it. Say the truth instead -- there is
+ // nothing to validate against yet -- so the caller can retry rather than correct a
+ // configuration that is already right.
+ String unavailable = "the router is still starting up: no endpoint can be validated yet";
+ throw new ServiceUnavailableException(unavailable,
+ Response.status(Response.Status.SERVICE_UNAVAILABLE)
+ .type(MediaType.TEXT_PLAIN).entity(unavailable).build());
+ }
+
+ String refusal = EndpointValidator.validateForTenant(endpointUri, allowedSchemes, permittedBaseDirs, tenantId);
+ if (refusal != null) {
+ throw new BadRequestException(refusal, Response.status(Response.Status.BAD_REQUEST)
+ .type(MediaType.TEXT_PLAIN).entity(refusal).build());
+ }
+ }
+
+ /**
+ * The tenant the file endpoint is confined to.
+ *
+ * <p>A caller who is not the system is confined to their own tenant, whatever tenant the body names:
+ * the configuration service will reject a different one, and judging the path against the claimed
+ * tenant would let the check pass for a directory the caller does not own. A system caller may name
+ * the tenant. When the body names none, the current context supplies it.
+ */
+ protected String tenantToConfine(String configuredTenantId) {
+ ExecutionContext context = executionContextManager == null ? null : executionContextManager.getCurrentContext();
+ if (context != null && !context.isSystem()) {
+ return context.getTenantId();
+ }
+ if (configuredTenantId != null && !configuredTenantId.trim().isEmpty()) {
+ return configuredTenantId;
+ }
+ return context == null ? null : context.getTenantId();
+ }
+
/**
* Returns all router configurations of this type.
*
diff --git a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ExportConfigurationServiceEndPoint.java b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ExportConfigurationServiceEndPoint.java
index a77325160..01e909fba 100644
--- a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ExportConfigurationServiceEndPoint.java
+++ b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ExportConfigurationServiceEndPoint.java
@@ -18,9 +18,12 @@ package org.apache.unomi.router.rest;
import org.apache.cxf.rs.security.cors.CrossOriginResourceSharing;
import org.apache.unomi.api.security.UnomiRoles;
+import org.apache.unomi.api.services.ConfigSharingService;
+import org.apache.unomi.api.services.ExecutionContextManager;
import org.apache.unomi.api.services.ProfileService;
import org.apache.unomi.rest.security.RequiresRole;
import org.apache.unomi.router.api.ExportConfiguration;
+import org.apache.unomi.router.api.RouterConstants;
import org.apache.unomi.router.api.services.ImportExportConfigurationService;
import org.apache.unomi.router.api.services.ProfileExportService;
import org.osgi.service.component.annotations.Component;
@@ -70,6 +73,16 @@ public class ExportConfigurationServiceEndPoint extends AbstractConfigurationSer
configurationService = exportConfigurationService;
}
+ @Reference
+ public void setConfigSharingService(ConfigSharingService configSharingService) {
+ this.configSharingService = configSharingService;
+ }
+
+ @Reference
+ public void setExecutionContextManager(ExecutionContextManager executionContextManager) {
+ this.executionContextManager = executionContextManager;
+ }
+
public void setProfileExportService(ProfileExportService profileExportService) {
this.profileExportService = profileExportService;
}
@@ -113,9 +126,12 @@ public class ExportConfigurationServiceEndPoint extends AbstractConfigurationSer
*/
@Override
public ExportConfiguration saveConfiguration(ExportConfiguration exportConfiguration) {
- ExportConfiguration exportConfigSaved = configurationService.save(exportConfiguration, true);
+ if (RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT.equals(exportConfiguration.getConfigType())) {
+ refuseIfEndpointCannotBeHonoured((String) exportConfiguration.getProperties().get("destination"),
+ RouterConstants.CONFIG_EXPORT_BASE_DIRS, tenantToConfine(exportConfiguration.getTenantId()));
+ }
- return exportConfigSaved;
+ return configurationService.save(exportConfiguration, true);
}
/**
diff --git a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java
index 8c7e4b1ef..72c6515ce 100644
--- a/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java
+++ b/extensions/router/router-rest/src/main/java/org/apache/unomi/router/rest/ImportConfigurationServiceEndPoint.java
@@ -65,15 +65,11 @@ public class ImportConfigurationServiceEndPoint extends AbstractConfigurationSer
private static final Logger LOGGER = LoggerFactory.getLogger(ImportConfigurationServiceEndPoint.class.getName());
@Reference
- protected ConfigSharingService configSharingService;
-
- @Reference
- protected ExecutionContextManager executionContextManager;
-
public void setConfigSharingService(ConfigSharingService configSharingService) {
this.configSharingService = configSharingService;
}
+ @Reference
public void setExecutionContextManager(ExecutionContextManager executionContextManager) {
this.executionContextManager = executionContextManager;
}
@@ -122,10 +118,12 @@ public class ImportConfigurationServiceEndPoint extends AbstractConfigurationSer
*/
@Override
public ImportConfiguration saveConfiguration(ImportConfiguration importConfiguration) {
+ if (RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT.equals(importConfiguration.getConfigType())) {
+ refuseIfEndpointCannotBeHonoured((String) importConfiguration.getProperties().get("source"),
+ RouterConstants.CONFIG_IMPORT_BASE_DIRS, tenantToConfine(importConfiguration.getTenantId()));
+ }
- ImportConfiguration importConfigSaved = configurationService.save(importConfiguration, true);
-
- return importConfigSaved;
+ return configurationService.save(importConfiguration, true);
}
/**
diff --git a/extensions/router/router-rest/src/test/java/org/apache/unomi/router/rest/ConfigurationEndpointValidationTest.java b/extensions/router/router-rest/src/test/java/org/apache/unomi/router/rest/ConfigurationEndpointValidationTest.java
new file mode 100644
index 000000000..812bbb1c2
--- /dev/null
+++ b/extensions/router/router-rest/src/test/java/org/apache/unomi/router/rest/ConfigurationEndpointValidationTest.java
@@ -0,0 +1,385 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.router.rest;
+
+import org.apache.unomi.api.ExecutionContext;
+import org.apache.unomi.api.services.ConfigSharingService;
+import org.apache.unomi.api.services.ExecutionContextManager;
+import org.apache.unomi.router.api.ExportConfiguration;
+import org.apache.unomi.router.api.ImportConfiguration;
+import org.apache.unomi.router.api.RouterConstants;
+import org.apache.unomi.router.api.services.ImportExportConfigurationService;
+import org.junit.Before;
+import org.junit.Rule;
+import org.junit.Test;
+import org.junit.rules.TemporaryFolder;
+
+import javax.ws.rs.WebApplicationException;
+import java.io.File;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.Assert.assertFalse;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+import static org.junit.Assert.assertTrue;
+import static org.junit.Assert.fail;
+
+/**
+ * A configuration whose endpoint cannot be honoured must be refused when it is saved, not silently
+ * accepted and then dropped when its route fails to build.
+ *
+ * <p>Route construction happens asynchronously, well after the REST call has answered, so a
+ * configuration that only fails there is stored, answered {@code 200}, and leaves nothing but a log
+ * line behind — the caller cannot tell it apart from a configuration that works. Validating at save
+ * time gives the caller a synchronous, actionable answer, and keeps the rejected configuration out
+ * of the store.
+ *
+ * <p>Only configurations that name an endpoint are concerned: a oneshot import carries no source, its
+ * file being uploaded separately, and must keep being saved.
+ */
+public class ConfigurationEndpointValidationTest {
+
+ @Rule
+ public TemporaryFolder tmp = new TemporaryFolder();
+
+ private static final String TENANT = "acme";
+
+ private File importRoot;
+ private File exportRoot;
+ private File permittedImportDir;
+ private File permittedExportDir;
+ private File arbitraryDir;
+
+ private InMemoryConfigurationService<ImportConfiguration> importConfigurations;
+ private InMemoryConfigurationService<ExportConfiguration> exportConfigurations;
+
+ private ImportConfigurationServiceEndPoint importEndpoint;
+ private ExportConfigurationServiceEndPoint exportEndpoint;
+
+ @Before
+ public void setUp() throws Exception {
+ importRoot = tmp.newFolder("permitted-import");
+ permittedImportDir = new File(importRoot, TENANT);
+ assertTrue(permittedImportDir.mkdirs());
+ exportRoot = tmp.newFolder("permitted-export");
+ permittedExportDir = new File(exportRoot, TENANT);
+ assertTrue(permittedExportDir.mkdirs());
+ arbitraryDir = tmp.newFolder("arbitrary");
+
+ InMemoryConfigSharingService configSharingService = new InMemoryConfigSharingService();
+ configSharingService.setProperty(RouterConstants.CONFIG_ALLOWED_ENDPOINTS, "file,ftp,sftp,ftps");
+ configSharingService.setProperty(RouterConstants.CONFIG_IMPORT_BASE_DIRS, importRoot.getAbsolutePath());
+ configSharingService.setProperty(RouterConstants.CONFIG_EXPORT_BASE_DIRS, exportRoot.getAbsolutePath());
+
+ importConfigurations = new InMemoryConfigurationService<>();
+ importEndpoint = new ImportConfigurationServiceEndPoint();
+ importEndpoint.setImportConfigurationService(importConfigurations);
+ importEndpoint.setConfigSharingService(configSharingService);
+
+ exportConfigurations = new InMemoryConfigurationService<>();
+ exportEndpoint = new ExportConfigurationServiceEndPoint();
+ exportEndpoint.setExportConfigurationService(exportConfigurations);
+ exportEndpoint.setConfigSharingService(configSharingService);
+ }
+
+ @Test
+ public void savingARecurrentImportWhoseSourceIsInsideThePermittedBaseDirsStoresIt() {
+ ImportConfiguration saved = importEndpoint.saveConfiguration(
+ recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv")));
+
+ assertEquals("in-bounds", saved.getItemId());
+ assertTrue("the configuration should have been stored", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportBeforeTheRouterPublishedItsSettingsIsNotRefused() throws Exception {
+ // the router's Camel context has not started yet, so it has published nothing
+ ImportConfigurationServiceEndPoint starting = new ImportConfigurationServiceEndPoint();
+ starting.setImportConfigurationService(importConfigurations);
+ starting.setConfigSharingService(new InMemoryConfigSharingService());
+
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+
+ assertUnavailable(() -> starting.saveConfiguration(configuration));
+ assertFalse("nothing is stored while the endpoint cannot be judged", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportWhoseRemoteSourceStagesDownloadsOutsideThePermittedBaseDirsIsRefused() {
+ // ftp is an allowed scheme, but localWorkDirectory names a local directory all the same
+ ImportConfiguration configuration = recurrentImport("ftp://ftp.example.com/profiles"
+ + "?fileName=profiles.csv&localWorkDirectory=" + arbitraryDir.getAbsolutePath());
+
+ assertRefused(() -> importEndpoint.saveConfiguration(configuration));
+ assertFalse("a refused configuration must not be stored", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportBeforeThePermittedDirectoriesArePublishedIsNotRefused() throws Exception {
+ // the scheme allow-list is there, the directories are not: still nothing to judge against
+ InMemoryConfigSharingService partial = new InMemoryConfigSharingService();
+ partial.setProperty(RouterConstants.CONFIG_ALLOWED_ENDPOINTS, "file,ftp,sftp,ftps");
+ ImportConfigurationServiceEndPoint starting = new ImportConfigurationServiceEndPoint();
+ starting.setImportConfigurationService(importConfigurations);
+ starting.setConfigSharingService(partial);
+
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+
+ assertUnavailable(() -> starting.saveConfiguration(configuration));
+ }
+
+ @Test
+ public void savingARecurrentExportBeforeTheRouterPublishedItsSettingsIsNotRefused() throws Exception {
+ ExportConfigurationServiceEndPoint starting = new ExportConfigurationServiceEndPoint();
+ starting.setExportConfigurationService(exportConfigurations);
+ starting.setConfigSharingService(new InMemoryConfigSharingService());
+
+ ExportConfiguration configuration = recurrentExport(fileUri(permittedExportDir, "?fileName=profiles.csv"));
+
+ assertUnavailable(() -> starting.saveConfiguration(configuration));
+ assertFalse("nothing is stored while the endpoint cannot be judged", exportConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportWhoseSourceIsOutsideThePermittedBaseDirsIsRefused() {
+ ImportConfiguration configuration = recurrentImport(fileUri(arbitraryDir, "?fileName=profiles.csv"));
+
+ assertRefused(() -> importEndpoint.saveConfiguration(configuration));
+ assertFalse("a refused configuration must not be stored", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportWhoseFileNameOptionEscapesThePermittedBaseDirsIsRefused() {
+ ImportConfiguration configuration = recurrentImport(
+ fileUri(permittedImportDir, "?fileName=../" + arbitraryDir.getName() + "/profiles.csv"));
+
+ assertRefused(() -> importEndpoint.saveConfiguration(configuration));
+ }
+
+ @Test
+ public void savingARecurrentImportWhoseSourceIsNotAUsablePathIsRefused() {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles%00.csv"));
+
+ assertRefused(() -> importEndpoint.saveConfiguration(configuration));
+ assertFalse("a refused configuration must not be stored", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportInAnotherTenantsDirectoryIsRefused() {
+ File otherTenant = new File(importRoot, "beta");
+ assertTrue(otherTenant.mkdirs());
+
+ assertRefused(() -> importEndpoint.saveConfiguration(
+ recurrentImport(fileUri(otherTenant, "?fileName=profiles.csv"))));
+ assertFalse("a refused configuration must not be stored", importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportWithNoTenantIsRefused() {
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setTenantId(null);
+
+ assertRefused(() -> importEndpoint.saveConfiguration(configuration));
+ assertFalse("a configuration with no tenant must not fall back to the shared directory",
+ importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentImportUsesTheCallerTenantWhenTheBodyNamesNone() {
+ ExecutionContextManager contexts = mock(ExecutionContextManager.class);
+ when(contexts.getCurrentContext()).thenReturn(new ExecutionContext(TENANT, null, null));
+ importEndpoint.setExecutionContextManager(contexts);
+
+ ImportConfiguration configuration = recurrentImport(fileUri(permittedImportDir, "?fileName=profiles.csv"));
+ configuration.setTenantId(null);
+
+ ImportConfiguration saved = importEndpoint.saveConfiguration(configuration);
+
+ assertEquals("in-bounds", saved.getItemId());
+ assertTrue(importConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingAOneshotImportThatCarriesNoSourceStoresIt() {
+ ImportConfiguration configuration = new ImportConfiguration();
+ configuration.setItemId("oneshot");
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_ONESHOT);
+ configuration.getProperties().put("mapping", Collections.singletonMap("email", 0));
+
+ importEndpoint.saveConfiguration(configuration);
+
+ assertTrue("a oneshot import names no endpoint and must keep being stored",
+ importConfigurations.contains("oneshot"));
+ }
+
+ @Test
+ public void savingARecurrentExportWhoseDestinationIsInsideThePermittedBaseDirsStoresIt() {
+ ExportConfiguration saved = exportEndpoint.saveConfiguration(
+ recurrentExport(fileUri(permittedExportDir, "?fileName=profiles.csv")));
+
+ assertEquals("in-bounds", saved.getItemId());
+ assertTrue("the configuration should have been stored", exportConfigurations.contains("in-bounds"));
+ }
+
+ @Test
+ public void savingARecurrentExportWhoseDestinationIsOutsideThePermittedBaseDirsIsRefused() {
+ ExportConfiguration configuration = recurrentExport(fileUri(arbitraryDir, "?fileName=profiles.csv"));
+
+ assertRefused(() -> exportEndpoint.saveConfiguration(configuration));
+ assertFalse("a refused configuration must not be stored", exportConfigurations.contains("in-bounds"));
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Fixtures
+ // ---------------------------------------------------------------------------------------------
+
+ /**
+ * A refused configuration answers {@code 400 Bad Request}, and says why: the caller has to be able
+ * to correct the endpoint from the answer alone.
+ */
+ private void assertUnavailable(Runnable save) {
+ try {
+ save.run();
+ fail("saving the configuration should not have been answered yet");
+ } catch (WebApplicationException e) {
+ assertEquals("settings that are not published yet make the service unavailable, not the "
+ + "configuration wrong", 503, e.getResponse().getStatus());
+ }
+ }
+
+ private void assertRefused(Runnable save) {
+ try {
+ save.run();
+ fail("saving the configuration should have been refused");
+ } catch (WebApplicationException e) {
+ assertEquals("a refused configuration is a bad request", 400, e.getResponse().getStatus());
+ assertTrue("the refusal must say why", e.getMessage() != null && !e.getMessage().trim().isEmpty());
+ }
+ }
+
+ private String fileUri(File directory, String suffix) {
+ return "file://" + directory.getAbsolutePath() + suffix;
+ }
+
+ private ImportConfiguration recurrentImport(String source) {
+ ImportConfiguration configuration = new ImportConfiguration();
+ configuration.setItemId("in-bounds");
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.getProperties().put("source", source);
+ configuration.getProperties().put("mapping", Collections.singletonMap("email", 0));
+ return configuration;
+ }
+
+ private ExportConfiguration recurrentExport(String destination) {
+ ExportConfiguration configuration = new ExportConfiguration();
+ configuration.setItemId("in-bounds");
+ configuration.setTenantId(TENANT);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.getProperties().put("destination", destination);
+ configuration.getProperties().put("mapping", Collections.singletonMap("0", "firstName"));
+ configuration.getProperties().put("segment", "exportSegment");
+ configuration.getProperties().put("period", "1m");
+ return configuration;
+ }
+
+ /**
+ * Stores what it is given, so that a test can tell a configuration that was persisted from one that
+ * was refused before reaching the store.
+ */
+ private static final class InMemoryConfigurationService<T> implements ImportExportConfigurationService<T> {
+
+ private final Map<String, T> stored = new LinkedHashMap<>();
+
+ boolean contains(String configId) {
+ return stored.containsKey(configId);
+ }
+
+ @Override
+ public List<T> getAll() {
+ return new ArrayList<>(stored.values());
+ }
+
+ @Override
+ public T load(String configId) {
+ return stored.get(configId);
+ }
+
+ @Override
+ public T save(T configuration, boolean updateRunningRoute) {
+ stored.put(itemIdOf(configuration), configuration);
+ return configuration;
+ }
+
+ @Override
+ public void delete(String configId) {
+ stored.remove(configId);
+ }
+
+ @Override
+ public Map<String, Map<String, RouterConstants.CONFIG_CAMEL_REFRESH>> consumeConfigsToBeRefresh() {
+ return Collections.emptyMap();
+ }
+
+ @Override
+ public void requeueForRefresh(String tenantId, String configId, RouterConstants.CONFIG_CAMEL_REFRESH refreshType) {
+ }
+
+ private String itemIdOf(T configuration) {
+ return configuration instanceof ImportConfiguration
+ ? ((ImportConfiguration) configuration).getItemId()
+ : ((ExportConfiguration) configuration).getItemId();
+ }
+ }
+
+ private static final class InMemoryConfigSharingService implements ConfigSharingService {
+
+ private final Map<String, Object> properties = new HashMap<>();
+
+ @Override
+ public Object getProperty(String name) {
+ return properties.get(name);
+ }
+
+ @Override
+ public Object setProperty(String name, Object value) {
+ return properties.put(name, value);
+ }
+
+ @Override
+ public boolean hasProperty(String name) {
+ return properties.containsKey(name);
+ }
+
+ @Override
+ public Object removeProperty(String name) {
+ return properties.remove(name);
+ }
+
+ @Override
+ public Set<String> getPropertyNames() {
+ return properties.keySet();
+ }
+ }
+}
diff --git a/itests/src/test/java/org/apache/unomi/itests/AllITs.java b/itests/src/test/java/org/apache/unomi/itests/AllITs.java
index 552ad1599..61e39077e 100644
--- a/itests/src/test/java/org/apache/unomi/itests/AllITs.java
+++ b/itests/src/test/java/org/apache/unomi/itests/AllITs.java
@@ -46,6 +46,7 @@ import org.junit.runners.Suite.SuiteClasses;
ProfileImportRankingIT.class,
ProfileImportActorsIT.class,
ProfileExportIT.class,
+ ProfileImportExportContainmentIT.class,
ProfileMergeIT.class,
EventServiceIT.class,
PropertiesUpdateActionIT.class,
diff --git a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
index f699cf9c9..05d784176 100644
--- a/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/BaseIT.java
@@ -651,11 +651,11 @@ public abstract class BaseIT extends KarafTestSupport {
Option[] options = new Option[]{
replaceConfigurationFile("etc/org.apache.unomi.router.cfg", resolveTestResource("src/test/resources/org.apache.unomi.router.cfg")),
replaceConfigurationFile("data/tmp/1-basic-test.csv", resolveTestResource("src/test/resources/1-basic-test.csv")),
- replaceConfigurationFile("data/tmp/recurrent_import/2-surfers-test.csv", resolveTestResource("src/test/resources/2-surfers-test.csv")),
- replaceConfigurationFile("data/tmp/recurrent_import/3-surfers-overwrite-test.csv", resolveTestResource("src/test/resources/3-surfers-overwrite-test.csv")),
- replaceConfigurationFile("data/tmp/recurrent_import/4-surfers-delete-test.csv", resolveTestResource("src/test/resources/4-surfers-delete-test.csv")),
- replaceConfigurationFile("data/tmp/recurrent_import/5-ranking-test.csv", resolveTestResource("src/test/resources/5-ranking-test.csv")),
- replaceConfigurationFile("data/tmp/recurrent_import/6-actors-test.csv", resolveTestResource("src/test/resources/6-actors-test.csv")),
+ replaceConfigurationFile("data/tmp/recurrent_import/" + TEST_TENANT_ID + "/2-surfers-test.csv", resolveTestResource("src/test/resources/2-surfers-test.csv")),
+ replaceConfigurationFile("data/tmp/recurrent_import/" + TEST_TENANT_ID + "/3-surfers-overwrite-test.csv", resolveTestResource("src/test/resources/3-surfers-overwrite-test.csv")),
+ replaceConfigurationFile("data/tmp/recurrent_import/" + TEST_TENANT_ID + "/4-surfers-delete-test.csv", resolveTestResource("src/test/resources/4-surfers-delete-test.csv")),
+ replaceConfigurationFile("data/tmp/recurrent_import/" + TEST_TENANT_ID + "/5-ranking-test.csv", resolveTestResource("src/test/resources/5-ranking-test.csv")),
+ replaceConfigurationFile("data/tmp/recurrent_import/" + TEST_TENANT_ID + "/6-actors-test.csv", resolveTestResource("src/test/resources/6-actors-test.csv")),
replaceConfigurationFile("data/tmp/testLogin.json", resolveTestResource("src/test/resources/testLogin.json")),
replaceConfigurationFile("data/tmp/testCopyProperties.json", resolveTestResource("src/test/resources/testCopyProperties.json")),
replaceConfigurationFile("data/tmp/testCopyPropertiesWithoutSystemTags.json", resolveTestResource("src/test/resources/testCopyPropertiesWithoutSystemTags.json")),
@@ -685,6 +685,14 @@ public abstract class BaseIT extends KarafTestSupport {
// CAS" every checker tick. Widen it for ITs so heartbeats/checker never starve.
editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.scheduler.thread.poolSize", "10"),
+ // The router's base directories have to be set here rather than in
+ // etc/org.apache.unomi.router.cfg: Karaf's configuration plugin overrides every
+ // configuration property whose <pid>.<key> exists as a system property, and
+ // custom.system.properties declares both of these, so a value written to the cfg
+ // never reaches the router.
+ editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.router.config.import.baseDir", "${karaf.data}/tmp/recurrent_import"),
+ editConfigurationFilePut("etc/custom.system.properties", "org.apache.unomi.router.config.export.baseDir", "${karaf.data}/tmp/recurrent_export"),
+
systemProperty("org.ops4j.pax.exam.rbc.rmi.port").value("1199"),
systemProperty("org.apache.unomi.healthcheck.enabled").value("true"),
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileExportIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileExportIT.java
index 8a9c58d67..a497b63ee 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileExportIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileExportIT.java
@@ -99,13 +99,13 @@ public class ProfileExportIT extends BaseIT {
exportConfiguration.getProperties().put("mapping", mapping);
exportConfiguration.getProperties().put("segment", "exportItSeg");
exportConfiguration.getProperties().put("period", "1m");
- File exportDir = new File("data/tmp/");
+ File exportDir = new File("data/tmp/recurrent_export/" + TEST_TENANT_ID);
exportConfiguration.getProperties().put("destination", "file://" + exportDir.getAbsolutePath() + "?fileName=profiles-export.csv");
exportConfiguration.setActive(true);
exportConfigurationService.save(exportConfiguration, true);
- final File exportResult = new File("data/tmp/profiles-export.csv");
+ final File exportResult = new File("data/tmp/recurrent_export/" + TEST_TENANT_ID + "/profiles-export.csv");
keepTrying("Failed waiting for export file to be created", () -> exportResult, File::exists, 1000, 100);
LOGGER.info("PATH : {}", exportResult.getAbsolutePath());
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileImportActorsIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileImportActorsIT.java
index 5ae6f595c..9e31c291a 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileImportActorsIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileImportActorsIT.java
@@ -86,7 +86,7 @@ public class ProfileImportActorsIT extends BaseIT {
mappingActors.put("city", 4);
importConfigActors.getProperties().put("mapping", mappingActors);
- File importSurfersFile = new File("data/tmp/recurrent_import/");
+ File importSurfersFile = new File("data/tmp/recurrent_import/" + TEST_TENANT_ID);
importConfigActors.getProperties().put("source",
"file://" + importSurfersFile.getAbsolutePath() + "?fileName=6-actors-test.csv&move=.done");
importConfigActors.setActive(true);
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileImportExportContainmentIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileImportExportContainmentIT.java
new file mode 100644
index 000000000..a15417f94
--- /dev/null
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileImportExportContainmentIT.java
@@ -0,0 +1,260 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.unomi.itests;
+
+import org.apache.http.client.methods.CloseableHttpResponse;
+import org.apache.http.client.methods.HttpPost;
+import org.apache.http.entity.ContentType;
+import org.apache.http.entity.StringEntity;
+import org.apache.http.util.EntityUtils;
+import org.apache.unomi.router.api.ExportConfiguration;
+import org.apache.unomi.router.api.ImportConfiguration;
+import org.apache.unomi.router.api.RouterConstants;
+import org.junit.After;
+import org.junit.Assert;
+import org.junit.Test;
+import org.junit.runner.RunWith;
+import org.ops4j.pax.exam.junit.PaxExam;
+import org.ops4j.pax.exam.spi.reactors.ExamReactorStrategy;
+import org.ops4j.pax.exam.spi.reactors.PerSuite;
+
+import java.io.File;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.util.HashMap;
+import java.util.Map;
+
+/**
+ * A recurrent import or export configuration using the {@code file} scheme must resolve inside the base
+ * directory the deployment permits — {@code config.import.baseDir} and {@code config.export.baseDir},
+ * set for these tests in {@code etc/custom.system.properties} by {@link BaseIT}.
+ *
+ * <p>The unit tests decide the containment rules. What only a running Unomi can show is that the
+ * settings actually reach the two places that need them: the REST layer, which refuses a configuration
+ * as it is saved, and the route builders, which refuse one that is already stored. Those two read the
+ * setting through different paths, and neither is exercised by a unit test.
+ */
+@RunWith(PaxExam.class)
+@ExamReactorStrategy(PerSuite.class)
+public class ProfileImportExportContainmentIT extends BaseIT {
+
+ private static final String IMPORT_CONFIGURATION_URL = "/cxs/importConfiguration";
+ private static final String EXPORT_CONFIGURATION_URL = "/cxs/exportConfiguration";
+
+ /** The deployment's base directories. A file endpoint must sit under the test tenant's subdirectory. */
+ private static final String PERMITTED_IMPORT_DIR = "data/tmp/recurrent_import/" + TEST_TENANT_ID;
+ private static final String PERMITTED_EXPORT_DIR = "data/tmp/recurrent_export/" + TEST_TENANT_ID;
+
+ /** Not permitted by anything: a sibling of the two above. */
+ private static final String ARBITRARY_DIR = "data/tmp/it-containment-arbitrary";
+
+ private String createdImportConfigId;
+ private String createdExportConfigId;
+
+ @After
+ public void cleanup() {
+ if (createdImportConfigId != null) {
+ importConfigurationService.delete(createdImportConfigId);
+ createdImportConfigId = null;
+ }
+ if (createdExportConfigId != null) {
+ exportConfigurationService.delete(createdExportConfigId);
+ createdExportConfigId = null;
+ }
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Refused as it is saved
+ // ---------------------------------------------------------------------------------------------
+
+ @Test
+ public void savingAnImportConfigurationOutsideThePermittedDirectoryIsRefused() throws Exception {
+ ImportConfiguration configuration = recurrentImport("it-import-refused",
+ fileUri(ARBITRARY_DIR, "?fileName=containment-it.csv"));
+
+ Response response = postJson(IMPORT_CONFIGURATION_URL, configuration);
+
+ Assert.assertEquals("a configuration whose source cannot be honoured is a bad request",
+ 400, response.status);
+ Assert.assertFalse("the refusal must say why, so the caller can correct it",
+ response.body.trim().isEmpty());
+ Assert.assertNull("a refused configuration must not be stored",
+ importConfigurationService.load("it-import-refused"));
+ }
+
+ @Test
+ public void savingAnExportConfigurationOutsideThePermittedDirectoryIsRefused() throws Exception {
+ ExportConfiguration configuration = recurrentExport("it-export-refused",
+ fileUri(ARBITRARY_DIR, "?fileName=containment-it.csv"));
+
+ Response response = postJson(EXPORT_CONFIGURATION_URL, configuration);
+
+ Assert.assertEquals("a configuration whose destination cannot be honoured is a bad request",
+ 400, response.status);
+ Assert.assertFalse("the refusal must say why, so the caller can correct it",
+ response.body.trim().isEmpty());
+ Assert.assertNull("a refused configuration must not be stored",
+ exportConfigurationService.load("it-export-refused"));
+ }
+
+ @Test
+ public void savingAnImportConfigurationInsideThePermittedDirectoryIsAccepted() throws Exception {
+ createdImportConfigId = "it-import-accepted";
+ ImportConfiguration configuration = recurrentImport(createdImportConfigId,
+ fileUri(PERMITTED_IMPORT_DIR, "?fileName=containment-it.csv&consumer.delay=10m"));
+
+ Response response = postJson(IMPORT_CONFIGURATION_URL, configuration);
+
+ Assert.assertEquals("a configuration inside the permitted directory is legitimate",
+ 200, response.status);
+ keepTrying("the accepted configuration should have been stored",
+ () -> importConfigurationService.load(createdImportConfigId), c -> c != null, 1000, 20);
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Already stored, refused when its route is built
+ // ---------------------------------------------------------------------------------------------
+
+ @Test
+ public void aStoredImportConfigurationOutsideThePermittedDirectoryIsMarkedAndConsumesNothing() throws Exception {
+ File arbitraryDir = new File(ARBITRARY_DIR);
+ Assert.assertTrue("could not prepare the test fixture", arbitraryDir.exists() || arbitraryDir.mkdirs());
+ File waiting = new File(arbitraryDir, "must-not-be-consumed.csv");
+ Files.write(waiting.toPath(), "email,firstName\nnobody@example.com,Nobody\n".getBytes(StandardCharsets.UTF_8));
+
+ createdImportConfigId = "it-import-stored-out-of-bounds";
+ ImportConfiguration configuration = recurrentImport(createdImportConfigId,
+ fileUri(ARBITRARY_DIR, "?fileName=must-not-be-consumed.csv"));
+
+ // bypasses the REST layer on purpose: this is the configuration that was already there when the
+ // deployment's permitted directories changed
+ importConfigurationService.save(configuration, true);
+
+ keepTrying("a configuration whose route cannot be built must be marked, not silently dropped",
+ () -> importConfigurationService.load(createdImportConfigId),
+ c -> c != null && RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT.equals(c.getStatus()), 1000, 30);
+
+ Assert.assertTrue("no route may consume a source outside the permitted directory", waiting.exists());
+ }
+
+ @Test
+ public void aStoredExportConfigurationOutsideThePermittedDirectoryIsMarkedAndWritesNothing() throws Exception {
+ File arbitraryDir = new File(ARBITRARY_DIR);
+ Assert.assertTrue("could not prepare the test fixture", arbitraryDir.exists() || arbitraryDir.mkdirs());
+ File shouldNeverAppear = new File(arbitraryDir, "export-must-not-appear.csv");
+ Files.deleteIfExists(shouldNeverAppear.toPath());
+
+ createdExportConfigId = "it-export-stored-out-of-bounds";
+ ExportConfiguration configuration = recurrentExport(createdExportConfigId,
+ fileUri(ARBITRARY_DIR, "?fileName=export-must-not-appear.csv"));
+
+ exportConfigurationService.save(configuration, true);
+
+ keepTrying("a configuration whose route cannot be built must be marked, not silently dropped",
+ () -> exportConfigurationService.load(createdExportConfigId),
+ c -> c != null && RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT.equals(c.getStatus()), 1000, 30);
+
+ Assert.assertFalse("no route may write to a destination outside the permitted directory",
+ shouldNeverAppear.exists());
+ }
+
+ @Test
+ public void aMarkedConfigurationRecoversOnItsOwnWhenItsEndpointBecomesAcceptableAgain() throws Exception {
+ createdImportConfigId = "it-import-recovering";
+ ImportConfiguration configuration = recurrentImport(createdImportConfigId,
+ fileUri(ARBITRARY_DIR, "?fileName=containment-it.csv"));
+ importConfigurationService.save(configuration, true);
+
+ ImportConfiguration marked = keepTrying("the configuration should first be marked",
+ () -> importConfigurationService.load(createdImportConfigId),
+ c -> c != null && RouterConstants.CONFIG_STATUS_INVALID_ENDPOINT.equals(c.getStatus()), 1000, 30);
+
+ // what operations would do: put the endpoint back where it is permitted
+ marked.getProperties().put("source", fileUri(PERMITTED_IMPORT_DIR, "?fileName=containment-it.csv&consumer.delay=10m"));
+ importConfigurationService.save(marked, true);
+
+ keepTrying("restoring the endpoint must clear the mark without anyone touching the status",
+ () -> importConfigurationService.load(createdImportConfigId),
+ c -> c != null && c.getStatus() == null, 1000, 30);
+ }
+
+ // ---------------------------------------------------------------------------------------------
+ // Fixtures
+ // ---------------------------------------------------------------------------------------------
+
+ private String fileUri(String directory, String suffix) {
+ return "file://" + new File(directory).getAbsolutePath() + suffix;
+ }
+
+ /**
+ * Executes the request against the shared client rather than through {@code executeHttpRequest},
+ * which consumes the response body to log it whenever the status is not {@code 200} — these tests
+ * need to read that body themselves.
+ */
+ private Response postJson(String url, Object body) throws Exception {
+ HttpPost request = new HttpPost(getFullUrl(url));
+ request.setEntity(new StringEntity(objectMapper.writeValueAsString(body), ContentType.APPLICATION_JSON));
+ try (CloseableHttpResponse response = httpClient.execute(request)) {
+ return new Response(response.getStatusLine().getStatusCode(),
+ response.getEntity() == null ? "" : EntityUtils.toString(response.getEntity()));
+ }
+ }
+
+ private static final class Response {
+ private final int status;
+ private final String body;
+
+ private Response(int status, String body) {
+ this.status = status;
+ this.body = body;
+ }
+ }
+
+ private ImportConfiguration recurrentImport(String itemId, String source) {
+ ImportConfiguration configuration = new ImportConfiguration();
+ configuration.setItemId(itemId);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setColumnSeparator(",");
+ configuration.setActive(true);
+
+ Map<String, Integer> mapping = new HashMap<>();
+ mapping.put("email", 0);
+ mapping.put("firstName", 1);
+ configuration.getProperties().put("mapping", mapping);
+ configuration.getProperties().put("source", source);
+ configuration.setMergingProperty("email");
+ return configuration;
+ }
+
+ private ExportConfiguration recurrentExport(String itemId, String destination) {
+ ExportConfiguration configuration = new ExportConfiguration();
+ configuration.setItemId(itemId);
+ configuration.setConfigType(RouterConstants.IMPORT_EXPORT_CONFIG_TYPE_RECURRENT);
+ configuration.setColumnSeparator(";");
+ configuration.setMultiValueDelimiter("()");
+ configuration.setMultiValueSeparator(";");
+ configuration.setActive(true);
+
+ Map<String, String> mapping = new HashMap<>();
+ mapping.put("0", "firstName");
+ configuration.getProperties().put("mapping", mapping);
+ configuration.getProperties().put("segment", "itContainmentSegment");
+ configuration.getProperties().put("period", "1m");
+ configuration.getProperties().put("destination", destination);
+ return configuration;
+ }
+}
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileImportRankingIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileImportRankingIT.java
index b675ae97f..403f8851b 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileImportRankingIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileImportRankingIT.java
@@ -86,7 +86,7 @@ public class ProfileImportRankingIT extends BaseIT {
mappingRanking.put("city", 5);
importConfigRanking.getProperties().put("mapping", mappingRanking);
- File importSurfersFile = new File("data/tmp/recurrent_import/");
+ File importSurfersFile = new File("data/tmp/recurrent_import/" + TEST_TENANT_ID);
importConfigRanking.getProperties().put("source",
"file://" + importSurfersFile.getAbsolutePath() + "?fileName=5-ranking-test.csv&move=.done");
importConfigRanking.setActive(true);
diff --git a/itests/src/test/java/org/apache/unomi/itests/ProfileImportSurfersIT.java b/itests/src/test/java/org/apache/unomi/itests/ProfileImportSurfersIT.java
index 5ddbf4401..6495cbe95 100644
--- a/itests/src/test/java/org/apache/unomi/itests/ProfileImportSurfersIT.java
+++ b/itests/src/test/java/org/apache/unomi/itests/ProfileImportSurfersIT.java
@@ -84,7 +84,7 @@ public class ProfileImportSurfersIT extends BaseIT {
mappingSurfers.put("city", 6);
importConfigSurfers.getProperties().put("mapping", mappingSurfers);
- File importSurfersFile = new File("data/tmp/recurrent_import/");
+ File importSurfersFile = new File("data/tmp/recurrent_import/" + TEST_TENANT_ID);
importConfigSurfers.getProperties().put("source",
"file://" + importSurfersFile.getAbsolutePath() + "?fileName=2-surfers-test.csv&move=.done");
importConfigSurfers.setActive(true);
diff --git a/itests/src/test/resources/org.apache.unomi.router.cfg b/itests/src/test/resources/org.apache.unomi.router.cfg
index ddcc351b1..ea0af5d09 100644
--- a/itests/src/test/resources/org.apache.unomi.router.cfg
+++ b/itests/src/test/resources/org.apache.unomi.router.cfg
@@ -41,3 +41,11 @@ executions.error.report.size=200
#Allowed source endpoints
config.allowedEndpoints=file,ftp,sftp,ftps
+
+#Base directories a file endpoint may resolve into. The values that reach the router are the ones
+#BaseIT writes to etc/custom.system.properties: Karaf's configuration plugin overrides a property
+#whose <pid>.<key> is a system property, and both of these are. The keys are kept here because the
+#plugin only overrides keys the configuration already carries. Each tenant is confined to a
+#subdirectory of these, named after the tenant.
+config.import.baseDir=${karaf.data}/tmp/recurrent_import
+config.export.baseDir=${karaf.data}/tmp/recurrent_export
diff --git a/manual/src/main/asciidoc/profile-import-export.adoc b/manual/src/main/asciidoc/profile-import-export.adoc
index 839ce37a1..d97f2f7a1 100644
--- a/manual/src/main/asciidoc/profile-import-export.adoc
+++ b/manual/src/main/asciidoc/profile-import-export.adoc
@@ -230,10 +230,26 @@ To change this you have to change the default values of these properties.
#errors report size
org.apache.unomi.router.executions.error.report.size=200
-Final one is about the allowed endpoints you can use when building the source or destionation path, as mentioned above
+Next one is about the allowed endpoints you can use when building the source or destionation path, as mentioned above
we can have a path of type `file`, `ftp`, `ftps`, `sftp`. You can make it less if you want to omit some endpoints (eg.
you don't want to permit the use of non secure FTP).
#Allowed source endpoints
org.apache.unomi.router.config.allowedEndpoints=file,ftp,sftp,ftps
+Final ones say where a `file` endpoint may resolve. A recurrent import source or export destination using the `file`
+scheme is refused unless it resolves inside one of these base directories, under the directory named after its
+tenant (`{baseDir}/{tenantId}`), at any depth, and so is any of its options that names a path (`fileName`, `move`,
+`moveFailed`, ...). The same applies to `localWorkDirectory`, whatever the scheme, since that one names a local
+directory even on a remote endpoint. Each property accepts a comma-separated list. A tenant cannot point a file
+endpoint at another tenant's directory.
+
+ #Base directories a file endpoint may resolve into
+ org.apache.unomi.router.config.import.baseDir=${karaf.data}/router/import/
+ org.apache.unomi.router.config.export.baseDir=${karaf.data}/router/export/
+
+Import and export are kept apart on purpose: with a single shared directory, an export writing a `.csv` could be picked
+up by an import route polling the same place. Point both properties at the same directory only if that is what you
+mean. A configuration that resolves outside them builds no route and is marked `INVALID_ENDPOINT`; restoring the
+directories clears that mark on its own at the next rebuild.
+
diff --git a/package/src/main/resources/etc/custom.system.properties b/package/src/main/resources/etc/custom.system.properties
index 9da27617d..a0ef8c6da 100644
--- a/package/src/main/resources/etc/custom.system.properties
+++ b/package/src/main/resources/etc/custom.system.properties
@@ -484,6 +484,14 @@ org.apache.unomi.router.config.allowedEndpoints=${env:UNOMI_ROUTER_CONFIG_ALLOWE
#Configs refresh interval
org.apache.unomi.router.configs.refresh.interval=${env:UNOMI_ROUTER_CONFIGS_REFRESH_INTERVAL:-1000}
+#Base directories a file endpoint may resolve into, comma-separated. A recurrent import source or
+#export destination using the file scheme is refused unless it resolves inside one of them, under
+#the directory named after its tenant ({baseDir}/{tenantId}), at any depth. Import and export are
+#kept apart so that an export cannot write into a directory an import route is polling; point them
+#at the same directory only if that is what you mean.
+org.apache.unomi.router.config.import.baseDir=${env:UNOMI_ROUTER_CONFIG_IMPORT_BASEDIR:-${karaf.data}/router/import/}
+org.apache.unomi.router.config.export.baseDir=${env:UNOMI_ROUTER_CONFIG_EXPORT_BASEDIR:-${karaf.data}/router/export/}
+
#######################################################################################################################
## Salesforce connector settings ##
#######################################################################################################################