You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ossie.apache.org by jb...@apache.org on 2026/10/01 14:56:55 UTC
(ossie) branch main updated: fix(salesforce): translate metric expressions and validate field references (#404)
This is an automated email from the ASF dual-hosted git repository.
jbonofre pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ossie.git
The following commit(s) were added to refs/heads/main by this push:
new a2e0eb3f fix(salesforce): translate metric expressions and validate field references (#404)
a2e0eb3f is described below
commit a2e0eb3f1eb74b5c1cb43441358be4cfbf6e8d7c
Author: Saurabh Deshpande <43...@users.noreply.github.com>
AuthorDate: Thu Oct 1 07:56:49 2026 -0700
fix(salesforce): translate metric expressions and validate field references (#404)
* fix(salesforce): export OSI metrics as semanticCalculatedMeasurements
MetricMappingHandler.mapOssieToSalesforce() was a stub that discarded
every metric mapping and returned without writing anything, so an
Ossie model's metrics never appeared in the exported Salesforce
semanticCalculatedMeasurements.
Wire up the existing generic name/description mapping (mappings.yaml
already declared metrics.name/metrics.description, unused until now),
and add unwrapExpressions() to flatten each metric's expression into a
Salesforce-compatible string. TABLEAU is preferred when present since
that is what Salesforce/Tableau CRM speaks; a model authored without
one falls back to its ANSI_SQL expression unresolved/untranslated,
since actually resolving/rewriting into TABLEAU syntax depends on
#222's still-open expression-language work, not this fix. A metric
with neither dialect fails the conversion with an actionable error
naming the metric, rather than being silently dropped.
Fixes #399
* fix(salesforce): translate metric expressions and validate references
* fix(salesforce): exclude disabled relationship paths
* feat(salesforce): compile bound OSI expressions to Tua
Use JSqlParser with a shared typed expression pipeline for derived fields
and dependent metrics. Preserve semantic identities through environment
bindings, validate relationships and final model references, and reject
unsupported expressions or ambiguous metadata before export.
Validation: 442 tests passed with no skips, Apache RAT passed, and packaged
CLI export, import, and physical-binding smoke checks passed.
* refactor(salesforce): keep metric conversion focused on issue 403
Defer physical bindings, derived fields, metric dependencies, extension
changes and whole-model validation. Keep JSqlParser-backed metric parsing,
typed Tua generation and targeted field/relationship reference checks.
Consolidate the compiler around metric-specific components and retain
focused semantic and integration regressions.
Validation: 254 tests passed with no skips, Apache RAT passed, and the
packaged CLI export smoke check passed.
* fix(salesforce): preserve redundant metric syntax and repair semantic tests
Normalize bounded unary chains and redundant grouping before JSqlParser,
while preserving operand validation, quotes, function arguments and limits.
Reject malformed null predicates and tuple arguments explicitly.
Fix ELSEIF evaluation and signed-zero equality, distinct counting and
zero-divisor checks in the independent semantic test helper. Add focused
compatibility, semantic and rejection regressions.
Validation: 280 tests passed with no skips; Apache RAT passed. Packaged CLI
checks cover successful conversion and failure without partial output.
* test(salesforce): port metric tests to the flat Ossie document format
* ci: allow dual-licensed JSqlParser in dependency review
JSqlParser is licensed LGPL-2.1 OR Apache-2.0. We use it under Apache-2.0,
but dependency-review-action matches the LGPL-2.1 side against deny-licenses.
---------
Co-authored-by: Ashfaq <10...@users.noreply.github.com>
Co-authored-by: JB Onofré <jb...@apache.org>
---
.github/workflows/dependency-review.yml | 2 +
converters/salesforce/README.md | 133 ++++-
converters/salesforce/pom.xml | 14 +
.../apache/ossie/converter/ConverterConstants.java | 1 +
.../apache/ossie/converter/MetricExpression.java | 61 +++
.../converter/MetricExpressionTranslator.java | 329 +++++++++++++
.../ossie/converter/MetricFieldResolver.java | 305 ++++++++++++
.../ossie/converter/MetricMappingHandler.java | 44 +-
.../ossie/converter/SqlMetricExpressionParser.java | 250 ++++++++++
.../ossie/converter/TuaMetricExpressionParser.java | 230 +++++++++
.../test/java/org/apache/ossie/MetricCliTest.java | 60 +++
.../apache/ossie/MetricExportIntegrationTest.java | 427 ++++++++++++++++
.../ossie/OssieToSalesforceConverterTest.java | 83 +++-
.../converter/MetricExpressionSemanticsTest.java | 543 +++++++++++++++++++++
.../converter/MetricExpressionTranslatorTest.java | 399 +++++++++++++++
.../ossie/converter/MetricFieldResolverTest.java | 331 +++++++++++++
16 files changed, 3208 insertions(+), 4 deletions(-)
diff --git a/.github/workflows/dependency-review.yml b/.github/workflows/dependency-review.yml
index 35bc9da8..0aa25398 100644
--- a/.github/workflows/dependency-review.yml
+++ b/.github/workflows/dependency-review.yml
@@ -34,4 +34,6 @@ jobs:
- uses: actions/dependency-review-action@a1d282b36b6f3519aa1f3fc636f609c47dddb294 # v5.0.0
with:
deny-licenses: GPL-1.0-only, GPL-1.0-or-later, GPL-2.0-only, GPL-2.0-or-later, GPL-3.0-only, GPL-3.0-or-later, LGPL-2.0-only, LGPL-2.0-or-later, LGPL-2.1-only, LGPL-2.1-or-later, LGPL-3.0-only, LGPL-3.0-or-later, AGPL-1.0-only, AGPL-1.0-or-later, AGPL-3.0-only, AGPL-3.0-or-later, SSPL-1.0, EUPL-1.1, EUPL-1.2
+ # JSqlParser is dual-licensed (LGPL-2.1 OR Apache-2.0); we use it under Apache-2.0.
+ allow-dependencies-licenses: pkg:maven/com.github.jsqlparser/jsqlparser
comment-summary-in-pr: always
diff --git a/converters/salesforce/README.md b/converters/salesforce/README.md
index 4160f9f3..c157aa74 100644
--- a/converters/salesforce/README.md
+++ b/converters/salesforce/README.md
@@ -55,6 +55,16 @@ The canonical Ossie schema is bundled automatically during the build. Obtain the
2. Copy the JSON schema content from the page
3. Save it to `src/main/resources/schemas/salesforce-semantic-model-schema.json`
+Run the complete suite, including Salesforce schema checks, with:
+
+```bash
+mvn -DrequireSalesforceSchema=true clean verify
+```
+
+The property makes a missing Salesforce schema fail the test run. Without it,
+schema-dependent tests retain their existing skip behavior. `verify` also checks
+Apache license headers. Do not commit downloaded schemas.
+
### Apache Ossie Schema
The build copies [the canonical Ossie schema](../../core-spec/ossie-schema.json)
@@ -182,7 +192,7 @@ ossieToSf.convert(Paths.get("input/model.yaml"), Paths.get("output/"));
| Field `datatype` | Field `dataType` when a safe mapping exists |
| `relationships[]` | `semanticRelationships[]` |
| `from_columns` + `to_columns` | `criteria[]` |
-| `metrics[]` | Not currently exported |
+| `metrics[]` | Validated Tua expressions in `semanticCalculatedMeasurements[]` |
| `ai_context` | `businessPreferences` |
| `custom_extensions` (vendor: `SALESFORCE`) | Restored properties |
@@ -243,6 +253,127 @@ dimensions.
**Unsupported relationships** (containing Formula or SemanticField types) are stored in `custom_extensions` at the model level rather than being converted to Ossie relationships.
+### Metric expressions
+
+Metrics select `TABLEAU`, then `SNOWFLAKE`, then `ANSI_SQL`, independent of entry
+order. The selected expression is parsed and validated; an invalid preferred
+expression fails rather than falling back to another dialect. Duplicate selected
+dialect entries are errors. Successful conversion exports every declared metric.
+
+The target is the Salesforce/Tableau Next semantic model's
+[Tua calculation language](https://developer.salesforce.com/docs/data/semantic-layer/guide/query-api-in-depth-functions.html).
+Calculated measurements emit `syntax: Tua`, `dataType: Number`, and
+`aggregationType: UserAgg`, so an already aggregated formula is not aggregated
+again. See [calculated fields](https://developer.salesforce.com/docs/data/semantic-layer/guide/query-api-in-depth-calculated-fields.html)
+and [aggregation rules](https://developer.salesforce.com/docs/data/semantic-layer/guide/query-api-in-depth-aggregation.html).
+
+| SQL input | Tua output |
+|-----------|------------|
+| `SUM`, `AVG`, `MIN`, `MAX`, `COUNT(field)` | Same aggregate |
+| `COUNT(DISTINCT field)` | `COUNTD(field)` |
+| `+`, `-`, `*`, `/`, parentheses, numeric constants | Explicitly grouped arithmetic |
+| Searched `CASE WHEN` | `IF … THEN … ELSEIF … ELSE … END` |
+| Comparisons, `AND`, `OR`, `NOT` | Equivalent grouped operators |
+| `COALESCE(a, b, …)` | Nested `IFNULL` |
+| `NULLIF(a, b)` | `IF a = b THEN NULL ELSE a END` |
+| `IS NULL`, `IS NOT NULL` | `ISNULL`, `NOT ISNULL` |
+| `ABS`, `ROUND`, `CEIL`, `FLOOR` | `ABS`, `ROUND`, `CEILING`, `FLOOR` |
+
+These constructs compose. For example, with declared numeric fields `profit` and
+`revenue` in dataset `orders`:
+
+```yaml
+metrics:
+ - name: margin
+ datatype: Decimal
+ expression:
+ dialects:
+ - dialect: SNOWFLAKE
+ expression: SUM(orders.profit) / NULLIF(SUM(orders.revenue), 0)
+```
+
+The resulting expression is:
+
+```text
+(SUM([orders].[profit]) / (IF (SUM([orders].[revenue]) = 0) THEN NULL ELSE SUM([orders].[revenue]) END))
+```
+
+**Binding and types.** References resolve against declared dataset and field
+names, then against fields actually emitted to Salesforce. Physical column names
+and source paths are not aliases. Unqualified SQL fields must be unique. Regular
+SQL names normalize to uppercase; double-quoted names match the normalized
+declaration exactly, following the [expression specification](../../core-spec/expression_language.md).
+Thus `"ORDERS"."AMOUNT"` matches regular declarations `orders.amount`, while
+`"orders"."amount"` requires explicitly quoted lowercase declarations. Target
+API names are preserved; conversion does not rename fields or discover columns.
+Names containing brackets or control characters fail because their Tua escaping
+is not established. Referenced fields must also have a single, unqualified physical
+column binding in the exported model. This rejects derived expressions such as
+`profit+tax` that the existing field mapper can misclassify as physical columns.
+Derived-field compilation and normalization of qualified/quoted physical bindings
+belong to separate converter work.
+
+`TABLEAU` references use exact `[dataset].[field]` API names. Its supported formula
+subset is the Tua equivalents above, including `IF`, `IFNULL`, `ISNULL`, `COUNTD`
+and `CEILING`. Existing `ANSI_SQL` expressions using complete bracket notation
+retain that spelling as a compatibility case and receive the same validation.
+Bracket notation is not accepted as Snowflake SQL.
+
+Fields need a known compatible datatype, either declared in OSI or restored from
+an existing Salesforce field type. Arithmetic and `SUM`/`AVG` require numbers;
+`MIN`/`MAX` also permit text and temporal values inside numeric calculations.
+Comparisons and conditional/null-handling branches must have compatible types.
+Metrics must return numbers; a declared `Integer` result cannot conceal a
+fractional expression. Missing metric types are inferred. A formula must be
+aggregated or constant: mixed row/aggregate expressions, nested aggregates, and
+aggregates without a dataset field fail. A single aggregate cannot combine fields
+from multiple datasets. Separate aggregates can use datasets connected through
+explicitly enabled exported relationships. Each usable edge must match one source
+relationship by name, endpoints and ordered join-key pairs; join fields must resolve
+as direct fields. Missing or corrupted edges cannot establish connectivity. These
+checks protect metrics without changing the relationship mapper or proving join grain.
+
+**Limits and compatibility.** `COUNT`/`COUNTD` take a field. `ROUND` supports one
+argument or a second integer-literal precision; rounding-mode overloads are not
+supported. `CEIL`/`FLOOR` take one argument. They and `ROUND` at zero or negative
+precision infer integral values, so compatible `Integer` metrics are accepted. Simple `CASE`, date/time and string
+functions, casts, metric/calculated-field references, windows, LOD, `COUNT(*)`,
+SQL comments and backslash string escapes are outside this subset. String and
+Boolean literals are supported in predicates. No null-to-zero setting or implicit
+cast is added. Errors name the metric and explain the rejected construct or
+reference. Literal zero divisors fail; use `NULLIF` to make a zero denominator
+nullable. Expressions have bounded size, nesting and generated output.
+
+This replaces the unvalidated SQL fallback introduced in
+[#402](https://github.com/apache/ossie/pull/402). Previously accepted invalid,
+unsupported or untyped formulas now fail, including invalid `TABLEAU` input.
+Metric/model extension preservation remains owned by #402. CLI conversion errors
+are printed to stderr with exit code 3; this small change overlaps the conversion
+error handling in [#286](https://github.com/apache/ossie/pull/286).
+
+**Implementation choice.** The existing mapping pipeline calls a focused Java
+metric compiler. JSqlParser 5.3 parses SQL; a bounded Tua frontend handles native
+formulas. Both produce a small immutable metric AST. Type/aggregation checks finish
+before the private emitter writes Tua. No Python runtime is needed. JSqlParser is
+used under its Apache-2.0 option; its unused JMH benchmark dependency is excluded.
+
+The implementation has five components: `SqlMetricExpressionParser`,
+`TuaMetricExpressionParser`, `MetricExpression`, `MetricExpressionTranslator` and
+`MetricFieldResolver`. The resolver caches successful bindings and verified graph
+reachability within one model. To add supported syntax, update the relevant
+frontend and metric rule with composition and semantic tests. There is no generic
+model-planning or deployment framework in this change.
+
+**Validation boundary.** Unit tests cover parsing, binding and failure behavior.
+Independent local Tua evaluation tests use synthetic rows with nulls, duplicates,
+empty inputs and zero denominators. Those tests model the intended semantics;
+they are not native Tableau Next execution. The published Salesforce **output**
+schema checks structure, not formula syntax, catalog bindings or authoring API
+acceptance. Before deployment, validate authoring and native queries in a test
+org, including numeric precision, rounding ties, empty groups, null behavior and
+unguarded dynamic division and multi-dataset grain. This converter does not provision Data 360 bindings or enrich
+missing fields.
+
## Architecture
```
diff --git a/converters/salesforce/pom.xml b/converters/salesforce/pom.xml
index 4af6088e..e0cbed27 100644
--- a/converters/salesforce/pom.xml
+++ b/converters/salesforce/pom.xml
@@ -52,9 +52,23 @@
<json-schema-validator.version>1.5.9</json-schema-validator.version>
<maven-shade-plugin.version>3.6.2</maven-shade-plugin.version>
<maven-resources-plugin.version>3.5.0</maven-resources-plugin.version>
+ <jsqlparser.version>5.3</jsqlparser.version>
</properties>
<dependencies>
+ <dependency>
+ <groupId>com.github.jsqlparser</groupId>
+ <artifactId>jsqlparser</artifactId>
+ <version>${jsqlparser.version}</version>
+ <exclusions>
+ <!-- JSqlParser declares its benchmark harness as a runtime dependency.
+ The parser classes do not depend on JMH. -->
+ <exclusion>
+ <groupId>org.openjdk.jmh</groupId>
+ <artifactId>jmh-core</artifactId>
+ </exclusion>
+ </exclusions>
+ </dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/ConverterConstants.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/ConverterConstants.java
index 8e25e65e..55ab18e5 100644
--- a/converters/salesforce/src/main/java/org/apache/ossie/converter/ConverterConstants.java
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/ConverterConstants.java
@@ -88,6 +88,7 @@ public final class ConverterConstants {
public static final String DIALECTS = "dialects";
public static final String DIALECT = "dialect";
public static final String DIALECT_TABLEAU = "TABLEAU";
+ public static final String DIALECT_ANSI_SQL = "ANSI_SQL";
// Relationship properties
public static final String CRITERIA = "criteria";
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpression.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpression.java
new file mode 100644
index 00000000..9d84ca9b
--- /dev/null
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpression.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import java.math.BigDecimal;
+import java.util.List;
+import java.util.Set;
+
+/** Immutable compiler nodes. Neither source parsing nor type checking emits target text. */
+final class MetricExpression {
+ private MetricExpression() {}
+ sealed interface Node permits Literal, Field, Unary, Binary, Call, Conditional {}
+ record Literal(Object value) implements Node {}
+ record Field(List<MetricFieldResolver.Identifier> parts, boolean tableau) implements Node {
+ Field { parts = List.copyOf(parts); }
+ }
+ record Unary(String operator, Node operand) implements Node {}
+ record Binary(String operator, Node left, Node right) implements Node {}
+ record Call(String name, List<Node> arguments, boolean distinct) implements Node {
+ Call { arguments = List.copyOf(arguments); }
+ }
+ /** Alternating predicate/result pairs, followed by a separate ELSE expression. */
+ record Conditional(List<Node> branches, Node otherwise) implements Node {
+ Conditional { branches = List.copyOf(branches); }
+ }
+ enum Level { CONSTANT, ROW, AGGREGATE }
+ enum Type {
+ INTEGER("Integer"), DECIMAL("Decimal"), FLOAT("Float"), STRING("String"),
+ BOOLEAN("Boolean"), DATE("Date"), DATETIME("DateTime"), DATETIME_TZ("DateTimeTz"),
+ NULL(null), UNKNOWN(null);
+ final String datatype;
+ Type(String datatype) { this.datatype = datatype; }
+ boolean numeric() { return this == INTEGER || this == DECIMAL || this == FLOAT; }
+ static Type of(String datatype) {
+ if (datatype == null) return UNKNOWN;
+ for (Type type : values()) if (datatype.equals(type.datatype)) return type;
+ return UNKNOWN;
+ }
+ }
+ record Typed(Node node, Type type, Level level, Set<String> datasets,
+ List<Typed> children, MetricFieldResolver.ResolvedField binding, BigDecimal number) {
+ Typed { datasets = Set.copyOf(datasets); children = List.copyOf(children); }
+ }
+}
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpressionTranslator.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpressionTranslator.java
new file mode 100644
index 00000000..33160177
--- /dev/null
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricExpressionTranslator.java
@@ -0,0 +1,329 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.apache.ossie.converter.MetricExpression.*;
+import static org.apache.ossie.util.DataStructureUtils.*;
+
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import org.apache.ossie.exception.ConversionException;
+
+/** Compiles the documented metric subset: parse, bind/check, then emit bounded Tua text. */
+final class MetricExpressionTranslator {
+ private static final List<String> DIALECTS = List.of("TABLEAU", "SNOWFLAKE", "ANSI_SQL");
+ private static final Set<String> AGGREGATES = Set.of("SUM", "AVG", "MIN", "MAX", "COUNT", "COUNTD");
+ record Result(String expression, String dataType) {}
+ private MetricExpressionTranslator() {}
+
+ static Result translate(Map<String, Object> metric, Map<String, Object> sourceModel,
+ Map<String, Object> targetModel) {
+ return translate(metric, new MetricFieldResolver(sourceModel, targetModel));
+ }
+
+ static Result translate(Map<String, Object> metric, MetricFieldResolver resolver) {
+ String name = getString(metric, "name");
+ try {
+ Map<String, Object> expression = getMap(metric, "expression");
+ List<Object> dialects = expression == null ? null : getList(expression, "dialects");
+ if (dialects == null) {
+ throw new IllegalArgumentException("missing expression.dialects; provide TABLEAU, SNOWFLAKE or ANSI_SQL");
+ }
+ Map<String, String> candidates = new java.util.LinkedHashMap<>();
+ for (Object entry : dialects) {
+ Map<String, Object> value = asMap(entry);
+ String dialect = getString(value, "dialect");
+ if (DIALECTS.contains(dialect)) {
+ if (candidates.containsKey(dialect)) {
+ throw new IllegalArgumentException("ambiguous expression: multiple " + dialect + " entries");
+ }
+ candidates.put(dialect, getString(value, "expression"));
+ }
+ }
+ String dialect = DIALECTS.stream().filter(candidates::containsKey).findFirst()
+ .orElseThrow(() -> new IllegalArgumentException(
+ "no supported dialect; provide TABLEAU, SNOWFLAKE or ANSI_SQL"));
+ String text = candidates.get(dialect);
+ if (text == null || text.isBlank()) {
+ throw new IllegalArgumentException(dialect + " expression is empty");
+ }
+ Node parsed = dialect.equals("TABLEAU")
+ ? new TuaMetricExpressionParser(TuaMetricExpressionParser.tokenize(text, dialect)).parse() : SqlMetricExpressionParser.parse(text, dialect);
+ Typed result = new Analyzer(dialect, resolver).analyze(parsed);
+ resolver.validateDatasets(result.datasets());
+ Type declared = Type.of(getString(metric, "datatype"));
+ if (metric.containsKey("datatype") && declared == Type.UNKNOWN) {
+ throw new IllegalArgumentException("unsupported metric datatype " + getString(metric, "datatype"));
+ }
+ if (!result.type().numeric() && result.type() != Type.NULL) {
+ throw new IllegalArgumentException("calculated measurements must be numeric, found " + result.type());
+ }
+ if (declared != Type.UNKNOWN && (!declared.numeric()
+ || (declared == Type.INTEGER && result.type() != Type.INTEGER && result.type() != Type.NULL))) {
+ throw new IllegalArgumentException("datatype " + getString(metric, "datatype")
+ + " is incompatible with expression result " + result.type());
+ }
+ if (result.type() == Type.NULL && !declared.numeric()) {
+ throw new IllegalArgumentException("all-null result needs an explicit numeric datatype");
+ }
+ if (result.level() == Level.ROW) {
+ throw new IllegalArgumentException("unaggregated field in metric; use an explicit aggregate");
+ }
+ return new Result(new Emitter().emit(result), "Number");
+ } catch (IllegalArgumentException e) {
+ throw new ConversionException("Metric '" + name + "': " + e.getMessage(), e);
+ }
+ }
+
+ /** Checks the complete tree before any target text is emitted. */
+ private static final class Analyzer {
+ private final String dialect;
+ private final MetricFieldResolver resolver;
+ private int depth;
+ Analyzer(String dialect, MetricFieldResolver resolver) {
+ this.dialect = dialect; this.resolver = resolver;
+ }
+ Typed analyze(Node node) {
+ if (++depth > 128) throw new IllegalArgumentException("expression nesting exceeds 128 levels");
+ try { return analyzeNode(node); } finally { depth--; }
+ }
+ private Typed analyzeNode(Node node) {
+ if (node instanceof Literal literal) {
+ Object value = literal.value();
+ Type type = value == null ? Type.NULL : value instanceof Boolean ? Type.BOOLEAN
+ : value instanceof String ? Type.STRING
+ : ((BigDecimal) value).stripTrailingZeros().scale() <= 0 ? Type.INTEGER : Type.DECIMAL;
+ return new Typed(node, type, Level.CONSTANT, Set.of(), List.of(), null,
+ value instanceof BigDecimal number ? number : null);
+ }
+ if (node instanceof Field field) {
+ MetricFieldResolver.ResolvedField binding = resolver.resolve(field.parts(), field.tableau());
+ if (binding == null || binding.expression() == null || binding.expression().isBlank()) {
+ throw new IllegalArgumentException("field resolver returned no binding");
+ }
+ Type type = Type.of(binding.datatype());
+ if (type == Type.UNKNOWN) throw new IllegalArgumentException("field reference needs known field datatypes");
+ return new Typed(node, type, Level.ROW, Set.of(binding.dataset()), List.of(), binding, null);
+ }
+ if (node instanceof Unary unary) {
+ Typed child = analyze(unary.operand());
+ String operator = unary.operator();
+ Type result = child.type();
+ BigDecimal number = child.number();
+ if (operator.equals("ISNULL")) { result = Type.BOOLEAN; number = null; }
+ else if (operator.equals("NOT")) { require(child, Type.BOOLEAN, "NOT"); result = Type.BOOLEAN; number = null; }
+ else { numeric(child, "unary " + operator); if (operator.equals("-") && number != null) number = number.negate(); }
+ return new Typed(node, result, child.level(), child.datasets(), List.of(child), null, number);
+ }
+ if (node instanceof Binary binary) {
+ Typed left = analyze(binary.left()); Typed right = analyze(binary.right());
+ String op = binary.operator();
+ Type result;
+ if (op.equals("AND") || op.equals("OR")) {
+ require(left, Type.BOOLEAN, op); require(right, Type.BOOLEAN, op); result = Type.BOOLEAN;
+ } else if (Set.of("=", "!=", "<", "<=", ">", ">=").contains(op)) {
+ compatible(left.type(), right.type(), "comparison");
+ if (!Set.of("=", "!=").contains(op) && (left.type() == Type.BOOLEAN || right.type() == Type.BOOLEAN)) {
+ throw new IllegalArgumentException("ordered comparison requires numeric, text or temporal operands");
+ }
+ result = Type.BOOLEAN;
+ } else {
+ numeric(left, op); numeric(right, op);
+ result = compatible(left.type(), right.type(), op);
+ if (op.equals("/")) {
+ if (right.number() != null && right.number().signum() == 0) {
+ throw new IllegalArgumentException("division by literal zero; use NULLIF(denominator, 0) for a nullable denominator");
+ }
+ result = Type.DECIMAL;
+ }
+ }
+ return compose(node, result, List.of(left, right));
+ }
+ if (node instanceof Conditional conditional) {
+ List<Typed> children = new ArrayList<>();
+ Type result = Type.NULL;
+ for (int i = 0; i < conditional.branches().size(); i += 2) {
+ Typed predicate = analyze(conditional.branches().get(i));
+ require(predicate, Type.BOOLEAN, "conditional predicate");
+ Typed branch = analyze(conditional.branches().get(i + 1));
+ result = compatible(result, branch.type(), "conditional branches");
+ children.add(predicate); children.add(branch);
+ }
+ Typed otherwise = analyze(conditional.otherwise());
+ result = compatible(result, otherwise.type(), "conditional branches");
+ children.add(otherwise);
+ return compose(node, result, children);
+ }
+ Call call = (Call) node;
+ String name = call.name();
+ if (call.distinct() && (!name.equals("COUNT") || dialect.equals("TABLEAU"))) {
+ throw new IllegalArgumentException("DISTINCT is supported only by SQL COUNT(DISTINCT field)");
+ }
+ if (Set.of("COALESCE", "NULLIF", "CEIL").contains(name) && dialect.equals("TABLEAU")
+ || Set.of("IFNULL", "ISNULL", "CEILING", "COUNTD").contains(name) && !dialect.equals("TABLEAU")) {
+ throw new IllegalArgumentException(name + " is outside the supported " + dialect + " subset");
+ }
+ int maximum = switch (name) {
+ case "COALESCE" -> Integer.MAX_VALUE;
+ case "IFNULL", "NULLIF", "ROUND" -> 2;
+ case "SUM", "AVG", "MIN", "MAX", "COUNT", "COUNTD", "ISNULL", "ABS", "CEIL", "CEILING", "FLOOR" -> 1;
+ default -> throw new IllegalArgumentException("unsupported function " + name);
+ };
+ int minimum = Set.of("COALESCE", "IFNULL", "NULLIF").contains(name) ? 2 : 1;
+ if (call.arguments().size() < minimum || call.arguments().size() > maximum) {
+ throw new IllegalArgumentException(name + " expects "
+ + (minimum == maximum ? minimum : minimum + " to " + maximum) + " arguments");
+ }
+ List<Typed> arguments = call.arguments().stream().map(this::analyze).toList();
+ if (AGGREGATES.contains(name)) return aggregate(call, arguments);
+ return switch (name) {
+ case "COALESCE", "IFNULL" -> {
+ Type result = Type.NULL;
+ for (Typed argument : arguments) result = compatible(result, argument.type(), name + " arguments");
+ yield compose(node, result, arguments);
+ }
+ case "NULLIF" -> {
+ compatible(arguments.get(0).type(), arguments.get(1).type(), "NULLIF arguments");
+ yield compose(node, arguments.get(0).type(), arguments);
+ }
+ case "ISNULL" -> compose(node, Type.BOOLEAN, arguments);
+ default -> numericFunction(call, arguments);
+ };
+ }
+
+ private Typed aggregate(Call call, List<Typed> arguments) {
+ String name = call.name(); Typed argument = arguments.get(0);
+ if (argument.level() == Level.AGGREGATE) throw new IllegalArgumentException("nested aggregate " + name + " is unsupported");
+ if (argument.datasets().isEmpty()) throw new IllegalArgumentException(name + " needs a declared field to establish its dataset");
+ if (argument.datasets().size() > 1) throw new IllegalArgumentException("one aggregate cannot combine fields from multiple datasets");
+ boolean count = name.equals("COUNT") || name.equals("COUNTD");
+ if (count && !(argument.node() instanceof Field)) {
+ throw new IllegalArgumentException(name + " requires a declared field; counting expressions is unsupported");
+ }
+ if (name.equals("MIN") || name.equals("MAX")) {
+ if (argument.type() == Type.BOOLEAN || argument.type() == Type.UNKNOWN) {
+ throw new IllegalArgumentException(name + " requires numeric, text or temporal operands");
+ }
+ } else if (!count) numeric(argument, name);
+ Type result = count ? Type.INTEGER : name.equals("AVG") ? Type.DECIMAL : argument.type();
+ return new Typed(call, result, Level.AGGREGATE, argument.datasets(), arguments, null, null);
+ }
+ private Typed numericFunction(Call call, List<Typed> arguments) {
+ Typed value = arguments.get(0); numeric(value, call.name());
+ BigDecimal places = BigDecimal.ZERO;
+ if (arguments.size() == 2) {
+ places = arguments.get(1).number();
+ if (places == null || places.stripTrailingZeros().scale() > 0
+ || places.compareTo(BigDecimal.valueOf(Integer.MIN_VALUE)) < 0
+ || places.compareTo(BigDecimal.valueOf(Integer.MAX_VALUE)) > 0) {
+ throw new IllegalArgumentException("ROUND precision must be a 32-bit integer literal");
+ }
+ }
+ Type result = value.type();
+ if (result != Type.NULL && (Set.of("CEIL", "CEILING", "FLOOR").contains(call.name())
+ || call.name().equals("ROUND") && places.signum() <= 0)) result = Type.INTEGER;
+ return compose(call, result, arguments);
+ }
+ private Typed compose(Node node, Type type, List<Typed> arguments) {
+ Level level = Level.CONSTANT; Set<String> datasets = new HashSet<>();
+ for (Typed argument : arguments) {
+ if (level != Level.CONSTANT && argument.level() != Level.CONSTANT && level != argument.level()) {
+ throw new IllegalArgumentException("cannot mix aggregate and unaggregated field expressions");
+ }
+ if (argument.level() != Level.CONSTANT) level = argument.level();
+ datasets.addAll(argument.datasets());
+ }
+ return new Typed(node, type, level, datasets, arguments, null, null);
+ }
+ private static void numeric(Typed value, String context) {
+ if (!value.type().numeric() && value.type() != Type.NULL) {
+ throw new IllegalArgumentException(context + " requires numeric operands, found " + value.type() + "; declare a compatible field datatype");
+ }
+ }
+ private static void require(Typed value, Type expected, String context) {
+ if (value.type() != expected && value.type() != Type.NULL) throw new IllegalArgumentException(context + " requires " + expected + ", found " + value.type());
+ }
+ private static Type compatible(Type left, Type right, String context) {
+ if (left == Type.UNKNOWN || right == Type.UNKNOWN) throw new IllegalArgumentException(context + " needs known field datatypes");
+ if (left == Type.NULL) return right;
+ if (right == Type.NULL || left == right) return left;
+ if (left.numeric() && right.numeric()) return left == Type.FLOAT || right == Type.FLOAT ? Type.FLOAT : Type.DECIMAL;
+ throw new IllegalArgumentException(context + " has incompatible types " + left + " and " + right);
+ }
+ }
+
+ /** Streams checked nodes so NULLIF expansion cannot allocate an unbounded string. */
+ private static final class Emitter {
+ private static final int MAX_OUTPUT = 131072;
+ private final StringBuilder output = new StringBuilder();
+ String emit(Typed expression) { append(expression); return output.toString(); }
+ private void text(String text) {
+ if ((long) output.length() + text.length() > MAX_OUTPUT) {
+ throw new IllegalArgumentException("translated expression exceeds 131072 characters");
+ }
+ output.append(text);
+ }
+ private void append(Typed value) {
+ Node node = value.node(); List<Typed> children = value.children();
+ if (node instanceof Literal literal) {
+ Object content = literal.value();
+ text(content == null ? "NULL" : content instanceof String string ? "'" + string.replace("'", "''") + "'"
+ : content instanceof Boolean bool ? bool ? "TRUE" : "FALSE" : ((BigDecimal) content).toPlainString());
+ } else if (node instanceof Field) {
+ text(value.binding().expression());
+ } else if (node instanceof Unary unary) {
+ switch (unary.operator()) {
+ case "+" -> append(children.get(0));
+ case "-" -> { text("(-"); append(children.get(0)); text(")"); }
+ case "NOT" -> { text("(NOT "); append(children.get(0)); text(")"); }
+ case "ISNULL" -> { text("ISNULL("); append(children.get(0)); text(")"); }
+ default -> throw new IllegalStateException("unvalidated unary operator");
+ }
+ } else if (node instanceof Binary binary) {
+ text("("); append(children.get(0)); text(" " + binary.operator() + " "); append(children.get(1)); text(")");
+ } else if (node instanceof Conditional) {
+ text("(IF ");
+ for (int i = 0; i < children.size() - 1; i += 2) {
+ if (i > 0) text(" ELSEIF ");
+ append(children.get(i)); text(" THEN "); append(children.get(i + 1));
+ }
+ text(" ELSE "); append(children.get(children.size() - 1)); text(" END)");
+ } else {
+ Call call = (Call) node;
+ if (call.name().equals("COALESCE") || call.name().equals("IFNULL")) {
+ for (int i = 0; i < children.size() - 1; i++) { text("IFNULL("); append(children.get(i)); text(", "); }
+ append(children.get(children.size() - 1));
+ for (int i = 0; i < children.size() - 1; i++) text(")");
+ } else if (call.name().equals("NULLIF")) {
+ text("(IF ("); append(children.get(0)); text(" = "); append(children.get(1));
+ text(") THEN NULL ELSE "); append(children.get(0)); text(" END)");
+ } else {
+ text((call.distinct() ? "COUNTD" : (call.name().equals("CEIL") ? "CEILING" : call.name())) + "(");
+ for (int i = 0; i < children.size(); i++) { if (i > 0) text(", "); append(children.get(i)); }
+ text(")");
+ }
+ }
+ }
+ }
+}
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricFieldResolver.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricFieldResolver.java
new file mode 100644
index 00000000..46faf463
--- /dev/null
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricFieldResolver.java
@@ -0,0 +1,305 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.apache.ossie.util.DataStructureUtils.getList;
+import static org.apache.ossie.util.DataStructureUtils.getString;
+import static org.apache.ossie.util.DataStructureUtils.streamMaps;
+
+import java.util.ArrayList;
+import java.util.ArrayDeque;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+/** Binds metric references to declared fields that the Salesforce converter actually exported. */
+final class MetricFieldResolver {
+
+ record Identifier(String text, boolean quoted) {}
+
+ record ResolvedField(String expression, String datatype, String dataset) {}
+
+ private record Field(Map<String, Object> dataset, Map<String, Object> field) {}
+
+ private final List<Map<String, Object>> datasets;
+ private final List<Map<String, Object>> targetDatasets;
+ private final List<Map<String, Object>> relationships;
+ private final List<Map<String, Object>> declaredRelationships;
+ private record Reference(List<Identifier> parts, boolean tableau) {
+ Reference { parts = List.copyOf(parts); }
+ }
+ private final Map<Reference, ResolvedField> resolved = new HashMap<>();
+ private final Map<String, Set<String>> reachable = new HashMap<>();
+ private final List<String> invalidRelationships = new ArrayList<>();
+ private Map<String, Set<String>> graph;
+
+ MetricFieldResolver(Map<String, Object> sourceModel, Map<String, Object> targetModel) {
+ datasets = items(sourceModel, "datasets");
+ targetDatasets = items(targetModel, "semanticDataObjects");
+ relationships = items(targetModel, "semanticRelationships");
+ declaredRelationships = items(sourceModel, "relationships");
+ }
+
+ /** Only unchanged, enabled edges can establish a metric's dataset connectivity. */
+ void validateDatasets(Set<String> referenced) {
+ if (referenced.size() < 2) return;
+ if (graph == null) graph = validatedGraph();
+ String first = referenced.stream().sorted().findFirst().orElseThrow();
+ Set<String> visited = reachable.computeIfAbsent(first, this::reachableFrom);
+ if (!visited.containsAll(referenced)) {
+ throw new IllegalArgumentException("Metric references disconnected datasets "
+ + referenced.stream().sorted().collect(Collectors.joining(", "))
+ + "; declare supported relationships connecting them before exporting the metric"
+ + (invalidRelationships.isEmpty() ? "" : "; unusable relationships: " + String.join("; ", invalidRelationships)));
+ }
+ }
+
+ private Map<String, Set<String>> validatedGraph() {
+ Map<String, Set<String>> result = new HashMap<>();
+ for (Map<String, Object> dataset : targetDatasets) {
+ String name = getString(dataset, "apiName");
+ if (name != null) result.put(name, new HashSet<>());
+ }
+ for (Map<String, Object> relationship : relationships) {
+ if (!Boolean.TRUE.equals(relationship.get("isEnabled"))) continue;
+ String name = getString(relationship, "apiName");
+ String left = getString(relationship, "leftSemanticDefinitionApiName");
+ String right = getString(relationship, "rightSemanticDefinitionApiName");
+ try {
+ if (!result.containsKey(left) || !result.containsKey(right)) {
+ throw new IllegalArgumentException("missing exported endpoint");
+ }
+ validateRelationship(relationship, name, left, right);
+ result.get(left).add(right);
+ result.get(right).add(left);
+ } catch (IllegalArgumentException e) {
+ invalidRelationships.add("'" + name + "': " + e.getMessage());
+ }
+ }
+ return result;
+ }
+
+ private void validateRelationship(Map<String, Object> target, String name, String left, String right) {
+ List<Map<String, Object>> matches = declaredRelationships.stream()
+ .filter(source -> name != null && name.equals(getString(source, "name"))).toList();
+ if (matches.size() != 1) throw new IllegalArgumentException("expected one source relationship");
+ Map<String, Object> source = matches.get(0);
+ if (!left.equals(getString(source, "from")) || !right.equals(getString(source, "to"))) {
+ throw new IllegalArgumentException("changed endpoints");
+ }
+ List<Object> from = getList(source, "from_columns");
+ List<Object> to = getList(source, "to_columns");
+ List<Map<String, Object>> criteria = items(target, "criteria");
+ if (from == null || to == null || from.isEmpty() || from.size() != to.size() || from.size() != criteria.size()) {
+ throw new IllegalArgumentException("changed or missing composite join keys");
+ }
+ for (int i = 0; i < from.size(); i++) {
+ Map<String, Object> pair = criteria.get(i);
+ if (!(from.get(i) instanceof String leftKey) || !(to.get(i) instanceof String rightKey)
+ || !leftKey.equals(pair.get("leftSemanticFieldApiName"))
+ || !rightKey.equals(pair.get("rightSemanticFieldApiName"))) {
+ throw new IllegalArgumentException("changed join key correspondence");
+ }
+ for (String side : List.of("leftFieldType", "rightFieldType")) {
+ if (pair.containsKey(side) && !"TableField".equals(pair.get(side))) {
+ throw new IllegalArgumentException("calculated join keys are unsupported");
+ }
+ }
+ resolve(List.of(new Identifier(left, true), new Identifier(leftKey, true)), true);
+ resolve(List.of(new Identifier(right, true), new Identifier(rightKey, true)), true);
+ }
+ }
+
+ private Set<String> reachableFrom(String start) {
+ Set<String> visited = new HashSet<>();
+ ArrayDeque<String> pending = new ArrayDeque<>();
+ pending.add(start);
+ while (!pending.isEmpty()) {
+ String dataset = pending.removeFirst();
+ if (visited.add(dataset)) pending.addAll(graph.getOrDefault(dataset, Set.of()));
+ }
+ return Set.copyOf(visited);
+ }
+
+ ResolvedField resolve(List<Identifier> parts, boolean tableau) {
+ return resolved.computeIfAbsent(new Reference(parts, tableau), key -> resolveUncached(key.parts(), key.tableau()));
+ }
+
+ private ResolvedField resolveUncached(List<Identifier> parts, boolean tableau) {
+ String reference = parts.stream().map(Identifier::text).collect(Collectors.joining("."));
+ if (parts.isEmpty() || parts.size() > 2) {
+ throw new IllegalArgumentException("Reference '" + reference
+ + "' must name a declared field or dataset.field; physical source paths are unsupported");
+ }
+ if (tableau && parts.size() != 2) {
+ throw new IllegalArgumentException("TABLEAU field reference '" + reference
+ + "' must use [dataset].[field]");
+ }
+
+ List<Map<String, Object>> candidates = datasets;
+ if (parts.size() == 2) {
+ candidates = datasets.stream()
+ .filter(dataset -> matches(parts.get(0), getString(dataset, "name"), tableau))
+ .toList();
+ if (candidates.isEmpty()) {
+ throw new IllegalArgumentException("Unknown dataset in reference '" + reference
+ + "'; use a declared dataset name, not its physical source");
+ }
+ if (candidates.size() > 1) {
+ throw new IllegalArgumentException("Ambiguous dataset in reference '" + reference
+ + "'; dataset declarations must have distinct names");
+ }
+ }
+
+ Identifier fieldName = parts.get(parts.size() - 1);
+ List<Field> fields = new ArrayList<>();
+ for (Map<String, Object> dataset : candidates) {
+ for (Map<String, Object> field : items(dataset, "fields")) {
+ if (matches(fieldName, getString(field, "name"), tableau)) {
+ fields.add(new Field(dataset, field));
+ }
+ }
+ }
+ if (fields.isEmpty()) {
+ throw new IllegalArgumentException("Unknown field reference '" + reference
+ + "'; declare the field under datasets[].fields before exporting the metric");
+ }
+ if (fields.size() > 1) {
+ throw new IllegalArgumentException("Ambiguous field reference '" + reference
+ + "'; qualify the dataset and remove duplicate field declarations");
+ }
+
+ Field match = fields.get(0);
+ String datasetName = getString(match.dataset(), "name");
+ // An unqualified field must not accidentally select one of two equivalent datasets.
+ if (datasets.stream().filter(dataset -> equivalentDeclaration(
+ datasetName, getString(dataset, "name"), tableau)).count() > 1) {
+ throw new IllegalArgumentException("Ambiguous dataset for reference '" + reference
+ + "'; dataset declarations must have distinct names");
+ }
+ String sourceFieldName = getString(match.field(), "name");
+ Map<String, Object> targetDataset = exportedItem(targetDatasets, datasetName,
+ "dataset", reference);
+ List<Map<String, Object>> targetFields = new ArrayList<>(items(targetDataset, "semanticDimensions"));
+ targetFields.addAll(items(targetDataset, "semanticMeasurements"));
+ Map<String, Object> targetField = exportedItem(targetFields, sourceFieldName, "field", reference);
+
+ String datatype = getString(match.field(), "datatype");
+ String targetType = getString(targetField, "dataType");
+ if (datatype == null || datatype.isBlank()) {
+ datatype = SalesforceDataTypeMapper.toOssie(targetType);
+ }
+ if (SalesforceDataTypeMapper.toSalesforce(datatype) == null) {
+ throw new IllegalArgumentException("Field reference '" + reference
+ + "' has no supported datatype; declare a portable field datatype");
+ }
+ if (targetType == null || targetType.isBlank()
+ || !SalesforceDataTypeMapper.areCompatible(datatype, targetType)) {
+ throw new IllegalArgumentException("Field reference '" + reference + "' has datatype '"
+ + datatype + "' but exported Salesforce dataType '" + targetType
+ + "'; use compatible field types");
+ }
+
+ validateDirectBinding(targetField, reference);
+ String targetDatasetName = getString(targetDataset, "apiName");
+ String targetFieldName = getString(targetField, "apiName");
+ return new ResolvedField(bracket(targetDatasetName) + "." + bracket(targetFieldName),
+ datatype, targetDatasetName);
+ }
+
+ private static void validateDirectBinding(Map<String, Object> targetField, String reference) {
+ String column = getString(targetField, "dataObjectFieldName");
+ try {
+ if (column == null || column.isBlank()) throw new IllegalArgumentException("missing physical column");
+ MetricExpression.Node parsed = SqlMetricExpressionParser.parse(column, "ANSI_SQL");
+ if (!(parsed instanceof MetricExpression.Field field) || field.parts().size() != 1
+ || field.parts().get(0).quoted() || !column.equals(field.parts().get(0).text())) {
+ throw new IllegalArgumentException("expected one unquoted physical column");
+ }
+ } catch (IllegalArgumentException e) {
+ throw new IllegalArgumentException("Field reference '" + reference
+ + "' was not exported with a supported direct physical binding; derived, qualified or quoted "
+ + "field bindings need separate conversion support: " + e.getMessage(), e);
+ }
+ }
+
+ private static Map<String, Object> exportedItem(List<Map<String, Object>> items,
+ String name, String kind, String reference) {
+ List<Map<String, Object>> matches = items.stream()
+ .filter(item -> name.equals(getString(item, "apiName"))).toList();
+ if (matches.isEmpty()) {
+ throw new IllegalArgumentException("Declared " + kind + " in reference '" + reference
+ + "' was not exported as a direct Salesforce semantic " + kind
+ + "; calculated or omitted fields are unsupported in metric references");
+ }
+ if (matches.size() > 1) {
+ throw new IllegalArgumentException("Ambiguous exported Salesforce " + kind + " for reference '"
+ + reference + "'; apiName values must be unique");
+ }
+ return matches.get(0);
+ }
+
+ private static boolean matches(Identifier reference, String declaration, boolean tableau) {
+ if (declaration == null) {
+ return false;
+ }
+ return tableau ? reference.text().equals(declaration)
+ : normalize(reference).equals(normalizeDeclaration(declaration));
+ }
+
+ private static boolean equivalentDeclaration(String first, String second, boolean tableau) {
+ return second != null && (tableau ? first.equals(second)
+ : normalizeDeclaration(first).equals(normalizeDeclaration(second)));
+ }
+
+ private static String normalize(Identifier identifier) {
+ return identifier.quoted() ? identifier.text() : identifier.text().toUpperCase(Locale.ROOT);
+ }
+
+ private static String normalizeDeclaration(String name) {
+ if (name.startsWith("\"") && name.endsWith("\"") && name.length() >= 2) {
+ String text = name.substring(1, name.length() - 1);
+ String unescaped = text.replace("\"\"", "");
+ if (text.isEmpty() || unescaped.contains("\"")) {
+ throw new IllegalArgumentException("Invalid quoted declaration name '" + name + "'");
+ }
+ return text.replace("\"\"", "\"");
+ }
+ return name.toUpperCase(Locale.ROOT);
+ }
+
+ private static String bracket(String name) {
+ if (name == null || name.isBlank() || name.indexOf('[') >= 0 || name.indexOf(']') >= 0
+ || name.chars().anyMatch(Character::isISOControl)) {
+ throw new IllegalArgumentException("Exported apiName '" + name
+ + "' cannot be represented safely in a TABLEAU field reference; rename it");
+ }
+ return "[" + name + "]";
+ }
+
+ private static List<Map<String, Object>> items(Map<String, Object> map, String key) {
+ List<Object> values = getList(map, key);
+ return values == null ? List.of() : streamMaps(values).toList();
+ }
+}
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricMappingHandler.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricMappingHandler.java
index 6add2b9b..5278abaf 100644
--- a/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricMappingHandler.java
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/MetricMappingHandler.java
@@ -24,6 +24,7 @@ import static org.apache.ossie.util.DataStructureUtils.*;
import org.apache.ossie.converter.ConverterConstants.Level;
import org.apache.ossie.converter.pipeline.PipelineStep;
+import org.apache.ossie.exception.ConversionException;
import java.util.*;
import org.apache.ossie.util.MappingUtils;
@@ -76,11 +77,29 @@ public class MetricMappingHandler implements PipelineStep {
return;
}
+ Set<String> names = new HashSet<>();
+ for (Object metric : ossieMetrics) {
+ String name = getString(asMap(metric), NAME);
+ if (!names.add(name)) {
+ throw new ConversionException("Metric '" + name + "': duplicate metric name");
+ }
+ }
+
// Filter mappings to get only metric-related entries
Map<String, String> metricMappings = MappingUtils.filterMappingsByPrefix(mappings, METRICS);
+
+ Map<String, Object> mappedData = GenericMappingEngine.applyMappings(sourceData, metricMappings);
metricMappings.keySet().forEach(mappings::remove);
- logger.debug("Metrics are not mapped in Ossie to Salesforce direction");
+ outputData.putAll(mappedData);
+
+ List<Object> sfMetrics = getList(outputData, SEMANTIC_CALCULATED_MEASUREMENTS);
+ if (sfMetrics != null) {
+ unwrapExpressions(ossieMetrics, sfMetrics, sourceData, outputData);
+ } else if (!ossieMetrics.isEmpty()) {
+ throw new ConversionException("Metric '" + getString(asMap(ossieMetrics.get(0)), NAME)
+ + "': metric mappings produced no calculated measurements");
+ }
}
/**
@@ -118,6 +137,29 @@ public class MetricMappingHandler implements PipelineStep {
}
+ /**
+ * Compiles each metric to Tua after fields have been mapped. Binding checks both
+ * the OSI declarations and the actual emitted fields, including their types.
+ */
+ private void unwrapExpressions(List<Object> ossieMetrics, List<Object> sfMetrics,
+ Map<String, Object> sourceData, Map<String, Object> outputData) {
+ if (ossieMetrics.size() != sfMetrics.size()) {
+ throw new ConversionException("Metric export count differs from declared metrics: "
+ + streamMaps(ossieMetrics).map(metric -> getString(metric, NAME)).toList());
+ }
+ MetricFieldResolver resolver = new MetricFieldResolver(sourceData, outputData);
+ for (int i = 0; i < ossieMetrics.size(); i++) {
+ Map<String, Object> ossieMetric = asMap(ossieMetrics.get(i));
+ Map<String, Object> sfMetric = asMap(sfMetrics.get(i));
+ MetricExpressionTranslator.Result translated =
+ MetricExpressionTranslator.translate(ossieMetric, resolver);
+ sfMetric.put(EXPRESSION, translated.expression());
+ sfMetric.put(DATA_TYPE, translated.dataType());
+ sfMetric.put("syntax", "Tua");
+ sfMetric.put("aggregationType", "UserAgg");
+ }
+ }
+
/**
* Wraps expressions for SF→Ossie conversion.
*/
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/SqlMetricExpressionParser.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/SqlMetricExpressionParser.java
new file mode 100644
index 00000000..3464ae33
--- /dev/null
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/SqlMetricExpressionParser.java
@@ -0,0 +1,250 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.apache.ossie.converter.MetricExpression.*;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+import net.sf.jsqlparser.expression.*;
+import net.sf.jsqlparser.expression.operators.relational.IsNullExpression;
+import net.sf.jsqlparser.expression.operators.relational.ParenthesedExpressionList;
+import net.sf.jsqlparser.parser.CCJSqlParserUtil;
+import net.sf.jsqlparser.schema.Column;
+import net.sf.jsqlparser.statement.select.AllColumns;
+import org.apache.ossie.converter.TuaMetricExpressionParser.Kind;
+import org.apache.ossie.converter.TuaMetricExpressionParser.Token;
+
+/** Adapts a completely consumed JSqlParser expression into the explicitly supported compiler AST. */
+final class SqlMetricExpressionParser {
+ private final String dialect;
+ private int depth;
+ private SqlMetricExpressionParser(String dialect) { this.dialect = dialect; }
+
+ static Node parse(String text, String dialect) {
+ text = normalize(text, dialect);
+ try {
+ Expression expression = CCJSqlParserUtil.parseCondExpression(text, false,
+ parser -> parser.withSquareBracketQuotation(dialect.equals("ANSI_SQL")));
+ if (expression == null) throw new IllegalArgumentException("could not parse a complete SQL expression");
+ return new SqlMetricExpressionParser(dialect).adapt(expression);
+ } catch (net.sf.jsqlparser.JSQLParserException e) {
+ throw new IllegalArgumentException(dialect + " expression has unsupported or unexpected token: "
+ + e.getMessage(), e);
+ } catch (StackOverflowError e) {
+ throw new IllegalArgumentException("expression nesting exceeds parser limits", e);
+ }
+ }
+
+ /** Simplifies redundant syntax within the original input bounds, without reparsing SQL. */
+ private static String normalize(String text, String dialect) {
+ List<Token> tokens = TuaMetricExpressionParser.tokenize(text, dialect);
+ int[] closing = new int[tokens.size()];
+ int[] openings = new int[128];
+ int nesting = 0;
+ for (int i = 0; i < tokens.size() - 1; i++) {
+ if (symbol(tokens.get(i), "(")) openings[nesting++] = i;
+ if (symbol(tokens.get(i), ")")) {
+ if (nesting == 0) throw new IllegalArgumentException("unexpected closing parenthesis");
+ closing[openings[--nesting]] = i;
+ }
+ }
+ boolean[] redundant = new boolean[tokens.size()];
+ for (int i = 1; i < tokens.size() - 2; i++) {
+ // Only remove a group inside another opening parenthesis. Keep the
+ // function argument list and innermost group, including tuple/modifier syntax.
+ if (symbol(tokens.get(i - 1), "(") && symbol(tokens.get(i), "(")
+ && symbol(tokens.get(i + 1), "(") && closing[i] == closing[i + 1] + 1) {
+ redundant[i] = redundant[closing[i]] = true;
+ }
+ }
+ StringBuilder result = new StringBuilder(text.length());
+ for (int i = 0; i < tokens.size() - 1;) {
+ if (redundant[i]) { i++; continue; }
+ Token token = tokens.get(i);
+ boolean sign = symbol(token, "+") || symbol(token, "-");
+ boolean not = token.kind() == Kind.WORD && token.text().equalsIgnoreCase("NOT");
+ // Preserve binary +/- and predicate modifiers such as IS NOT. Only
+ // prefix operators can be simplified; malformed modifiers must still fail.
+ if ((not || sign) && (i == 0 || startsOperandAfter(tokens.get(i - 1)))) {
+ int end = i;
+ int negatives = 0;
+ while (end < tokens.size() - 1) {
+ Token next = tokens.get(end);
+ if (not ? next.kind() != Kind.WORD || !next.text().equalsIgnoreCase("NOT")
+ : !symbol(next, "+") && !symbol(next, "-")) break;
+ if (symbol(next, "-")) negatives++;
+ if (++end - i > 128) throw new IllegalArgumentException("too many unary operators");
+ }
+ // Retain a unary operation even when parity is even: dropping all
+ // operators would bypass numeric/Boolean checks and COUNT(field) rules.
+ result.append(not ? (end - i) % 2 == 0 ? "NOT NOT " : "NOT "
+ : negatives % 2 == 0 ? "+ " : "- ");
+ i = end;
+ } else {
+ // Raw slices preserve escaped strings and quoted identifier spelling.
+ result.append(text, token.offset(), tokens.get(i + 1).offset()).append(' ');
+ i++;
+ }
+ }
+ return result.toString();
+ }
+
+ private static boolean symbol(Token token, String value) {
+ return token.kind() == Kind.SYMBOL && token.text().equals(value);
+ }
+
+ private static boolean startsOperandAfter(Token token) {
+ return token.kind() == Kind.SYMBOL
+ && Set.of("(", ",", "+", "-", "*", "/", "=", "!=", "<>", "<", "<=", ">", ">=").contains(token.text())
+ || token.kind() == Kind.WORD
+ && Set.of("WHEN", "THEN", "ELSE", "AND", "OR", "NOT", "DISTINCT").contains(token.text().toUpperCase(Locale.ROOT));
+ }
+
+ private Node adapt(Expression expression) {
+ if (++depth > 128) throw new IllegalArgumentException("expression nesting exceeds 128 levels");
+ try { return adaptNode(expression); }
+ finally { depth--; }
+ }
+
+ private Node adaptNode(Expression expression) {
+ if (expression instanceof ParenthesedExpressionList<?> list && list.size() == 1) {
+ return adapt(list.get(0));
+ }
+ if (expression instanceof LongValue || expression instanceof DoubleValue) {
+ return new Literal(TuaMetricExpressionParser.number(expression.toString()));
+ }
+ if (expression instanceof NullValue) return new Literal(null);
+ if (expression instanceof BooleanValue value) return new Literal(value.getValue());
+ if (expression instanceof StringValue value) {
+ if (value.getPrefix() != null) throw unsupported(expression);
+ return new Literal(value.getValue().replace("''", "'"));
+ }
+ if (expression instanceof Column column) return column(column);
+ if (expression instanceof SignedExpression signed) {
+ if (signed.getSign() != '+' && signed.getSign() != '-') throw unsupported(expression);
+ return new Unary(String.valueOf(signed.getSign()), adapt(signed.getExpression()));
+ }
+ if (expression instanceof NotExpression not) {
+ if (not.isExclamationMark()) throw unsupported(expression);
+ return new Unary("NOT", adapt(not.getExpression()));
+ }
+ if (expression instanceof IsNullExpression test) {
+ if (test.isUseIsNull() || test.isUseNotNull()) throw unsupported(expression);
+ Node result = new Unary("ISNULL", adapt(test.getLeftExpression()));
+ return test.isNot() ? new Unary("NOT", result) : result;
+ }
+ if (expression instanceof BinaryExpression binary) {
+ if (binary instanceof net.sf.jsqlparser.expression.operators.relational.SupportsOldOracleJoinSyntax oracle
+ && (oracle.getOldOracleJoinSyntax() != 0 || oracle.getOraclePriorPosition() != 0)) {
+ throw unsupported(expression);
+ }
+ String operator = binary.getStringExpression().toUpperCase(Locale.ROOT);
+ if (!Set.of("+", "-", "*", "/", "AND", "OR", "=", "!=", "<>", "<", ">", "<=", ">=").contains(operator)) {
+ throw unsupported(expression);
+ }
+ return new Binary(operator.equals("<>") ? "!=" : operator,
+ adapt(binary.getLeftExpression()), adapt(binary.getRightExpression()));
+ }
+ if (expression instanceof Function function) return function(function);
+ if (expression instanceof CaseExpression conditional) {
+ List<Node> branches = new ArrayList<>();
+ // The original bounded contract supports searched CASE. Simple CASE can be
+ // added with explicit type checking and evaluation-count guarantees later.
+ if (conditional.getSwitchExpression() != null) throw unsupported(expression);
+ for (WhenClause branch : conditional.getWhenClauses()) {
+ branches.add(adapt(branch.getWhenExpression()));
+ branches.add(adapt(branch.getThenExpression()));
+ }
+ return new Conditional(branches, conditional.getElseExpression() == null
+ ? new Literal(null) : adapt(conditional.getElseExpression()));
+ }
+ if (expression instanceof AllColumns) {
+ throw new IllegalArgumentException("COUNT(*) is unsupported; name a declared field to count");
+ }
+ throw unsupported(expression);
+ }
+
+ private Node function(Function function) {
+ String name = function.getName();
+ if (name == null || !name.matches("[A-Za-z_][A-Za-z_0-9]*")) {
+ throw new IllegalArgumentException("quoted or qualified function names are unsupported");
+ }
+ if (function.isUnique() || function.isEscaped() || function.getNamedParameters() != null
+ || function.getAttribute() != null || function.getKeep() != null
+ || function.getNullHandling() != null || function.isIgnoreNullsOutside()
+ || function.isIgnoreNulls() || function.getLimit() != null
+ || function.getHavingClause() != null || function.getExtraKeyword() != null
+ || function.getOnOverflowTruncate() != null
+ || function.getOrderByElements() != null && !function.getOrderByElements().isEmpty()) {
+ throw new IllegalArgumentException("unsupported function modifiers for " + name);
+ }
+ if (function.isAllColumns()) {
+ throw new IllegalArgumentException("explicit ALL function modifier is outside the supported SQL subset");
+ }
+ if (function.getParameters() instanceof ParenthesedExpressionList<?> grouped && grouped.size() != 1) {
+ throw new IllegalArgumentException("tuple-valued function arguments are unsupported");
+ }
+ List<Node> arguments = new ArrayList<>();
+ if (function.getParameters() != null) {
+ for (Expression argument : function.getParameters()) arguments.add(adapt(argument));
+ }
+ return new Call(name.toUpperCase(Locale.ROOT), arguments, function.isDistinct());
+ }
+
+ private Node column(Column column) {
+ if (column.getArrayConstructor() != null) throw unsupported(column);
+ // JSqlParser 5.3's Table accessors split a quoted name containing a dot
+ // ("Order.Items") into schema/table parts. Retain the original token
+ // boundaries instead of binding that expression to a different object.
+ List<String> raw = new ArrayList<>();
+ var source = column.getASTNode();
+ if (source == null) throw new IllegalArgumentException("field reference has no source identifier tokens");
+ boolean identifier = true;
+ for (var token = source.jjtGetFirstToken(); token != null; token = token.next) {
+ if (identifier) raw.add(token.image);
+ else if (!token.image.equals(".")) throw unsupported(column);
+ identifier = !identifier;
+ if (token == source.jjtGetLastToken()) break;
+ }
+ if (raw.isEmpty() || identifier) throw unsupported(column);
+ boolean bracket = raw.get(0).startsWith("[");
+ List<MetricFieldResolver.Identifier> parts = new ArrayList<>();
+ for (String part : raw) {
+ if (bracket != part.startsWith("[")) {
+ throw new IllegalArgumentException("do not mix bracketed and SQL field identifiers");
+ }
+ boolean quoted = part.startsWith("\"") || part.startsWith("[");
+ if (quoted) {
+ String end = bracket ? "]" : "\"";
+ part = part.substring(1, part.length() - 1).replace(end + end, end);
+ }
+ parts.add(new MetricFieldResolver.Identifier(part, quoted));
+ }
+ return new Field(parts, bracket);
+ }
+
+ private IllegalArgumentException unsupported(Expression expression) {
+ return new IllegalArgumentException(dialect + " unsupported SQL expression "
+ + expression.getClass().getSimpleName()
+ + "; only documented expression capabilities can be converted");
+ }
+}
diff --git a/converters/salesforce/src/main/java/org/apache/ossie/converter/TuaMetricExpressionParser.java b/converters/salesforce/src/main/java/org/apache/ossie/converter/TuaMetricExpressionParser.java
new file mode 100644
index 00000000..1472c592
--- /dev/null
+++ b/converters/salesforce/src/main/java/org/apache/ossie/converter/TuaMetricExpressionParser.java
@@ -0,0 +1,230 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.apache.ossie.converter.MetricExpression.*;
+import java.math.BigDecimal;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Locale;
+import java.util.Set;
+
+/** Native Tua frontend for the bounded supported grammar; produces the same AST as SQL. */
+final class TuaMetricExpressionParser {
+ private final List<Token> tokens;
+ private int position;
+ private int depth;
+ TuaMetricExpressionParser(List<Token> tokens) { this.tokens = tokens; }
+
+ Node parse() {
+ Node node = expression();
+ if (peek().kind() != Kind.END) throw error("unsupported or unexpected token '" + peek().text() + "'");
+ return node;
+ }
+ private Node expression() {
+ if (++depth > 128) throw error("expression nesting exceeds 128 levels");
+ try { return or(); } finally { depth--; }
+ }
+ private Node or() {
+ Node value = and();
+ while (take("OR")) value = new Binary("OR", value, and());
+ return value;
+ }
+ private Node and() {
+ Node value = not();
+ while (take("AND")) value = new Binary("AND", value, not());
+ return value;
+ }
+ private Node not() {
+ int count = 0;
+ while (take("NOT")) if (++count > 128) throw error("too many unary operators");
+ Node value = comparison();
+ while (count-- > 0) value = new Unary("NOT", value);
+ return value;
+ }
+ private Node comparison() {
+ Node value = additive();
+ if (at("IS")) throw error("use ISNULL in TABLEAU expressions");
+ if (Set.of("=", "!=", "<>", "<", "<=", ">", ">=").contains(peek().text())) {
+ String op = next().text();
+ return new Binary(op.equals("<>") ? "!=" : op, value, additive());
+ }
+ return value;
+ }
+ private Node additive() {
+ Node value = multiplicative();
+ while (at("+") || at("-")) value = new Binary(next().text(), value, multiplicative());
+ return value;
+ }
+ private Node multiplicative() {
+ Node value = unary();
+ while (at("*") || at("/")) value = new Binary(next().text(), value, unary());
+ return value;
+ }
+ private Node unary() {
+ List<String> signs = new ArrayList<>();
+ while (at("+") || at("-")) {
+ if (signs.size() >= 128) throw error("too many unary operators");
+ signs.add(next().text());
+ }
+ Node value = primary();
+ for (int i = signs.size() - 1; i >= 0; i--) value = new Unary(signs.get(i), value);
+ return value;
+ }
+ private Node primary() {
+ if (take("(")) { Node value = expression(); expect(")"); return value; }
+ if (at("CASE")) throw error("searched CASE is SQL; use IF in TABLEAU");
+ if (take("IF")) return conditional();
+ if (take("NULL")) return new Literal(null);
+ if (at("TRUE") || at("FALSE")) return new Literal(Boolean.valueOf(next().text()));
+ Token token = next();
+ if (token.kind() == Kind.NUMBER) return new Literal(number(token.text()));
+ if (token.kind() == Kind.STRING) return new Literal(token.text());
+ if (token.kind() != Kind.WORD && token.kind() != Kind.IDENTIFIER) {
+ throw error("expected a value, found '" + token.text() + "'");
+ }
+ if (take("(")) {
+ if (token.kind() != Kind.WORD) throw error("quoted function names are unsupported");
+ return function(token.text().toUpperCase(Locale.ROOT));
+ }
+ List<MetricFieldResolver.Identifier> parts = new ArrayList<>();
+ addIdentifier(parts, token);
+ while (take(".")) addIdentifier(parts, next());
+ if (parts.size() != 2) throw error("TABLEAU fields must use [dataset].[field] notation");
+ return new Field(parts, true);
+ }
+ private void addIdentifier(List<MetricFieldResolver.Identifier> parts, Token token) {
+ if (token.kind() != Kind.IDENTIFIER || !token.bracket()) {
+ throw error("TABLEAU fields must use [dataset].[field] notation");
+ }
+ parts.add(new MetricFieldResolver.Identifier(token.text(), true));
+ }
+ private Node conditional() {
+ List<Node> branches = new ArrayList<>();
+ do {
+ branches.add(expression()); expect("THEN"); branches.add(expression());
+ } while (take("ELSEIF"));
+ Node otherwise = take("ELSE") ? expression() : new Literal(null);
+ expect("END");
+ return new Conditional(branches, otherwise);
+ }
+ private Node function(String name) {
+ boolean distinct = take("DISTINCT");
+ if (at("*")) throw error("COUNT(*) is unsupported; name a declared field to count");
+ List<Node> arguments = new ArrayList<>();
+ if (!at(")")) do { arguments.add(expression()); } while (take(","));
+ expect(")");
+ return new Call(name, arguments, distinct);
+ }
+ private Token peek() { return tokens.get(position); }
+ private Token next() { Token token = peek(); if (token.kind() != Kind.END) position++; return token; }
+ private boolean at(String text) {
+ return (peek().kind() == Kind.WORD || peek().kind() == Kind.SYMBOL) && peek().text().equalsIgnoreCase(text);
+ }
+ private boolean take(String text) { if (!at(text)) return false; next(); return true; }
+ private void expect(String text) { if (!take(text)) throw error("expected " + text + ", found '" + peek().text() + "'"); }
+ private IllegalArgumentException error(String message) {
+ return new IllegalArgumentException("TABLEAU at character " + (peek().offset() + 1) + ": " + message);
+ }
+
+ // Preflight bounds SQL parsing too; native parsing also consumes these tokens.
+ enum Kind { WORD, IDENTIFIER, STRING, NUMBER, SYMBOL, END }
+ record Token(Kind kind, String text, int offset, boolean bracket) {}
+ static List<Token> tokenize(String text, String dialect) {
+ if (text.length() > 32768) throw new IllegalArgumentException("expression exceeds 32768 characters");
+ List<Token> tokens = new ArrayList<>();
+ int nesting = 0;
+ for (int i = 0; i < text.length();) {
+ char c = text.charAt(i);
+ if (Character.isWhitespace(c)) { i++; continue; }
+ int start = i;
+ if (c == '\'' || c == '"' || c == '[') {
+ if (c == '[' && dialect.equals("SNOWFLAKE")) throw lexical(dialect, i, "use double-quoted SQL identifiers");
+ boolean string = c == '\'' || (c == '"' && dialect.equals("TABLEAU"));
+ char end = c == '[' ? ']' : c;
+ StringBuilder value = new StringBuilder();
+ boolean closed = false;
+ i++;
+ while (i < text.length()) {
+ char part = text.charAt(i++);
+ if (part == end) {
+ if (i < text.length() && text.charAt(i) == end) { value.append(end); i++; }
+ else { closed = true; break; }
+ } else {
+ if (Character.isISOControl(part) || (string && part == '\\')) {
+ throw lexical(dialect, i - 1, "control characters and backslash string escapes are unsupported");
+ }
+ value.append(part);
+ }
+ }
+ if (!closed) throw lexical(dialect, start, "unterminated quoted value");
+ tokens.add(new Token(string ? Kind.STRING : Kind.IDENTIFIER, value.toString(), start, c == '['));
+ } else if (Character.isDigit(c) || (c == '.' && i + 1 < text.length() && Character.isDigit(text.charAt(i + 1)))) {
+ i++;
+ while (i < text.length() && (Character.isDigit(text.charAt(i)) || text.charAt(i) == '.')) i++;
+ if (i < text.length() && (text.charAt(i) == 'e' || text.charAt(i) == 'E')) {
+ i++;
+ if (i < text.length() && (text.charAt(i) == '+' || text.charAt(i) == '-')) i++;
+ while (i < text.length() && Character.isDigit(text.charAt(i))) i++;
+ }
+ tokens.add(new Token(Kind.NUMBER, text.substring(start, i), start, false));
+ } else if (Character.isLetter(c) || c == '_') {
+ i++;
+ while (i < text.length() && (Character.isLetterOrDigit(text.charAt(i)) || text.charAt(i) == '_' || text.charAt(i) == '$')) i++;
+ tokens.add(new Token(Kind.WORD, text.substring(start, i), start, false));
+ } else {
+ if (i + 1 < text.length() && (text.startsWith("--", i) || text.startsWith("/*", i))) {
+ throw lexical(dialect, i, "comments are unsupported in metric expressions");
+ }
+ String symbol = String.valueOf(c);
+ if (i + 1 < text.length() && Set.of("<=", ">=", "<>", "!=").contains(text.substring(i, i + 2))) {
+ symbol = text.substring(i, i + 2);
+ i++;
+ }
+ if (!"()+-*/.,=<>!".contains(String.valueOf(c))) throw lexical(dialect, start, "unsupported character '" + c + "'");
+ tokens.add(new Token(Kind.SYMBOL, symbol, start, false));
+ i++;
+ }
+ Token added = tokens.get(tokens.size() - 1);
+ if (added.kind() == Kind.NUMBER) number(added.text());
+ if (added.kind() == Kind.SYMBOL && added.text().equals("(") && ++nesting > 128) {
+ throw lexical(dialect, start, "expression nesting exceeds 128 levels");
+ }
+ if (added.kind() == Kind.SYMBOL && added.text().equals(")")) nesting--;
+ if (tokens.size() > 8192) throw lexical(dialect, start, "too many expression tokens");
+ }
+ if (nesting > 0) throw lexical(dialect, text.length(), "expected )");
+ tokens.add(new Token(Kind.END, "end of expression", text.length(), false));
+ return tokens;
+ }
+
+ private static IllegalArgumentException lexical(String dialect, int offset, String message) {
+ return new IllegalArgumentException(dialect + " at character " + (offset + 1) + ": " + message);
+ }
+ static BigDecimal number(String text) {
+ BigDecimal value;
+ try { value = new BigDecimal(text); }
+ catch (NumberFormatException e) { throw new IllegalArgumentException("invalid numeric literal '" + text + "'"); }
+ if (Math.abs((long) value.scale()) > 1000 || value.precision() > 1000) {
+ throw new IllegalArgumentException("numeric literal is too large");
+ }
+ return value;
+ }
+}
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/MetricCliTest.java b/converters/salesforce/src/test/java/org/apache/ossie/MetricCliTest.java
new file mode 100644
index 00000000..93528956
--- /dev/null
+++ b/converters/salesforce/src/test/java/org/apache/ossie/MetricCliTest.java
@@ -0,0 +1,60 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie;
+
+import static org.junit.jupiter.api.Assertions.*;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.concurrent.TimeUnit;
+import org.apache.ossie.app.OssieSalesforceConverter;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+class MetricCliTest {
+ @TempDir
+ Path directory;
+
+ @Test
+ void reportsMetricFailureToStderrWithoutWritingAModel() throws Exception {
+ Path input = directory.resolve("input.yaml");
+ Files.writeString(input, Files.readString(Path.of("src/test/resources/examples/ossieToSalesforce.yaml"))
+ .replace("SUM([Orders].[amount])", "SUM([Orders].[missing])"));
+ Path stderr = directory.resolve("stderr.txt");
+ Process process = new ProcessBuilder(
+ Path.of(System.getProperty("java.home"), "bin", "java").toString(),
+ "-cp", System.getProperty("java.class.path"), OssieSalesforceConverter.class.getName(),
+ "toSF", input.toString())
+ .redirectError(stderr.toFile())
+ .redirectOutput(directory.resolve("stdout.txt").toFile())
+ .start();
+ try {
+ assertTrue(process.waitFor(30, TimeUnit.SECONDS), "CLI did not terminate");
+ assertEquals(3, process.exitValue());
+ String error = Files.readString(stderr);
+ assertTrue(error.contains("Metric 'total_revenue'"), error);
+ assertTrue(error.contains("Unknown field reference"), error);
+ assertTrue(error.contains("missing"), error);
+ assertFalse(Files.exists(directory.resolve("Customer_Orders_Model.json")));
+ } finally {
+ process.destroyForcibly();
+ }
+ }
+}
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/MetricExportIntegrationTest.java b/converters/salesforce/src/test/java/org/apache/ossie/MetricExportIntegrationTest.java
new file mode 100644
index 00000000..2bb1ae41
--- /dev/null
+++ b/converters/salesforce/src/test/java/org/apache/ossie/MetricExportIntegrationTest.java
@@ -0,0 +1,427 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
+import org.apache.ossie.converter.ConversionDirection;
+import org.apache.ossie.converter.Converter;
+import org.apache.ossie.converter.ConverterFactory;
+import org.apache.ossie.exception.ConversionException;
+import org.apache.ossie.exception.ValidationException;
+import org.apache.ossie.validator.SchemaValidator;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.*;
+import static org.junit.jupiter.api.Assumptions.assumeTrue;
+
+/** Exercises metric compilation through the public converter and target schema. */
+class MetricExportIntegrationTest {
+ private static final ObjectMapper JSON = new ObjectMapper();
+ private static final ObjectMapper YAML = new ObjectMapper(new YAMLFactory());
+ private static boolean salesforceSchemaExists;
+ private static boolean ossieSchemaExists;
+
+ private Converter converter;
+
+ @TempDir
+ Path temporaryDirectory;
+
+ @BeforeAll
+ static void checkSchemaAvailability() {
+ salesforceSchemaExists = MetricExportIntegrationTest.class
+ .getResource(SchemaValidator.SALESFORCE_SCHEMA_PATH) != null;
+ ossieSchemaExists = MetricExportIntegrationTest.class
+ .getResource(SchemaValidator.OSSIE_SCHEMA_PATH) != null;
+ if (Boolean.getBoolean("requireSalesforceSchema")) {
+ assertTrue(salesforceSchemaExists,
+ "-DrequireSalesforceSchema=true requires the Salesforce schema; see README setup instructions");
+ }
+ }
+
+ @BeforeEach
+ void setUp() {
+ assumeTrue(ossieSchemaExists, "Ossie schema is required; see README setup instructions");
+ converter = ConverterFactory.getConverter(ConversionDirection.OSSIE_TO_SALESFORCE);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void translatesComposedMetricsThroughStringApi(String dialect) throws Exception {
+ Map<String, Object> output = convertOne(model("sales", List.of(
+ metric("margin", dialect, "SUM(orders.profit) / NULLIF(SUM(orders.revenue), 0)"),
+ metric("adjusted_average", dialect,
+ "ROUND(AVG(CASE WHEN orders.revenue IS NOT NULL AND NOT orders.profit < 0 "
+ + "THEN COALESCE(orders.profit, 0) ELSE 0 END), 2)"),
+ metric("customers", dialect, "COUNT(DISTINCT orders.customer_id)"))));
+
+ List<Map<String, Object>> measurements = measurements(output);
+ assertEquals(List.of("margin", "adjusted_average", "customers"),
+ measurements.stream().map(item -> item.get("apiName")).toList());
+ for (Map<String, Object> measurement : measurements) {
+ assertMeasurementMetadata(measurement);
+ }
+ assertEquals("(SUM([orders].[profit]) / (IF (SUM([orders].[revenue]) = 0) "
+ + "THEN NULL ELSE SUM([orders].[revenue]) END))",
+ measurements.get(0).get("expression"));
+ String conditional = (String) measurements.get(1).get("expression");
+ assertTrue(conditional.startsWith("ROUND(AVG((IF "), conditional);
+ assertTrue(conditional.contains("ISNULL([orders].[revenue])"), conditional);
+ assertTrue(conditional.contains("IFNULL([orders].[profit], 0)"), conditional);
+ assertFalse(conditional.contains("CASE"), conditional);
+ assertEquals("COUNTD([orders].[customer_id])", measurements.get(2).get("expression"));
+
+ Map<String, Object> dataset = items(output, "semanticDataObjects").get(0);
+ Map<String, Object> profit = items(dataset, "semanticMeasurements").stream()
+ .filter(item -> item.get("apiName").equals("profit")).findFirst().orElseThrow();
+ assertEquals("profit__c", profit.get("dataObjectFieldName"));
+ assertFalse(measurements.get(0).get("expression").toString().contains("profit__c"),
+ "A metric binds semantic field names, not physical source columns");
+ }
+
+ @Test
+ void fileApiWritesTheSameCompleteModelAsStringApi() throws Exception {
+ String input = document(List.of(model("sales", List.of(
+ metric("revenue", "SNOWFLAKE", "SUM(orders.revenue)")))));
+ Path source = temporaryDirectory.resolve("input.yaml");
+ Path outputDirectory = Files.createDirectory(temporaryDirectory.resolve("output"));
+ Files.writeString(source, input);
+
+ converter.convert(source, outputDirectory);
+
+ assertEquals(JSON.readTree(converter.convert(input).get(0)),
+ JSON.readTree(Files.readString(outputDirectory.resolve("sales.json"))));
+ try (var files = Files.list(outputDirectory)) {
+ assertEquals(List.of("sales.json"), files.map(path -> path.getFileName().toString()).toList());
+ }
+ }
+
+ @Test
+ void missingFieldFailsWithMetricNameAndDoesNotWriteAnyModelFile() throws Exception {
+ String input = document(List.of(
+ model("invalid", List.of(metric("broken_margin", "SNOWFLAKE", "SUM(orders.missing)")))));
+ Path source = temporaryDirectory.resolve("input.yaml");
+ Path outputDirectory = Files.createDirectory(temporaryDirectory.resolve("output"));
+ Path existing = outputDirectory.resolve("existing.json");
+ Files.writeString(source, input);
+ Files.writeString(existing, "preserve this file");
+
+ ConversionException error = assertThrows(ConversionException.class,
+ () -> converter.convert(source, outputDirectory));
+
+ assertTrue(error.getMessage().contains("broken_margin"), error.getMessage());
+ assertTrue(error.getMessage().contains("orders.missing"), error.getMessage());
+ assertTrue(error.getMessage().contains("declare"), error.getMessage());
+ assertEquals("preserve this file", Files.readString(existing));
+ try (var files = Files.list(outputDirectory)) {
+ assertEquals(List.of("existing.json"), files.map(path -> path.getFileName().toString()).toList(),
+ "A failed conversion must not leave partial output behind");
+ }
+ }
+
+ @Test
+ void selectedUnsupportedTableauExpressionDoesNotFallBackToSql() throws Exception {
+ Map<String, Object> metric = metric("chosen_tableau", "TABLEAU", "BOGUS([orders].[profit])");
+ metric.put("expression", Map.of("dialects", List.of(
+ dialect("ANSI_SQL", "SUM(orders.profit)"),
+ dialect("TABLEAU", "BOGUS([orders].[profit])"),
+ dialect("SNOWFLAKE", "SUM(orders.profit)"))));
+ String input = document(List.of(model("sales", List.of(metric))));
+
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(input));
+
+ assertTrue(error.getMessage().contains("chosen_tableau"), error.getMessage());
+ assertTrue(error.getMessage().contains("BOGUS"), error.getMessage());
+ }
+
+ @Test
+ void preservesSupportedTableauExpressionMeaningAndValidatesReferences() throws Exception {
+ Map<String, Object> output = convertOne(model("sales", List.of(
+ metric("revenue", "TABLEAU", "IFNULL(SUM([orders].[revenue]), 0)"))));
+
+ Map<String, Object> measurement = measurements(output).get(0);
+ assertEquals("IFNULL(SUM([orders].[revenue]), 0)", measurement.get("expression"));
+ assertMeasurementMetadata(measurement);
+
+ String invalid = document(List.of(model("sales", List.of(
+ metric("unknown_tableau", "TABLEAU", "SUM([orders].[missing])")))));
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(invalid));
+ assertTrue(error.getMessage().contains("unknown_tableau"), error.getMessage());
+ assertTrue(error.getMessage().contains("orders.missing"), error.getMessage());
+ }
+
+ @Test
+ void modelWithoutMetricsRetainsDatasetExport() throws Exception {
+ Map<String, Object> source = model("sales", List.of());
+ source.remove("metrics");
+
+ Map<String, Object> output = convertOne(source);
+
+ assertTrue(measurements(output).isEmpty());
+ assertEquals(3, items(items(output, "semanticDataObjects").get(0), "semanticMeasurements").size());
+ }
+
+ @Test
+ void emptyMetricsRemainEmpty() throws Exception {
+ Map<String, Object> output = convertOne(model("sales", List.of()));
+
+ assertTrue(measurements(output).isEmpty());
+ assertEquals("sales", output.get("apiName"));
+ }
+
+ @Test
+ void metricNamesAndFieldTypesAreIsolatedAcrossSemanticModels() throws Exception {
+ Map<String, Object> first = model("first", List.of(
+ metric("value", "SNOWFLAKE", "SUM(orders.profit)")));
+ Map<String, Object> second = model("second", List.of(
+ metric("value", "ANSI_SQL", "COUNT(orders.profit)")));
+ items(items(second, "datasets").get(0), "fields").get(0).put("datatype", "String");
+ items(items(second, "datasets").get(0), "fields").get(0).put("dimension", Map.of("is_time", false));
+
+ Map<String, Object> firstOutput = convertOne(first);
+ Map<String, Object> secondOutput = convertOne(second);
+
+ assertEquals("SUM([orders].[profit])", measurements(firstOutput).get(0).get("expression"));
+ assertEquals("COUNT([orders].[profit])", measurements(secondOutput).get(0).get("expression"));
+ assertEquals("first", firstOutput.get("apiName"));
+ assertEquals("second", secondOutput.get("apiName"));
+ }
+
+ @Test
+ void fieldsFromAnotherSemanticModelCannotSatisfyAMetricReference() throws Exception {
+ Map<String, Object> first = model("first", List.of(
+ metric("valid", "ANSI_SQL", "SUM(orders.profit)")));
+ Map<String, Object> second = model("second", List.of(
+ metric("must_not_leak", "ANSI_SQL", "SUM(orders.profit)")));
+ items(second, "datasets").get(0).put("fields", List.of(field("revenue", "Decimal")));
+ assertDoesNotThrow(() -> convertOne(first));
+ String input = document(second);
+
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(input));
+
+ assertTrue(error.getMessage().contains("must_not_leak"), error.getMessage());
+ assertTrue(error.getMessage().contains("orders.profit"), error.getMessage());
+ }
+
+ @Test
+ void declaredButOmittedCalculatedSqlFieldCannotSatisfyAMetricReference() throws Exception {
+ Map<String, Object> source = model("sales", List.of());
+ Map<String, Object> calculated = field("adjusted", "Decimal");
+ calculated.put("expression", Map.of("dialects", List.of(dialect("ANSI_SQL", "profit__c + 1"))));
+ items(source, "datasets").get(0).put("fields", List.of(field("profit", "Decimal"), calculated));
+ Map<String, Object> output = convertOne(source);
+ assertEquals(List.of("profit"), items(items(output, "semanticDataObjects").get(0),
+ "semanticMeasurements").stream().map(item -> item.get("apiName")).toList());
+
+ source.put("metrics", List.of(metric("adjusted_total", "ANSI_SQL", "SUM(orders.adjusted)")));
+ String input = document(List.of(source));
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(input));
+
+ assertTrue(error.getMessage().contains("adjusted_total"), error.getMessage());
+ assertTrue(error.getMessage().contains("orders.adjusted"), error.getMessage());
+ assertTrue(error.getMessage().contains("not exported"), error.getMessage());
+ }
+
+ @Test
+ void duplicateMetricNamesAreRejectedBeforeExport() throws Exception {
+ String input = document(List.of(model("sales", List.of(
+ metric("duplicated", "ANSI_SQL", "SUM(orders.profit)"),
+ metric("duplicated", "ANSI_SQL", "SUM(orders.revenue)")))));
+
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(input));
+
+ assertTrue(error.getMessage().contains("duplicated"), error.getMessage());
+ assertTrue(error.getMessage().contains("duplicate metric"), error.getMessage());
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"AVG(orders.quantity)", "SUM(orders.quantity) / 2", "1.5"})
+ void integerMetricDeclarationRejectsFractionalResult(String expression) throws Exception {
+ Map<String, Object> metric = metric("integer_result", "ANSI_SQL", expression);
+ metric.put("datatype", "Integer");
+ String input = document(List.of(model("sales", List.of(metric))));
+
+ ConversionException error = assertThrows(ConversionException.class, () -> converter.convert(input));
+
+ assertTrue(error.getMessage().contains("integer_result"), error.getMessage());
+ assertTrue(error.getMessage().contains("Integer"), error.getMessage());
+ assertTrue(error.getMessage().contains("incompatible"), error.getMessage());
+ }
+
+ @Test
+ void integerCountIsExportedAsSalesforceNumber() throws Exception {
+ Map<String, Object> metric = metric("customer_count", "ANSI_SQL", "COUNT(orders.customer_id)");
+ metric.put("datatype", "Integer");
+
+ Map<String, Object> output = convertOne(model("sales", List.of(metric)));
+
+ assertMeasurementMetadata(measurements(output).get(0));
+ assertEquals("COUNT([orders].[customer_id])", measurements(output).get(0).get("expression"));
+ }
+
+ @Test
+ void translatedComposedMetricsValidateAgainstTheSalesforceSchema() throws Exception {
+ assumeTrue(salesforceSchemaExists, "Salesforce schema is required; see README setup instructions");
+ Map<String, Object> output = convertOne(model("sales", List.of(
+ metric("margin", "SNOWFLAKE", "SUM(orders.profit) / NULLIF(SUM(orders.revenue), 0)"),
+ metric("customers", "ANSI_SQL", "COUNT(DISTINCT orders.customer_id)"),
+ metric("rounding", "ANSI_SQL", "ROUND(AVG(ABS(orders.profit)), 2) + CEIL(1.2) - FLOOR(1.2)"))));
+ SchemaValidator validator = new SchemaValidator(JSON, SchemaValidator.SALESFORCE_SCHEMA_PATH);
+
+ for (Map<String, Object> metric : measurements(output)) {
+ assertMeasurementMetadata(metric);
+ }
+ assertDoesNotThrow(() -> validator.validate(output));
+
+ // OSI expression objects cannot be emitted where Salesforce requires a scalar formula.
+ Map<String, Object> measurement = measurements(output).get(0);
+ Object expression = measurement.put("expression", Map.of("dialects", List.of(
+ dialect("ANSI_SQL", "SUM(orders.profit)"))));
+ ValidationException error = assertThrows(ValidationException.class, () -> validator.validate(output));
+ assertTrue(error.getMessage().contains("expression"), error.getMessage());
+ measurement.put("expression", expression);
+ measurement.put("dataType", "Integer");
+ assertThrows(ValidationException.class, () -> validator.validate(output));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"profit__c+1", "profit__c-1", "profit__c=1", "1"})
+ void rejectsDerivedExpressionsMisclassifiedAsPhysicalColumns(String expression) throws Exception {
+ Map<String, Object> calculated = field("adjusted", "Decimal");
+ calculated.put("expression", Map.of("dialects", List.of(dialect("ANSI_SQL", expression))));
+ Map<String, Object> source = model("sales", List.of(metric("adjusted_total", "ANSI_SQL", "SUM(orders.adjusted)")));
+ items(source, "datasets").get(0).put("fields", List.of(field("profit", "Decimal"), calculated));
+ var error = assertThrows(ConversionException.class, () -> convertOne(source));
+ assertTrue(error.getMessage().contains("adjusted_total"), error.getMessage());
+ assertTrue(error.getMessage().contains("direct physical binding"), error.getMessage());
+ }
+
+ @Test
+ void metricCannotUseJoinCriteriaCorruptedByInheritedRelationshipFiltering() throws Exception {
+ Map<String, Object> source = model("sales", List.of(metric("combined", "ANSI_SQL",
+ "SUM(orders.profit) + SUM(returns.profit)")));
+ Map<String, Object> returns = new LinkedHashMap<>(items(source, "datasets").get(0));
+ returns.put("name", "returns");
+ returns.put("source", "returns__dll");
+ source.put("datasets", List.of(items(source, "datasets").get(0), returns));
+ Map<String, Object> valid = Map.of("name", "orders_returns", "from", "orders", "to", "returns",
+ "from_columns", List.of("customer_id"), "to_columns", List.of("customer_id"));
+ source.put("relationships", List.of(valid));
+ assertTrue(measurements(convertOne(source)).get(0).get("expression").toString().contains("[returns].[profit]"));
+
+ source.put("relationships", List.of(Map.of("name", "removed_first", "from", "orders", "to", "returns",
+ "from_columns", List.of("missing"), "to_columns", List.of("customer_id")), valid));
+ var error = assertThrows(ConversionException.class, () -> convertOne(source));
+ assertTrue(error.getMessage().contains("combined"), error.getMessage());
+ assertTrue(error.getMessage().contains("orders_returns"), error.getMessage());
+ assertTrue(error.getMessage().contains("join key correspondence"), error.getMessage());
+ }
+
+ private Map<String, Object> convertOne(Map<String, Object> model) throws IOException {
+ List<String> outputs = converter.convert(document(List.of(model)));
+ assertEquals(1, outputs.size());
+ return parse(outputs.get(0));
+ }
+
+ private static String document(List<Map<String, Object>> models) throws IOException {
+ assertEquals(1, models.size(), "An Ossie document contains exactly one model");
+ return document(models.get(0));
+ }
+
+ private static String document(Map<String, Object> model) throws IOException {
+ Map<String, Object> root = new LinkedHashMap<>();
+ root.put("version", "0.2.0.dev0");
+ root.putAll(model);
+ return YAML.writeValueAsString(root);
+ }
+
+ private static Map<String, Object> parse(String json) throws IOException {
+ return JSON.readValue(json, new TypeReference<>() {});
+ }
+
+ private static Map<String, Object> model(String name, List<Map<String, Object>> metrics) {
+ Map<String, Object> model = new LinkedHashMap<>();
+ model.put("name", name);
+ model.put("custom_extensions", List.of(Map.of("vendor_name", "SALESFORCE",
+ "data", "{\"dataspace\":\"default\"}")));
+ Map<String, Object> dataset = new LinkedHashMap<>();
+ dataset.put("name", "orders");
+ dataset.put("source", "orders__dll");
+ dataset.put("custom_extensions", List.of(Map.of("vendor_name", "SALESFORCE",
+ "data", "{\"dataObjectType\":\"Dlo\"}")));
+ dataset.put("fields", List.of(field("profit", "Decimal"), field("revenue", "Decimal"),
+ field("quantity", "Integer"), field("customer_id", "String")));
+ model.put("datasets", List.of(dataset));
+ model.put("metrics", metrics);
+ return model;
+ }
+
+ private static Map<String, Object> field(String name, String datatype) {
+ Map<String, Object> field = new LinkedHashMap<>();
+ field.put("name", name);
+ field.put("datatype", datatype);
+ field.put("expression", Map.of("dialects", List.of(dialect("ANSI_SQL", name + "__c"))));
+ if (datatype.equals("String")) {
+ field.put("dimension", Map.of("is_time", false));
+ }
+ return field;
+ }
+
+ private static Map<String, Object> metric(String name, String dialect, String expression) {
+ Map<String, Object> metric = new LinkedHashMap<>();
+ metric.put("name", name);
+ metric.put("datatype", "Decimal");
+ metric.put("expression", Map.of("dialects", List.of(dialect(dialect, expression))));
+ return metric;
+ }
+
+ private static Map<String, Object> dialect(String dialect, String expression) {
+ return Map.of("dialect", dialect, "expression", expression);
+ }
+
+ private static void assertMeasurementMetadata(Map<String, Object> measurement) {
+ assertInstanceOf(String.class, measurement.get("expression"));
+ assertEquals("Tua", measurement.get("syntax"));
+ assertEquals("UserAgg", measurement.get("aggregationType"));
+ assertEquals("Number", measurement.get("dataType"));
+ }
+
+ private static List<Map<String, Object>> measurements(Map<String, Object> output) {
+ return items(output, "semanticCalculatedMeasurements");
+ }
+
+ @SuppressWarnings("unchecked")
+ private static List<Map<String, Object>> items(Map<String, Object> object, String key) {
+ return (List<Map<String, Object>>) object.getOrDefault(key, List.of());
+ }
+}
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/OssieToSalesforceConverterTest.java b/converters/salesforce/src/test/java/org/apache/ossie/OssieToSalesforceConverterTest.java
index 6d5c1237..15c1c7d9 100644
--- a/converters/salesforce/src/test/java/org/apache/ossie/OssieToSalesforceConverterTest.java
+++ b/converters/salesforce/src/test/java/org/apache/ossie/OssieToSalesforceConverterTest.java
@@ -287,12 +287,91 @@ class OssieToSalesforceConverterTest {
}
@Test
- void testMetricsNotConvertedInOssieToSalesforce() throws Exception {
+ void testMetricsConvertedToSemanticCalculatedMeasurements() throws Exception {
List<String> results = converter.convert(ossieYaml);
Map<String, Object> sfModel = jsonMapper.readValue(results.get(0), new TypeReference<Map<String, Object>>() {});
List<Map<String, Object>> calcMeasurements = (List<Map<String, Object>>) sfModel.get("semanticCalculatedMeasurements");
- assertNull(calcMeasurements, "Metrics from Ossie are not converted to semanticCalculatedMeasurements in Ossie->SF direction");
+ assertNotNull(calcMeasurements, "Metrics from Ossie should convert to semanticCalculatedMeasurements");
+ assertEquals(2, calcMeasurements.size());
+
+ Map<String, Object> totalRevenue = calcMeasurements.stream()
+ .filter(m -> "total_revenue".equals(m.get("apiName")))
+ .findFirst()
+ .orElse(null);
+ assertNotNull(totalRevenue);
+ assertEquals("Sum of all order amounts", totalRevenue.get("description"));
+ assertEquals("Number", totalRevenue.get("dataType"));
+ // Legacy ANSI_SQL bracket references are validated and emitted as Tua.
+ assertEquals("SUM([Orders].[amount])", totalRevenue.get("expression"));
+ assertEquals("Tua", totalRevenue.get("syntax"));
+ assertEquals("UserAgg", totalRevenue.get("aggregationType"));
+
+ Map<String, Object> avgOrderValue = calcMeasurements.stream()
+ .filter(m -> "avg_order_value".equals(m.get("apiName")))
+ .findFirst()
+ .orElse(null);
+ assertNotNull(avgOrderValue);
+ assertEquals("AVG([Orders].[amount])", avgOrderValue.get("expression"));
+ }
+
+ @Test
+ void testMetricExpressionPrefersTableauDialectOverAnsiSql() throws Exception {
+ // Normalize line endings first: the fixture file may check out with CRLF depending on
+ // the platform's autocrlf setting, but the substitution below is written with LF.
+ String yamlWithTableauMetric = ossieYaml.replace("\r\n", "\n").replace(
+ "metrics:\n"
+ + "- description: Sum of all order amounts\n"
+ + " name: total_revenue\n"
+ + " datatype: Decimal\n"
+ + " expression:\n"
+ + " dialects:\n"
+ + " - dialect: ANSI_SQL\n"
+ + " expression: SUM([Orders].[amount])\n",
+ "metrics:\n"
+ + "- description: Sum of all order amounts\n"
+ + " name: total_revenue\n"
+ + " datatype: Decimal\n"
+ + " expression:\n"
+ + " dialects:\n"
+ + " - dialect: ANSI_SQL\n"
+ + " expression: SUM([Orders].[amount])\n"
+ + " - dialect: TABLEAU\n"
+ + " expression: MAX([Orders].[amount])\n");
+ assertTrue(yamlWithTableauMetric.contains("dialect: TABLEAU"), "fixture text substitution did not match");
+
+ List<String> results = converter.convert(yamlWithTableauMetric);
+ Map<String, Object> sfModel = jsonMapper.readValue(results.get(0), new TypeReference<Map<String, Object>>() {});
+ List<Map<String, Object>> calcMeasurements = (List<Map<String, Object>>) sfModel.get("semanticCalculatedMeasurements");
+
+ Map<String, Object> totalRevenue = calcMeasurements.stream()
+ .filter(m -> "total_revenue".equals(m.get("apiName")))
+ .findFirst()
+ .orElse(null);
+ assertNotNull(totalRevenue);
+ assertEquals("MAX([Orders].[amount])", totalRevenue.get("expression"),
+ "TABLEAU dialect should be preferred over ANSI_SQL when both are present");
+ }
+
+ @Test
+ void testMetricWithNoConvertibleDialectFailsConversion() throws Exception {
+ // Normalize line endings first: the fixture file may check out with CRLF depending on
+ // the platform's autocrlf setting, but the substitution below is written with LF.
+ // The Ossie schema requires every metric to have an expression and restricts `dialect`
+ // to its own enum, so this uses BIGQUERY (a valid dialect, but neither TABLEAU nor
+ // ANSI_SQL) rather than omitting the expression or inventing an unrecognized dialect.
+ String yamlWithUnconvertibleDialect = ossieYaml.replace("\r\n", "\n").replace(
+ " - dialect: ANSI_SQL\n"
+ + " expression: SUM([Orders].[amount])\n",
+ " - dialect: BIGQUERY\n"
+ + " expression: SUM(Orders.amount)\n");
+ assertTrue(yamlWithUnconvertibleDialect.contains("dialect: BIGQUERY"),
+ "fixture text substitution did not match");
+
+ Exception exception =
+ assertThrows(Exception.class, () -> converter.convert(yamlWithUnconvertibleDialect));
+ String message = exception.getMessage() != null ? exception.getMessage() : exception.getCause().getMessage();
+ assertTrue(message.contains("total_revenue"), "error should name the unconvertible metric: " + message);
}
@Test
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionSemanticsTest.java b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionSemanticsTest.java
new file mode 100644
index 00000000..4e3fc8e4
--- /dev/null
+++ b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionSemanticsTest.java
@@ -0,0 +1,543 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.junit.jupiter.api.Assertions.*;
+
+import java.math.BigDecimal;
+import java.math.RoundingMode;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+/**
+ * Hand-computed examples evaluated independently from the production parser/AST.
+ *
+ * <p>This is a local regression oracle for the emitted Tua subset, not a Tableau Next execution
+ * test. In particular, the evaluator models three-valued Boolean logic, null-ignoring aggregates,
+ * and half-away-from-zero rounding; it does not establish native engine behavior or precision.
+ */
+class MetricExpressionSemanticsTest {
+
+ private static final List<Map<String, Object>> ORDERS = List.of(
+ row(10.0, 2.0, true),
+ row(10.0, 0.0, false),
+ row(-4.0, 4.0, null),
+ row(null, null, null));
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void aggregatesIgnoreNullsButPreserveDuplicates(String dialect) {
+ assertValue(dialect, "SUM(orders.amount)", ORDERS, 16.0);
+ assertValue(dialect, "AVG(orders.amount)", ORDERS, 16.0 / 3);
+ assertValue(dialect, "MIN(orders.amount)", ORDERS, -4.0);
+ assertValue(dialect, "MAX(orders.amount)", ORDERS, 10.0);
+ assertValue(dialect, "COUNT(orders.amount)", ORDERS, 3.0);
+ assertValue(dialect, "COUNT(DISTINCT orders.amount)", ORDERS, 2.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void emptyAndAllNullInputsHaveDifferentCountAndSumResults(String dialect) {
+ for (List<Map<String, Object>> rows : List.of(
+ List.<Map<String, Object>>of(), List.of(row(null, null, null)))) {
+ for (String aggregate : List.of("SUM", "AVG", "MIN", "MAX")) {
+ assertValue(dialect, aggregate + "(orders.amount)", rows, null);
+ }
+ assertValue(dialect, "COUNT(orders.amount)", rows, 0.0);
+ assertValue(dialect, "COUNT(DISTINCT orders.amount)", rows, 0.0);
+ assertValue(dialect, "COALESCE(SUM(orders.amount), 0)", rows, 0.0);
+ }
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void composedConditionalArithmeticPreservesNullsAndGrouping(String dialect) {
+ // Per-row numerator contributions are 20, 10, -4, 0; COUNT ignores the null amount.
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.flag AND orders.amount > 0 "
+ + "THEN orders.amount * 2 ELSE COALESCE(orders.amount, 0) END) "
+ + "/ NULLIF(COUNT(orders.amount), 0)",
+ ORDERS, 26.0 / 3);
+ assertValue(dialect,
+ "(SUM(orders.amount) + 2) * (MAX(orders.cost) - MIN(orders.cost))",
+ ORDERS, 72.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount < 0 THEN -orders.amount END)",
+ ORDERS, 4.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void guardedRatiosReturnNullForZeroAndEmptyDenominators(String dialect) {
+ String ratio = "SUM(orders.amount) / NULLIF(SUM(orders.cost), 0)";
+ assertValue(dialect, ratio, ORDERS, 16.0 / 6);
+ assertValue(dialect, ratio, List.of(row(8.0, 1.0, true), row(4.0, -1.0, false)), null);
+ assertValue(dialect, ratio, List.of(row(8.0, null, true)), null);
+ assertValue(dialect, ratio, List.of(), null);
+ assertValue(dialect, "COALESCE(" + ratio + ", 0)", List.of(), 0.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void nestedCoalesceSelectsTheFirstNonNullValueIncludingZero(String dialect) {
+ String expression = "COALESCE(SUM(orders.amount), NULLIF(SUM(orders.cost), 0), 7)";
+ assertValue(dialect, expression, ORDERS, 16.0);
+ assertValue(dialect, expression, List.of(row(0.0, 9.0, true)), 0.0);
+ assertValue(dialect, expression, List.of(row(null, 9.0, true)), 9.0);
+ assertValue(dialect, expression, List.of(row(null, 0.0, true)), 7.0);
+ assertValue(dialect, expression, List.of(), 7.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void nullablePredicatesUseThreeValuedLogic(String dialect) {
+ assertValue(dialect,
+ "SUM(CASE WHEN NOT (orders.flag = TRUE) THEN orders.amount ELSE 0 END)",
+ ORDERS, 10.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount IS NOT NULL AND "
+ + "(orders.flag = FALSE OR orders.flag IS NULL) "
+ + "THEN orders.amount ELSE 0 END)",
+ ORDERS, 6.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN NOT (orders.flag OR orders.amount < 0) THEN 1 ELSE 0 END)",
+ ORDERS, 1.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.flag IS NULL THEN "
+ + "CASE WHEN orders.amount IS NULL THEN 3 ELSE 2 END ELSE 0 END)",
+ ORDERS, 5.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void numericFunctionsComposeAroundNegativeAggregates(String dialect) {
+ List<Map<String, Object>> negative = List.of(row(-2.55, 0.0, true), row(-2.55, 0.0, false));
+ assertValue(dialect, "ROUND(AVG(orders.amount), 1)", negative, -2.6);
+ assertValue(dialect, "ABS(ROUND(AVG(orders.amount), 1))", negative, 2.6);
+ assertValue(dialect, "CEIL(AVG(orders.amount)) + FLOOR(AVG(orders.amount))", negative, -5.0);
+ assertValue(dialect, "ABS(ROUND(AVG(orders.amount), 1))", List.of(), null);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void textPredicatesPreserveDuplicatesNullsAndEscapedApostrophes(String dialect) {
+ List<Map<String, Object>> orders = List.of(
+ row(10.0, 0.0, true, "paid"),
+ row(7.0, 0.0, true, "paid"),
+ row(4.0, 0.0, false, "pending"),
+ row(9.0, 0.0, null, null),
+ row(2.0, 0.0, true, "O'Brien"));
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.status = 'paid' THEN orders.amount ELSE 0 END)",
+ orders, 17.0);
+ assertValue(dialect, "COUNT(DISTINCT orders.status)", orders, 3.0);
+ assertValue(dialect, "COUNT(orders.status)", orders, 4.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN COALESCE(NULLIF(orders.status, 'pending'), 'missing') = 'missing' "
+ + "THEN orders.amount ELSE 0 END)",
+ orders, 13.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN COALESCE(orders.status, 'paid') = 'paid' THEN orders.amount ELSE 0 END)",
+ orders, 26.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.status = 'O''Brien' THEN orders.amount ELSE 0 END)",
+ orders, 2.0);
+ assertValue(dialect, "COUNT(DISTINCT orders.status)", List.of(), 0.0);
+ assertValue(dialect, "COUNT(DISTINCT orders.status)", List.of(row(null, null, null, null)), 0.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void conditionalBranchesPreserveOrderAndImplicitNull(String dialect) {
+ List<Map<String, Object>> orders = List.of(
+ row(10.0, 0.0, true), row(0.0, 0.0, false),
+ row(-4.0, 0.0, null), row(null, null, null));
+ // Positive amounts match both of the first two conditions; the first must win.
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount > 0 THEN 1 WHEN orders.amount >= 0 THEN 2 "
+ + "WHEN orders.amount < 0 THEN 3 ELSE 4 END)",
+ orders, 10.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount > 0 THEN 1 WHEN orders.amount >= 0 THEN 2 END)",
+ orders, 3.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount > 0 THEN 1 WHEN orders.amount = 0 THEN 2 END)",
+ List.of(row(-4.0, null, null), row(null, null, null)), null);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void nestedConditionalsSkipUnusedConditionsAndResults(String dialect) {
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount > 0 THEN "
+ + "CASE WHEN orders.flag THEN 1 WHEN 1 / orders.cost > 0 THEN 99 ELSE 99 END "
+ + "WHEN orders.amount = 0 THEN 2 WHEN orders.amount < 0 THEN "
+ + "CASE WHEN orders.flag IS NULL THEN 3 ELSE 1 / orders.cost END ELSE 4 END)",
+ List.of(row(10.0, 0.0, true), row(0.0, 0.0, false),
+ row(-4.0, 0.0, null), row(null, 0.0, null)), 10.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount >= 0 THEN 1 "
+ + "WHEN 1 / orders.cost > 0 THEN 2 ELSE 1 / orders.cost END)",
+ List.of(row(10.0, 0.0, true), row(0.0, 0.0, false)), 2.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void numericComparisonsTreatSignedZerosAsEqual(String dialect) {
+ List<Map<String, Object>> zeros = List.of(row(-0.0, null, null), row(0.0, null, null));
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount = 0 THEN 1 ELSE 0 END)", zeros, 2.0);
+ for (String operator : List.of("!=", "<>")) {
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount " + operator + " 0 THEN 1 ELSE 0 END)", zeros, 0.0);
+ }
+ assertValue(dialect,
+ "SUM(CASE WHEN orders.amount <= 0 AND orders.amount >= 0 THEN 1 ELSE 0 END)",
+ zeros, 2.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void distinctCountTreatsSignedZerosAsOneValue(String dialect) {
+ List<Map<String, Object>> orders = List.of(
+ row(-0.0, null, null), row(0.0, null, null),
+ row(1.0, null, null), row(null, null, null));
+ assertValue(dialect, "COUNT(DISTINCT orders.amount)", orders, 2.0);
+ assertValue(dialect, "COUNT(orders.amount)", orders, 3.0);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void nullifGuardsEitherSignOfZero(String dialect) {
+ assertValue(dialect, "SUM(1 / NULLIF(orders.cost, 0))",
+ List.of(row(null, -0.0, null), row(null, 0.0, null)), null);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"0", "-0"})
+ void evaluatorRejectsEitherSignOfUnguardedZero(String zero) {
+ AssertionError error = assertThrows(AssertionError.class,
+ () -> new TuaSubsetEvaluator("1 / " + zero).evaluate(List.of()));
+ assertTrue(error.getMessage().contains("unguarded zero divisor"));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SNOWFLAKE", "ANSI_SQL"})
+ void repeatedUnaryOperatorsPreserveValuesAndUnknownPredicates(String dialect) {
+ List<Map<String, Object>> orders = List.of(
+ row(3.0, null, true), row(7.0, null, false),
+ row(11.0, null, null), row(null, null, null));
+ assertValue(dialect, "SUM(- -orders.amount)", orders, 21.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN NOT NOT NOT orders.flag THEN orders.amount ELSE 0 END)", orders, 7.0);
+ assertValue(dialect,
+ "SUM(CASE WHEN NOT NOT NOT NOT orders.flag THEN orders.amount ELSE 0 END)", orders, 3.0);
+ }
+
+ private static void assertValue(
+ String dialect, String sql, List<Map<String, Object>> rows, Double expected) {
+ Map<String, Object> metric = Map.of(
+ "name", "fixture_metric",
+ "datatype", "Decimal",
+ "expression", Map.of("dialects", List.of(Map.of("dialect", dialect, "expression", sql))));
+ Map<String, Object> source = Map.of("datasets", List.of(Map.of(
+ "name", "orders",
+ "fields", List.of(
+ Map.of("name", "amount", "datatype", "Decimal"),
+ Map.of("name", "cost", "datatype", "Decimal"),
+ Map.of("name", "flag", "datatype", "Boolean"),
+ Map.of("name", "status", "datatype", "String")))));
+ Map<String, Object> target = Map.of("semanticDataObjects", List.of(Map.of(
+ "apiName", "orders",
+ "semanticMeasurements", List.of(
+ Map.of("apiName", "amount", "dataObjectFieldName", "amount__c", "dataType", "Number"),
+ Map.of("apiName", "cost", "dataObjectFieldName", "cost__c", "dataType", "Number")),
+ "semanticDimensions", List.of(
+ Map.of("apiName", "flag", "dataObjectFieldName", "flag__c", "dataType", "Boolean"),
+ Map.of("apiName", "status", "dataObjectFieldName", "status__c", "dataType", "Text")))));
+ MetricExpressionTranslator.Result translated = MetricExpressionTranslator.translate(metric, source, target);
+ assertEquals("Number", translated.dataType(), sql);
+ Object actual = new TuaSubsetEvaluator(translated.expression()).evaluate(rows);
+ String message = dialect + ": " + sql + " -> " + translated.expression();
+ if (expected == null) {
+ assertNull(actual, message);
+ } else {
+ assertInstanceOf(Number.class, actual, message);
+ assertEquals(expected, ((Number) actual).doubleValue(), 1e-12, message);
+ }
+ }
+
+ private static Map<String, Object> row(Double amount, Double cost, Boolean flag) {
+ return row(amount, cost, flag, null);
+ }
+
+ private static Map<String, Object> row(Double amount, Double cost, Boolean flag, String status) {
+ Map<String, Object> result = new LinkedHashMap<>();
+ result.put("amount", amount);
+ result.put("cost", cost);
+ result.put("flag", flag);
+ result.put("status", status);
+ return result;
+ }
+
+ /** Reads only generated Tua; it neither accepts SQL CASE/NULLIF nor uses production AST nodes. */
+ private static final class TuaSubsetEvaluator {
+ private static final Pattern TOKEN = Pattern.compile(
+ "\\s*(\\[[^\\]]+\\]|[0-9]+(?:\\.[0-9]+)?(?:[eE][+-]?[0-9]+)?"
+ + "|'(?:[^']|'')*'|[A-Za-z_][A-Za-z_0-9]*|<>|!=|<=|>=|[().,+*/=<>-])");
+ private final List<String> tokens = new ArrayList<>();
+ private int position;
+
+ TuaSubsetEvaluator(String expression) {
+ Matcher matcher = TOKEN.matcher(expression);
+ int offset = 0;
+ while (offset < expression.length()) {
+ if (expression.substring(offset).isBlank()) {
+ break;
+ }
+ if (!matcher.find(offset) || matcher.start() != offset) {
+ throw new AssertionError("Unexpected Tua token: " + expression.substring(offset));
+ }
+ tokens.add(matcher.group(1));
+ offset = matcher.end();
+ }
+ }
+
+ Object evaluate(List<Map<String, Object>> rows) {
+ Calculation calculation = expression(0);
+ assertEquals(tokens.size(), position, "Unconsumed generated Tua tokens");
+ return calculation.value(rows, Map.of());
+ }
+
+ private Calculation expression(int minimum) {
+ Calculation left = prefix();
+ while (position < tokens.size() && precedence(tokens.get(position)) >= minimum) {
+ String operator = next().toUpperCase(java.util.Locale.ROOT);
+ Calculation right = expression(precedence(operator) + 1);
+ Calculation previous = left;
+ left = (rows, row) -> binary(operator, previous.value(rows, row), right.value(rows, row));
+ }
+ return left;
+ }
+
+ private Calculation prefix() {
+ String token = next();
+ if (token.equals("(")) {
+ Calculation result = expression(0);
+ expect(")");
+ return result;
+ }
+ if (token.equalsIgnoreCase("IF")) {
+ List<Calculation> conditions = new ArrayList<>();
+ List<Calculation> results = new ArrayList<>();
+ do {
+ conditions.add(expression(0));
+ expect("THEN");
+ results.add(expression(0));
+ } while (take("ELSEIF"));
+ expect("ELSE");
+ Calculation otherwise = expression(0);
+ expect("END");
+ return (rows, row) -> {
+ for (int i = 0; i < conditions.size(); i++) {
+ if (Boolean.TRUE.equals(conditions.get(i).value(rows, row))) {
+ return results.get(i).value(rows, row);
+ }
+ }
+ return otherwise.value(rows, row);
+ };
+ }
+ if (token.equalsIgnoreCase("NOT") || token.equals("-")) {
+ Calculation child = expression(token.equals("-") ? 7 : 3);
+ return (rows, row) -> {
+ Object value = child.value(rows, row);
+ return value == null ? null : token.equals("-") ? -number(value) : !(Boolean) value;
+ };
+ }
+ if (token.equalsIgnoreCase("NULL")) {
+ return (rows, row) -> null;
+ }
+ if (token.equalsIgnoreCase("TRUE") || token.equalsIgnoreCase("FALSE")) {
+ return (rows, row) -> Boolean.valueOf(token);
+ }
+ if (token.startsWith("'")) {
+ String literal = token.substring(1, token.length() - 1).replace("''", "'");
+ return (rows, row) -> literal;
+ }
+ if (token.startsWith("[")) {
+ assertEquals("[orders]", token);
+ expect(".");
+ String field = next();
+ assertTrue(field.startsWith("[") && field.endsWith("]"));
+ String name = field.substring(1, field.length() - 1);
+ return (rows, row) -> {
+ assertTrue(row.containsKey(name), "Unknown generated field: " + name);
+ return row.get(name);
+ };
+ }
+ if (Character.isDigit(token.charAt(0))) {
+ return (rows, row) -> Double.valueOf(token);
+ }
+ expect("(");
+ List<Calculation> arguments = new ArrayList<>();
+ do {
+ arguments.add(expression(0));
+ } while (take(","));
+ expect(")");
+ return function(token.toUpperCase(java.util.Locale.ROOT), arguments);
+ }
+
+ private static Calculation function(String name, List<Calculation> arguments) {
+ if (List.of("SUM", "AVG", "MIN", "MAX", "COUNT", "COUNTD").contains(name)) {
+ assertEquals(1, arguments.size());
+ return (rows, row) -> {
+ List<Object> values = rows.stream()
+ .map(input -> arguments.getFirst().value(rows, input))
+ .filter(java.util.Objects::nonNull).toList();
+ if (name.equals("COUNT")) {
+ return (double) values.size();
+ }
+ if (name.equals("COUNTD")) {
+ return (double) values.stream()
+ .map(value -> value instanceof Number && number(value) == 0.0 ? 0.0 : value)
+ .distinct().count();
+ }
+ if (values.isEmpty()) {
+ return null;
+ }
+ return switch (name) {
+ case "SUM" -> values.stream().mapToDouble(TuaSubsetEvaluator::number).sum();
+ case "AVG" -> values.stream().mapToDouble(TuaSubsetEvaluator::number).average().orElseThrow();
+ case "MIN" -> values.stream().mapToDouble(TuaSubsetEvaluator::number).min().orElseThrow();
+ case "MAX" -> values.stream().mapToDouble(TuaSubsetEvaluator::number).max().orElseThrow();
+ default -> throw new AssertionError(name);
+ };
+ };
+ }
+ assertTrue(List.of("IFNULL", "ISNULL", "ABS", "CEILING", "FLOOR", "ROUND").contains(name),
+ "Unsupported generated function: " + name);
+ return (rows, row) -> {
+ Object value = arguments.getFirst().value(rows, row);
+ if (name.equals("IFNULL")) {
+ assertEquals(2, arguments.size());
+ return value != null ? value : arguments.get(1).value(rows, row);
+ }
+ if (name.equals("ISNULL")) {
+ assertEquals(1, arguments.size());
+ return value == null;
+ }
+ if (value == null) {
+ return null;
+ }
+ return switch (name) {
+ case "ABS" -> Math.abs(number(value));
+ case "CEILING" -> Math.ceil(number(value));
+ case "FLOOR" -> Math.floor(number(value));
+ case "ROUND" -> BigDecimal.valueOf(number(value)).setScale(
+ arguments.size() == 1 ? 0 : (int) number(arguments.get(1).value(rows, row)),
+ RoundingMode.HALF_UP).doubleValue();
+ default -> throw new AssertionError("Unsupported generated function: " + name);
+ };
+ };
+ }
+
+ private static Object binary(String operator, Object left, Object right) {
+ if (operator.equals("AND")) {
+ if (Boolean.FALSE.equals(left) || Boolean.FALSE.equals(right)) {
+ return false;
+ }
+ return left == null || right == null ? null : Boolean.TRUE;
+ }
+ if (operator.equals("OR")) {
+ if (Boolean.TRUE.equals(left) || Boolean.TRUE.equals(right)) {
+ return true;
+ }
+ return left == null || right == null ? null : Boolean.FALSE;
+ }
+ if (left == null || right == null) {
+ return null;
+ }
+ return switch (operator) {
+ case "+" -> number(left) + number(right);
+ case "-" -> number(left) - number(right);
+ case "*" -> number(left) * number(right);
+ case "/" -> {
+ assertTrue(number(right) != 0.0, "Generated expression evaluated an unguarded zero divisor");
+ yield number(left) / number(right);
+ }
+ case "=" -> equal(left, right);
+ case "!=", "<>" -> !equal(left, right);
+ case "<" -> number(left) < number(right);
+ case "<=" -> number(left) <= number(right);
+ case ">" -> number(left) > number(right);
+ case ">=" -> number(left) >= number(right);
+ default -> throw new AssertionError(operator);
+ };
+ }
+
+ private static boolean equal(Object left, Object right) {
+ return left instanceof Number && right instanceof Number
+ ? number(left) == number(right) : left.equals(right);
+ }
+
+ private static double number(Object value) {
+ return ((Number) value).doubleValue();
+ }
+
+ private static int precedence(String token) {
+ return switch (token.toUpperCase(java.util.Locale.ROOT)) {
+ case "OR" -> 1;
+ case "AND" -> 2;
+ case "=", "!=", "<>", "<", ">", "<=", ">=" -> 4;
+ case "+", "-" -> 5;
+ case "*", "/" -> 6;
+ default -> -1;
+ };
+ }
+
+ private boolean take(String token) {
+ if (position < tokens.size() && tokens.get(position).equalsIgnoreCase(token)) {
+ position++;
+ return true;
+ }
+ return false;
+ }
+
+ private String next() {
+ assertTrue(position < tokens.size(), "Unexpected end of generated Tua expression");
+ return tokens.get(position++);
+ }
+
+ private void expect(String token) {
+ assertEquals(token, next().toUpperCase(java.util.Locale.ROOT));
+ }
+ }
+
+ @FunctionalInterface
+ private interface Calculation {
+ Object value(List<Map<String, Object>> rows, Map<String, Object> row);
+ }
+}
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionTranslatorTest.java b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionTranslatorTest.java
new file mode 100644
index 00000000..0692e89e
--- /dev/null
+++ b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricExpressionTranslatorTest.java
@@ -0,0 +1,399 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.junit.jupiter.api.Assertions.*;
+
+import java.time.Duration;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Stream;
+import org.apache.ossie.exception.ConversionException;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.Arguments;
+import org.junit.jupiter.params.provider.MethodSource;
+import org.junit.jupiter.params.provider.ValueSource;
+
+class MetricExpressionTranslatorTest {
+ private static final Map<String, String> TYPES = Map.of(
+ "amount", "Decimal", "profit", "Decimal", "revenue", "Decimal", "discount", "Decimal",
+ "quantity", "Integer", "status", "String", "active", "Boolean", "ordered", "Date");
+
+ private static Map<String, Object> source() {
+ return Map.of("datasets", List.of(Map.of("name", "orders", "fields", TYPES.entrySet().stream()
+ .map(entry -> Map.of("name", entry.getKey(), "datatype", entry.getValue())).toList())));
+ }
+
+ private static Map<String, Object> target() {
+ return Map.of("semanticDataObjects", List.of(Map.of("apiName", "orders", "semanticMeasurements",
+ TYPES.entrySet().stream().map(entry -> Map.of("apiName", entry.getKey(), "dataObjectFieldName", entry.getKey() + "__c", "dataType",
+ SalesforceDataTypeMapper.toSalesforce(entry.getValue()))).toList())));
+ }
+
+ private static Map<String, Object> metric(String dialect, String expression) {
+ return new LinkedHashMap<>(Map.of("name", "net_value", "expression", Map.of("dialects",
+ List.of(Map.of("dialect", dialect, "expression", expression)))));
+ }
+
+ private static String translate(String dialect, String expression) {
+ var result = MetricExpressionTranslator.translate(metric(dialect, expression), source(), target());
+ assertEquals("Number", result.dataType());
+ return result.expression();
+ }
+
+ static Stream<Arguments> supported() {
+ return Stream.of(
+ Arguments.of("SUM(orders.amount)", "SUM([orders].[amount])"),
+ Arguments.of("AVG(amount)", "AVG([orders].[amount])"),
+ Arguments.of("MIN(ORDERS.amount)", "MIN([orders].[amount])"),
+ Arguments.of("MAX(orders.amount)", "MAX([orders].[amount])"),
+ Arguments.of("COUNT(orders.status)", "COUNT([orders].[status])"),
+ Arguments.of("COUNT(DISTINCT orders.status)", "COUNTD([orders].[status])"),
+ Arguments.of("SUM(orders.amount * (1 - orders.discount))",
+ "SUM(([orders].[amount] * (1 - [orders].[discount])))"),
+ Arguments.of("SUM(orders.profit) / NULLIF(SUM(orders.revenue), 0)",
+ "(SUM([orders].[profit]) / (IF (SUM([orders].[revenue]) = 0) THEN NULL ELSE SUM([orders].[revenue]) END))"),
+ Arguments.of("SUM(CASE WHEN orders.status = 'paid' THEN orders.amount ELSE 0 END)",
+ "SUM((IF ([orders].[status] = 'paid') THEN [orders].[amount] ELSE 0 END))"),
+ Arguments.of("COALESCE(SUM(orders.amount), AVG(orders.revenue), 0)",
+ "IFNULL(SUM([orders].[amount]), IFNULL(AVG([orders].[revenue]), 0))"),
+ Arguments.of("SUM(CASE WHEN orders.amount IS NOT NULL THEN orders.amount END)",
+ "SUM((IF (NOT ISNULL([orders].[amount])) THEN [orders].[amount] ELSE NULL END))"),
+ Arguments.of("ROUND(AVG(ABS(orders.amount)), 2)", "ROUND(AVG(ABS([orders].[amount])), 2)"),
+ Arguments.of("SUM(CEIL(orders.amount) - FLOOR(orders.amount))",
+ "SUM((CEILING([orders].[amount]) - FLOOR([orders].[amount])))"),
+ Arguments.of("1 + 2 * 3 - 4 / 2", "((1 + (2 * 3)) - (4 / 2))"),
+ Arguments.of(".25 + 1e2", "(0.25 + 100)"),
+ Arguments.of("SUM(orders.quantity) + -2", "(SUM([orders].[quantity]) + (-2))"),
+ Arguments.of("SUM(- -orders.amount)", "SUM([orders].[amount])"),
+ Arguments.of("SUM(+ - - +orders.amount)", "SUM([orders].[amount])"),
+ Arguments.of("SUM(- + - -orders.amount)", "SUM((-[orders].[amount]))"),
+ Arguments.of("SUM(orders.amount - -orders.discount)",
+ "SUM(([orders].[amount] - (-[orders].[discount])))"),
+ Arguments.of("SUM(orders.amount) / - -SUM(orders.quantity)",
+ "(SUM([orders].[amount]) / SUM([orders].[quantity]))"),
+ Arguments.of("SUM((((orders.amount + orders.discount))) * 2)",
+ "SUM((([orders].[amount] + [orders].[discount]) * 2))"),
+ Arguments.of("SUM(CASE WHEN NOT NOT NOT orders.active THEN orders.amount ELSE 0 END)",
+ "SUM((IF (NOT [orders].[active]) THEN [orders].[amount] ELSE 0 END))"),
+ Arguments.of("SUM(CASE WHEN NOT NOT NOT NOT orders.active THEN orders.amount ELSE 0 END)",
+ "SUM((IF (NOT (NOT [orders].[active])) THEN [orders].[amount] ELSE 0 END))"),
+ Arguments.of("SUM(CASE WHEN orders.status = '- - NOT NOT NOT ((x))' THEN - -orders.amount ELSE 0 END)",
+ "SUM((IF ([orders].[status] = '- - NOT NOT NOT ((x))') THEN [orders].[amount] ELSE 0 END))"),
+ Arguments.of("CASE WHEN MAX(orders.status) = 'z' THEN 1 ELSE 0 END",
+ "(IF (MAX([orders].[status]) = 'z') THEN 1 ELSE 0 END)")
+ );
+ }
+
+ @ParameterizedTest
+ @MethodSource("supported")
+ void compilesComposedSqlInBothDialects(String expression, String expected) {
+ assertEquals(expected, translate("SNOWFLAKE", expression));
+ assertEquals(expected, translate("ANSI_SQL", expression));
+ // Every generated expression can be read through the bounded TABLEAU path.
+ assertEquals(expected, translate("TABLEAU", expected));
+ }
+
+ @Test
+ void notUsesSqlPrecedenceAndPreservesThreeValuedPredicates() {
+ assertEquals("SUM((IF ((NOT ([orders].[amount] = 0)) OR ([orders].[active] AND ISNULL([orders].[discount])))"
+ + " THEN 1 ELSE 0 END))",
+ translate("SNOWFLAKE", "SUM(CASE WHEN NOT orders.amount = 0 OR orders.active AND orders.discount IS NULL THEN 1 ELSE 0 END)"));
+ }
+
+ @Test
+ void nestedCasesAndElseifRetainBranchOrder() {
+ String sql = "SUM(CASE WHEN orders.active THEN CASE WHEN orders.amount > 0 THEN 2 ELSE 3 END WHEN orders.status = 'paid' THEN 4 END)";
+ String expected = "SUM((IF [orders].[active] THEN (IF ([orders].[amount] > 0) THEN 2 ELSE 3 END)"
+ + " ELSEIF ([orders].[status] = 'paid') THEN 4 ELSE NULL END))";
+ assertEquals(expected, translate("SNOWFLAKE", sql));
+ assertEquals(expected, translate("TABLEAU", expected));
+ }
+
+ @Test
+ void identifiersAreBoundToSemanticNamesAndStringContentsAreNotReferences() {
+ assertEquals("SUM([orders].[amount])", translate("SNOWFLAKE", "SUM(\"ORDERS\".\"AMOUNT\")"));
+ assertEquals("SUM([orders].[amount])", translate("ANSI_SQL", "SUM([orders].[amount])"));
+ assertEquals("SUM((IF ([orders].[status] = 'missing.field O''Brien') THEN 1 ELSE 0 END))",
+ translate("SNOWFLAKE", "SUM(CASE WHEN orders.status = 'missing.field O''Brien' THEN 1 ELSE 0 END)"));
+ assertEquals("SUM((IF ([orders].[status] = 'paid') THEN 1 ELSE 0 END))",
+ translate("TABLEAU", "SUM(IF [orders].[status] = \"paid\" THEN 1 ELSE 0 END)"));
+ }
+
+ static Stream<Arguments> unsupported() {
+ return Stream.of(
+ Arguments.of("SUM(orders.missing)", "Unknown field"),
+ Arguments.of("SUM(missing.amount)", "Unknown dataset"),
+ Arguments.of("SUM(db.orders.amount)", "physical source paths"),
+ Arguments.of("SUM(orders.amount__c)", "Unknown field"),
+ Arguments.of("SUM(orders.amount) + orders.amount", "mix aggregate"),
+ Arguments.of("CASE WHEN orders.active THEN SUM(orders.amount) ELSE 0 END", "mix aggregate"),
+ Arguments.of("SUM(AVG(orders.amount))", "nested aggregate"),
+ Arguments.of("SUM(orders.status)", "numeric"),
+ Arguments.of("AVG(orders.active)", "numeric"),
+ Arguments.of("SUM(CASE WHEN orders.amount THEN 1 ELSE 0 END)", "BOOLEAN"),
+ Arguments.of("SUM(CASE WHEN orders.active THEN orders.amount ELSE 'zero' END)", "incompatible types"),
+ Arguments.of("COALESCE(SUM(orders.amount), '0')", "incompatible types"),
+ Arguments.of("NULLIF(SUM(orders.amount), '0')", "incompatible types"),
+ Arguments.of("SUM(orders.amount) AND TRUE", "BOOLEAN"),
+ Arguments.of("SUM(CASE WHEN orders.active > TRUE THEN 1 ELSE 0 END)", "ordered comparison"),
+ Arguments.of("orders.amount + 1", "unaggregated"),
+ Arguments.of("MAX(orders.status)", "must be numeric"),
+ Arguments.of("TRUE", "must be numeric"),
+ Arguments.of("COUNT(*)", "COUNT(*)"),
+ Arguments.of("COUNT(1)", "declared field"),
+ Arguments.of("COUNT(orders.quantity + 1)", "counting expressions"),
+ Arguments.of("SUM(1)", "establish its dataset"),
+ Arguments.of("SUM(DISTINCT orders.amount)", "DISTINCT"),
+ Arguments.of("COUNT(DISTINCT orders.amount, orders.quantity)", "expects 1"),
+ Arguments.of("ROUND(SUM(orders.amount), 2, 'HALF_TO_EVEN')", "expects 1 to 2"),
+ Arguments.of("ROUND(SUM(orders.amount), .5)", "integer literal"),
+ Arguments.of("ROUND(SUM(orders.amount), orders.quantity)", "integer literal"),
+ Arguments.of("CEIL(SUM(orders.amount), 2)", "expects 1"),
+ Arguments.of("FLOOR(SUM(orders.amount), 2)", "expects 1"),
+ Arguments.of("ABS()", "expects 1"),
+ Arguments.of("COALESCE(SUM(orders.amount))", "expects 2"),
+ Arguments.of("NULLIF(SUM(orders.amount))", "expects 2"),
+ Arguments.of("MEDIAN(orders.amount)", "unsupported function"),
+ Arguments.of("CAST(orders.amount AS DECIMAL)", "unsupported SQL expression CastExpression"),
+ Arguments.of("SUM(YEAR(orders.ordered))", "unsupported function"),
+ Arguments.of("SUM(LENGTH(orders.status))", "unsupported function"),
+ Arguments.of("SUM(orders.amount) OVER ()", "unsupported SQL expression AnalyticExpression"),
+ Arguments.of("SUM(orders.amount) FILTER (WHERE orders.active)", "unsupported SQL expression AnalyticExpression"),
+ Arguments.of("{ FIXED : SUM(orders.amount) }", "unsupported character"),
+ Arguments.of("SELECT SUM(orders.amount)", "unexpected token"),
+ Arguments.of("SUM(orders.amount);", "unsupported character"),
+ Arguments.of("SUM(orders.amount) -- comment", "comments"),
+ Arguments.of("SUM(orders.amount) /* comment */", "comments"),
+ Arguments.of("SUM(orders.amount", "expected )"),
+ Arguments.of("SUM(orders.amount) trailing", "unexpected token"),
+ Arguments.of("1.2.3", "invalid numeric"),
+ Arguments.of("1e", "invalid numeric"),
+ Arguments.of("1e1000000000", "too large"),
+ Arguments.of("SUM(CASE WHEN orders.status = 'unterminated THEN 1 END)", "unterminated"),
+ Arguments.of("SUM(orders.amount) % 2", "unsupported character"),
+ Arguments.of("SUM(orders.amount) ^ 2", "unsupported character"),
+ Arguments.of("SUM(orders.amount) || 'x'", "unsupported character"),
+ Arguments.of("SUM(orders.amount) / 0", "division by literal zero"),
+ Arguments.of("net_value + 1", "Unknown field")
+ );
+ }
+
+ @ParameterizedTest
+ @MethodSource("unsupported")
+ void rejectsWithMetricNameAndActionableReason(String expression, String reason) {
+ ConversionException exception = assertThrows(ConversionException.class,
+ () -> translate("SNOWFLAKE", expression));
+ assertTrue(exception.getMessage().contains("Metric 'net_value'"), exception.getMessage());
+ assertTrue(exception.getMessage().contains(reason), exception.getMessage());
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SUM(orders.amount)", "SUM([orders].[missing])", "COUNT(DISTINCT [orders].[amount])",
+ "COALESCE(SUM([orders].[amount]), 0)", "CASE WHEN TRUE THEN 1 ELSE 0 END", "not a formula",
+ "SUM([orders].[amount]) OVER ()", "SUM([orders].[amount] + [orders].[status])"})
+ void tableauCannotBypassParsingOrReferenceValidation(String expression) {
+ assertThrows(ConversionException.class, () -> translate("TABLEAU", expression));
+ }
+
+ @Test
+ void selectedDialectFailureNeverFallsBack() {
+ Map<String, Object> metric = metric("TABLEAU", "SUM([orders].[missing])");
+ metric.put("expression", Map.of("dialects", List.of(
+ Map.of("dialect", "ANSI_SQL", "expression", "SUM(orders.amount)"),
+ Map.of("dialect", "TABLEAU", "expression", "SUM([orders].[missing])"))));
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, source(), target()));
+ }
+
+ @Test
+ void rejectsDuplicateDialectAndMissingOrEmptyExpressions() {
+ Map<String, Object> metric = metric("SNOWFLAKE", "SUM(orders.amount)");
+ Map<String, Object> entry = Map.of("dialect", "SNOWFLAKE", "expression", "SUM(orders.amount)");
+ metric.put("expression", Map.of("dialects", List.of(entry, entry)));
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, source(), target()));
+ for (Object expression : List.of(Map.of(), Map.of("dialects", List.of()),
+ Map.of("dialects", List.of(Map.of("dialect", "BIGQUERY", "expression", "1"))))) {
+ metric.put("expression", expression);
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, source(), target()));
+ }
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", " "));
+ }
+
+ @Test
+ void outputDatatypeMustAgreeWithTheFormula() {
+ Map<String, Object> metric = metric("SNOWFLAKE", "AVG(orders.quantity)");
+ for (String datatype : List.of("Integer", "String", "Boolean", "Date", "Opaque", "Time")) {
+ metric.put("datatype", datatype);
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, source(), target()));
+ }
+ metric.put("datatype", "Decimal");
+ assertEquals("Number", MetricExpressionTranslator.translate(metric, source(), target()).dataType());
+ Map<String, Object> countMetric = metric("SNOWFLAKE", "COUNT(orders.quantity)");
+ countMetric.put("datatype", "Integer");
+ assertEquals("Number", MetricExpressionTranslator.translate(countMetric, source(), target()).dataType());
+ Map<String, Object> nullMetric = metric("SNOWFLAKE", "NULL");
+ nullMetric.put("datatype", "Decimal");
+ assertEquals("NULL", MetricExpressionTranslator.translate(nullMetric, source(), target()).expression());
+ }
+
+ @Test
+ void limitsNestingAndExpansionWithoutStackOverflow() {
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", "(".repeat(200) + "1" + ")".repeat(200)));
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", "-".repeat(200) + "1"));
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", "1+".repeat(5000) + "1"));
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", "1".repeat(32769)));
+ String formula = "SUM(orders.amount)";
+ for (int i = 0; i < 20; i++) formula = "NULLIF(" + formula + ", 0)";
+ String expanded = formula;
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", expanded));
+ }
+
+ @Test
+ void redundantParenthesesStayWithinTheBoundedFastParsingPath() {
+ assertTimeoutPreemptively(Duration.ofSeconds(5), () -> {
+ for (String dialect : List.of("SNOWFLAKE", "ANSI_SQL")) {
+ for (int depth : List.of(20, 60, 126)) {
+ String grouped = "(".repeat(depth) + "orders.amount" + ")".repeat(depth);
+ assertEquals("SUM([orders].[amount])", translate(dialect, "SUM(" + grouped + ")"));
+ assertEquals("SUM([orders].[amount])", translate(dialect,
+ "(".repeat(depth) + "SUM(orders.amount)" + ")".repeat(depth)));
+ }
+ assertEquals("SUM([orders].[amount])", translate(dialect, "SUM(" + "- ".repeat(128) + "orders.amount)"));
+ assertEquals("SUM((IF (NOT (NOT [orders].[active])) THEN 1 ELSE 0 END))",
+ translate(dialect, "SUM(CASE WHEN " + "NOT ".repeat(128) + "orders.active THEN 1 ELSE 0 END)"));
+ }
+ });
+ }
+
+ @Test
+ void normalizationRetainsOriginalLimitsAndOperandTypes() {
+ for (String dialect : List.of("SNOWFLAKE", "ANSI_SQL")) {
+ for (String expression : List.of("SUM(" + "- ".repeat(129) + "orders.amount)",
+ "SUM(CASE WHEN " + "NOT ".repeat(129) + "orders.active THEN 1 ELSE 0 END)",
+ "SUM(" + "(".repeat(128) + "orders.amount" + ")".repeat(128) + ")",
+ "SUM(- -orders.status)", "COUNT(- -orders.amount)",
+ "SUM(CASE WHEN NOT NOT NOT NOT orders.amount THEN 1 ELSE 0 END)",
+ "SUM(CASE WHEN orders.amount IS NOT NOT NOT NULL THEN 1 ELSE 0 END)",
+ "SUM(CASE WHEN orders.amount IS NOT NOT NOT NOT NULL THEN 1 ELSE 0 END)",
+ "COUNT(((DISTINCT orders.amount)))",
+ "SUM(orders.amount)) + (1", "SUM(--orders.amount)")) {
+ assertTrue(assertThrows(ConversionException.class, () -> translate(dialect, expression), expression)
+ .getMessage().contains("Metric 'net_value':"));
+ }
+ }
+ }
+
+ @Test
+ void redundantGroupingCannotTurnTuplesIntoFunctionArguments() {
+ for (String dialect : List.of("SNOWFLAKE", "ANSI_SQL")) {
+ for (String function : List.of("COALESCE", "ROUND")) {
+ for (int depth : List.of(1, 20)) {
+ String tuple = "(".repeat(depth) + "SUM(orders.amount), 2" + ")".repeat(depth);
+ assertTrue(assertThrows(ConversionException.class,
+ () -> translate(dialect, function + "(" + tuple + ")"))
+ .getMessage().contains("tuple-valued function arguments"));
+ }
+ }
+ assertEquals("IFNULL(SUM([orders].[amount]), 2)",
+ translate(dialect, "COALESCE((SUM(orders.amount)), 2)"));
+ }
+ }
+
+ @Test
+ void temporalAggregatesCanBeUsedInNumericPredicates() {
+ assertEquals("(IF (MIN([orders].[ordered]) = MAX([orders].[ordered])) THEN 1 ELSE 0 END)",
+ translate("SNOWFLAKE", "CASE WHEN MIN(orders.ordered) = MAX(orders.ordered) THEN 1 ELSE 0 END"));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"SUM(ALL orders.amount)", "SUM(UNIQUE orders.amount)",
+ "SUM(orders.amount IGNORE NULLS)", "SUM(orders.amount RESPECT NULLS)",
+ "SUM(orders.amount) IGNORE NULLS", "SUM(orders.amount LIMIT 1)",
+ "SUM(orders.amount HAVING MAX orders.quantity)", "SUM(orders.amount ORDER BY orders.quantity)",
+ "SUM(orders.amount) KEEP (DENSE_RANK LAST ORDER BY orders.quantity)",
+ "SUM(orders.amount).attribute", "private_schema.SUM(orders.amount)", "\"SUM\"(orders.amount)",
+ "SUM(CASE orders.amount WHEN 1 THEN 2 ELSE 0 END)", "N'prefixed'",
+ "orders.status ISNULL", "orders.status NOTNULL", "PRIOR orders.amount = orders.quantity",
+ "!orders.active", "(SELECT amount FROM orders)", "orders.amount IN (1, 2)",
+ "SUM(orders.amount) AS alias", "orders.amount(+) = orders.quantity"})
+ void parserAcceptanceNeverDiscardsUnsupportedSqlModifiers(String expression) {
+ assertThrows(ConversionException.class, () -> translate("SNOWFLAKE", expression), expression);
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"CEIL(AVG(orders.amount))", "FLOOR(AVG(orders.amount))",
+ "ROUND(AVG(orders.amount))", "ROUND(AVG(orders.amount), 0)", "ROUND(AVG(orders.amount), -2)"})
+ void integralRoundingSatisfiesAnIntegerMetricDeclaration(String expression) {
+ Map<String, Object> metric = metric("SNOWFLAKE", expression);
+ metric.put("datatype", "Integer");
+ String output = MetricExpressionTranslator.translate(metric, source(), target()).expression();
+ metric.put("expression", Map.of("dialects", List.of(Map.of("dialect", "TABLEAU", "expression", output))));
+ assertEquals(output, MetricExpressionTranslator.translate(metric, source(), target()).expression());
+ }
+
+ @Test
+ void positiveRoundingPrecisionDoesNotClaimAnIntegralResult() {
+ Map<String, Object> metric = metric("SNOWFLAKE", "ROUND(AVG(orders.amount), 2)");
+ metric.put("datatype", "Integer");
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, source(), target()));
+ assertEquals("0.12345678901234567890123456789", translate("SNOWFLAKE", "0.12345678901234567890123456789"));
+ }
+
+ @Test
+ void quotedDotsAndEscapesPreserveFieldReferenceBoundaries() {
+ Map<String, Object> source = Map.of("datasets", List.of(Map.of("name", "ORDER.ITEMS", "fields",
+ List.of(Map.of("name", "NET REVENUE", "datatype", "Decimal")))));
+ Map<String, Object> target = Map.of("semanticDataObjects", List.of(Map.of("apiName", "ORDER.ITEMS",
+ "semanticMeasurements", List.of(Map.of("apiName", "NET REVENUE", "dataType", "Number",
+ "dataObjectFieldName", "net__c")))));
+ for (String dialect : List.of("SNOWFLAKE", "ANSI_SQL")) {
+ assertEquals("SUM([ORDER.ITEMS].[NET REVENUE])", MetricExpressionTranslator.translate(
+ metric(dialect, "SUM(\"ORDER.ITEMS\".\"NET REVENUE\")"), source, target).expression());
+ var field = (MetricExpression.Field) SqlMetricExpressionParser.parse("\"a\"\"b.c\".\"d\"\"e\"", dialect);
+ assertEquals(List.of(new MetricFieldResolver.Identifier("a\"b.c", true),
+ new MetricFieldResolver.Identifier("d\"e", true)), field.parts());
+ }
+ assertThrows(IllegalArgumentException.class, () -> SqlMetricExpressionParser.parse("[a.b].[c]]d]", "ANSI_SQL"));
+ }
+
+ @Test
+ void separateAggregatesRequireConnectedDatasets() {
+ Map<String, Object> twoSources = Map.of("relationships", List.of(Map.of("name", "orders_returns",
+ "from", "orders", "to", "returns", "from_columns", List.of("amount"), "to_columns", List.of("amount"))),
+ "datasets", List.of(
+ Map.of("name", "orders", "fields", List.of(Map.of("name", "amount", "datatype", "Decimal"))),
+ Map.of("name", "returns", "fields", List.of(Map.of("name", "amount", "datatype", "Decimal")))));
+ Map<String, Object> twoTargets = new LinkedHashMap<>(Map.of("semanticDataObjects", List.of(
+ Map.of("apiName", "orders", "semanticMeasurements", List.of(Map.of("apiName", "amount", "dataObjectFieldName", "amount__c", "dataType", "Number"))),
+ Map.of("apiName", "returns", "semanticMeasurements", List.of(Map.of("apiName", "amount", "dataObjectFieldName", "amount__c", "dataType", "Number"))))));
+ Map<String, Object> metric = metric("SNOWFLAKE", "SUM(orders.amount) - SUM(returns.amount)");
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(metric, twoSources, twoTargets));
+ twoTargets.put("semanticRelationships", List.of(Map.of("apiName", "orders_returns", "isEnabled", true,
+ "leftSemanticDefinitionApiName", "orders", "rightSemanticDefinitionApiName", "returns", "criteria",
+ List.of(Map.of("leftSemanticFieldApiName", "amount", "rightSemanticFieldApiName", "amount")))));
+ assertEquals("(SUM([orders].[amount]) - SUM([returns].[amount]))",
+ MetricExpressionTranslator.translate(metric, twoSources, twoTargets).expression());
+ assertThrows(ConversionException.class, () -> MetricExpressionTranslator.translate(
+ metric("SNOWFLAKE", "SUM(orders.amount - returns.amount)"), twoSources, twoTargets));
+ }
+}
diff --git a/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricFieldResolverTest.java b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricFieldResolverTest.java
new file mode 100644
index 00000000..bb9d64fc
--- /dev/null
+++ b/converters/salesforce/src/test/java/org/apache/ossie/converter/MetricFieldResolverTest.java
@@ -0,0 +1,331 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.ossie.converter;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.List;
+import java.util.LinkedHashMap;
+import java.util.Map;
+import java.util.Set;
+import org.apache.ossie.converter.MetricFieldResolver.Identifier;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+class MetricFieldResolverTest {
+
+ @Test
+ void acceptsDatasetsConnectedByAnExportedRelationshipInEitherDirection() {
+ MetricFieldResolver resolver = graphResolver(List.of(
+ relationship("Orders", "Customers")), "Orders", "Customers");
+ resolver.validateDatasets(Set.of("Orders", "Customers"));
+ resolver.validateDatasets(Set.of("Orders"));
+ resolver.validateDatasets(Set.of());
+ }
+
+ @Test
+ void acceptsFactDatasetsConnectedThroughAnIntermediateSharedDimension() {
+ MetricFieldResolver resolver = graphResolver(List.of(
+ relationship("Orders", "Customers"), relationship("Returns", "Customers")),
+ "Orders", "Returns", "Customers");
+ resolver.validateDatasets(Set.of("Orders", "Returns"));
+ }
+
+ @Test
+ void rejectsDisconnectedDatasetsAndRelationshipsThroughMissingTargets() {
+ MetricFieldResolver resolver = graphResolver(List.of(
+ relationship("Orders", "MissingCustomers"), relationship("Returns", "MissingCustomers")),
+ "Orders", "Returns");
+ IllegalArgumentException error = assertThrows(IllegalArgumentException.class,
+ () -> resolver.validateDatasets(Set.of("Orders", "Returns")));
+ assertTrue(error.getMessage().contains("disconnected datasets Orders, Returns"), error.getMessage());
+ assertTrue(error.getMessage().contains("declare supported relationships"), error.getMessage());
+ }
+
+ @Test
+ void bindsQualifiedAndUniqueUnqualifiedSqlNamesToExportedApiNames() {
+ MetricFieldResolver resolver = resolver("Orders", "revenue", "Decimal", "Currency");
+ assertEquals("[Orders].[revenue]", resolver.resolve(sql("oRders", "REVENUE"), false).expression());
+ assertEquals("[Orders].[revenue]", resolver.resolve(sql("revenue"), false).expression());
+ assertEquals("Orders", resolver.resolve(sql("revenue"), false).dataset());
+ assertEquals("Decimal", resolver.resolve(sql("revenue"), false).datatype());
+ }
+
+ @Test
+ void normalizesQuotedSqlIdentifiersAsSpecified() {
+ MetricFieldResolver resolver = resolver("orders", "revenue", "Decimal", "Number");
+ assertEquals("[orders].[revenue]", resolver.resolve(
+ List.of(new Identifier("ORDERS", true), new Identifier("REVENUE", true)), false).expression());
+ assertError(resolver, List.of(new Identifier("orders", true), new Identifier("revenue", true)),
+ false, "Unknown dataset");
+ }
+
+ @Test
+ void resolvesQuotedDeclarationsWithoutRenamingExportedObjects() {
+ MetricFieldResolver resolver = resolver("\"Order Items\"", "\"Unit \"\"Price\"\"\"", "Decimal", "Number");
+ assertEquals("[\"Order Items\"].[\"Unit \"\"Price\"\"\"]", resolver.resolve(
+ List.of(new Identifier("Order Items", true), new Identifier("Unit \"Price\"", true)), false)
+ .expression());
+ assertError(resolver, sql("ORDER ITEMS", "UNIT PRICE"), false, "Unknown dataset");
+ }
+
+ @Test
+ void tableauReferencesUseExactApiNamesAndRequireDataset() {
+ MetricFieldResolver resolver = resolver("Orders", "revenue", "Integer", "Number");
+ assertEquals("[Orders].[revenue]", resolver.resolve(sql("Orders", "revenue"), true).expression());
+ assertEquals("[Orders].[revenue]", resolver.resolve(sql("orders", "revenue"), false).expression());
+ assertError(resolver, sql("orders", "revenue"), true, "Unknown dataset");
+ assertError(resolver, sql("Orders", "Revenue"), true, "Unknown field");
+ assertError(resolver, sql("revenue"), true, "must use [dataset].[field]");
+ }
+
+ @Test
+ void rejectsPhysicalSourceNamesAndUndeclaredFields() {
+ MetricFieldResolver resolver = resolver("Orders", "revenue", "Decimal", "Number");
+ assertError(resolver, sql("Orders__dll", "revenue"), false, "Unknown dataset");
+ assertError(resolver, sql("Orders", "revenue__c"), false, "Unknown field");
+ assertError(resolver, sql("warehouse", "Orders", "revenue"), false, "physical source paths");
+ assertError(resolver, List.of(), false, "must name a declared field");
+ }
+
+ @Test
+ void rejectsAmbiguousUnqualifiedFieldsAcrossDatasets() {
+ Map<String, Object> orders = dataset("Orders", field("amount", "Decimal"));
+ Map<String, Object> returns = dataset("Returns", field("amount", "Decimal"));
+ MetricFieldResolver resolver = new MetricFieldResolver(
+ Map.of("datasets", List.of(orders, returns)),
+ Map.of("semanticDataObjects", List.of(targetDataset("Orders", targetField("amount", "Number")),
+ targetDataset("Returns", targetField("amount", "Number")))));
+ assertError(resolver, sql("amount"), false, "Ambiguous field");
+ assertEquals("[Orders].[amount]", resolver.resolve(sql("Orders", "amount"), false).expression());
+ }
+
+ @Test
+ void rejectsDuplicateDatasetDeclarationsEvenWhenOnlyOneHasTheField() {
+ MetricFieldResolver resolver = new MetricFieldResolver(Map.of("datasets", List.of(
+ dataset("Orders", field("amount", "Decimal")), dataset("ORDERS", field("other", "Decimal")))),
+ Map.of());
+ assertError(resolver, sql("Orders", "amount"), false, "Ambiguous dataset");
+ assertError(resolver, sql("amount"), false, "Ambiguous dataset");
+ }
+
+ @Test
+ void rejectsDuplicateFieldDeclarations() {
+ MetricFieldResolver resolver = new MetricFieldResolver(Map.of("datasets", List.of(
+ dataset("Orders", field("amount", "Decimal"), field("AMOUNT", "Decimal")))), Map.of());
+ assertError(resolver, sql("Orders", "amount"), false, "Ambiguous field");
+ }
+
+ @Test
+ void checksThatDeclaredFieldsWereActuallyExportedAsDirectFields() {
+ Map<String, Object> source = Map.of("datasets", List.of(dataset("Orders", field("amount", "Decimal"))));
+ MetricFieldResolver missingDataset = new MetricFieldResolver(source, Map.of());
+ assertError(missingDataset, sql("Orders", "amount"), false, "was not exported");
+
+ MetricFieldResolver calculatedField = new MetricFieldResolver(source, Map.of(
+ "semanticDataObjects", List.of(targetDataset("Orders")),
+ "semanticCalculatedDimensions", List.of(Map.of("apiName", "amount", "expression", "1 + 2"))));
+ assertError(calculatedField, sql("Orders", "amount"), false, "calculated or omitted fields");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"profit+tax", "profit-tax", "profit=tax", "1", "TRUE", "(profit)",
+ "Orders.profit", "\"profit\"", "[profit]", "SUM(profit)", "profit;tax", ""})
+ void rejectsExpressionsMisclassifiedAsPhysicalFields(String binding) {
+ Map<String, Object> target = new LinkedHashMap<>(targetField("revenue", "Number"));
+ if (binding.isEmpty()) target.remove("dataObjectFieldName");
+ else target.put("dataObjectFieldName", binding);
+ MetricFieldResolver resolver = new MetricFieldResolver(
+ Map.of("datasets", List.of(dataset("Orders", field("revenue", "Decimal")))),
+ Map.of("semanticDataObjects", List.of(targetDataset("Orders", target))));
+ assertError(resolver, sql("Orders", "revenue"), false, "revenue");
+ }
+
+ @Test
+ void rejectsDuplicateExportedObjectsAndFieldsAcrossKinds() {
+ Map<String, Object> source = Map.of("datasets", List.of(dataset("Orders", field("amount", "Decimal"))));
+ Map<String, Object> target = targetDataset("Orders", targetField("amount", "Number"));
+ MetricFieldResolver duplicateDatasets = new MetricFieldResolver(source,
+ Map.of("semanticDataObjects", List.of(target, target)));
+ assertError(duplicateDatasets, sql("amount"), false, "Ambiguous exported Salesforce dataset");
+
+ Map<String, Object> duplicateFields = Map.of("apiName", "Orders",
+ "semanticDimensions", List.of(targetField("amount", "Number")),
+ "semanticMeasurements", List.of(targetField("amount", "Number")));
+ MetricFieldResolver resolver = new MetricFieldResolver(source,
+ Map.of("semanticDataObjects", List.of(duplicateFields)));
+ assertError(resolver, sql("amount"), false, "Ambiguous exported Salesforce field");
+ }
+
+ @Test
+ void infersMissingPortableTypeOnlyFromKnownExportedType() {
+ MetricFieldResolver resolver = new MetricFieldResolver(
+ Map.of("datasets", List.of(dataset("Orders", Map.of("name", "amount")))),
+ Map.of("semanticDataObjects", List.of(targetDataset("Orders", targetField("amount", "Currency")))));
+ assertEquals("Decimal", resolver.resolve(sql("amount"), false).datatype());
+ }
+
+ @Test
+ void rejectsUnknownOrConflictingTypes() {
+ assertError(resolver("Orders", "amount", "String", "Number"), sql("amount"), false,
+ "use compatible field types");
+ assertError(resolver("Orders", "amount", "Opaque", "Geo"), sql("amount"), false,
+ "no supported datatype");
+ MetricFieldResolver resolver = new MetricFieldResolver(
+ Map.of("datasets", List.of(dataset("Orders", Map.of("name", "amount")))),
+ Map.of("semanticDataObjects", List.of(targetDataset("Orders", targetField("amount", "Geo")))));
+ assertError(resolver, sql("amount"), false, "no supported datatype");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"bad[field", "bad]field", "bad\nfield", "bad\tfield", "bad\u0000field"})
+ void rejectsUnrepresentableExportedNames(String name) {
+ MetricFieldResolver resolver = resolver("Orders", name, "Decimal", "Number");
+ assertError(resolver, sql("Orders", name), true, "cannot be represented safely");
+ }
+
+ private static MetricFieldResolver resolver(String dataset, String field, String sourceType, String targetType) {
+ return new MetricFieldResolver(Map.of("datasets", List.of(dataset(dataset, field(field, sourceType)))),
+ Map.of("semanticDataObjects", List.of(targetDataset(dataset, targetField(field, targetType)))));
+ }
+
+ @Test
+ void disabledRelationshipsDoNotConnectDatasets() {
+ Map<String, Object> disabled = new LinkedHashMap<>(relationship("Orders", "Returns"));
+ disabled.put("isEnabled", false);
+ MetricFieldResolver resolver = graphResolver(List.of(disabled), "Orders", "Returns");
+ assertThrows(IllegalArgumentException.class,
+ () -> resolver.validateDatasets(java.util.Set.of("Orders", "Returns")));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"changed endpoint", "swapped keys", "empty criteria", "missing criterion",
+ "missing join field", "formula join", "missing enabled", "missing declaration", "duplicate declaration"})
+ void unverifiedRelationshipsCannotEstablishConnectivity(String defect) {
+ Map<String, Object> valid = relationship("Orders", "Customers");
+ Map<String, Object> changed = new LinkedHashMap<>(valid);
+ List<Map<String, Object>> declarations = List.of(sourceRelationship(valid));
+ switch (defect) {
+ case "changed endpoint" -> changed.put("rightSemanticDefinitionApiName", "Returns");
+ case "swapped keys" -> changed.put("criteria", List.of(criterion("id", "tenant"), criterion("tenant", "id")));
+ case "empty criteria" -> changed.put("criteria", List.of());
+ case "missing criterion" -> changed.put("criteria", List.of(criterion("id", "id")));
+ case "missing join field" -> {
+ changed.put("criteria", List.of(criterion("missing", "id"), criterion("tenant", "tenant")));
+ Map<String, Object> source = new LinkedHashMap<>(sourceRelationship(valid));
+ source.put("from_columns", List.of("missing", "tenant"));
+ declarations = List.of(source);
+ }
+ case "formula join" -> changed.put("criteria", List.of(
+ Map.of("leftSemanticFieldApiName", "id", "rightSemanticFieldApiName", "id", "leftFieldType", "Formula"),
+ criterion("tenant", "tenant")));
+ case "missing enabled" -> changed.remove("isEnabled");
+ case "missing declaration" -> declarations = List.of();
+ case "duplicate declaration" -> declarations = List.of(sourceRelationship(valid), sourceRelationship(valid));
+ default -> throw new AssertionError(defect);
+ }
+ MetricFieldResolver resolver = graphResolver(declarations, List.of(changed), "Orders", "Customers", "Returns");
+ Set<String> referenced = Set.of("Orders", defect.equals("changed endpoint") ? "Returns" : "Customers");
+ IllegalArgumentException error = assertThrows(IllegalArgumentException.class,
+ () -> resolver.validateDatasets(referenced), defect);
+ assertTrue(error.getMessage().contains("disconnected"), error.getMessage());
+ }
+
+ @Test
+ void computedPhysicalJoinKeyCannotEstablishConnectivity() {
+ Map<String, Object> edge = relationship("Orders", "Customers");
+ Map<String, Object> computed = new LinkedHashMap<>(targetField("id", "Number"));
+ computed.put("dataObjectFieldName", "profit+tax");
+ MetricFieldResolver resolver = new MetricFieldResolver(Map.of(
+ "datasets", List.of(dataset("Orders", field("id", "Integer"), field("tenant", "Integer")),
+ dataset("Customers", field("id", "Integer"), field("tenant", "Integer"))),
+ "relationships", List.of(sourceRelationship(edge))), Map.of(
+ "semanticDataObjects", List.of(targetDataset("Orders", computed, targetField("tenant", "Number")),
+ targetDataset("Customers", targetField("id", "Number"), targetField("tenant", "Number"))),
+ "semanticRelationships", List.of(edge)));
+ assertThrows(IllegalArgumentException.class,
+ () -> resolver.validateDatasets(Set.of("Orders", "Customers")));
+ }
+
+ private static MetricFieldResolver graphResolver(List<Map<String, Object>> relationships, String... datasets) {
+ return graphResolver(relationships.stream().map(MetricFieldResolverTest::sourceRelationship).toList(),
+ relationships, datasets);
+ }
+
+ private static MetricFieldResolver graphResolver(List<Map<String, Object>> declarations,
+ List<Map<String, Object>> relationships, String... datasets) {
+ return new MetricFieldResolver(Map.of(
+ "datasets", java.util.Arrays.stream(datasets)
+ .map(name -> dataset(name, field("id", "Integer"), field("tenant", "Integer"))).toList(),
+ "relationships", declarations), Map.of(
+ "semanticDataObjects", java.util.Arrays.stream(datasets)
+ .map(name -> targetDataset(name, targetField("id", "Number"), targetField("tenant", "Number"))).toList(),
+ "semanticRelationships", relationships));
+ }
+
+ private static Map<String, Object> relationship(String left, String right) {
+ return Map.of("apiName", left + "_to_" + right, "leftSemanticDefinitionApiName", left,
+ "rightSemanticDefinitionApiName", right, "isEnabled", true,
+ "criteria", List.of(criterion("id", "id"), criterion("tenant", "tenant")));
+ }
+
+ private static Map<String, Object> criterion(String left, String right) {
+ return Map.of("leftSemanticFieldApiName", left, "rightSemanticFieldApiName", right);
+ }
+
+ private static Map<String, Object> sourceRelationship(Map<String, Object> target) {
+ return Map.of("name", target.get("apiName"), "from", target.get("leftSemanticDefinitionApiName"),
+ "to", target.get("rightSemanticDefinitionApiName"),
+ "from_columns", List.of("id", "tenant"), "to_columns", List.of("id", "tenant"));
+ }
+
+ private static List<Identifier> sql(String... parts) {
+ return java.util.Arrays.stream(parts).map(part -> new Identifier(part, false)).toList();
+ }
+
+ @SafeVarargs
+ private static Map<String, Object> dataset(String name, Map<String, Object>... fields) {
+ return Map.of("name", name, "source", name + "__dll", "fields", List.of(fields));
+ }
+
+ private static Map<String, Object> field(String name, String datatype) {
+ return Map.of("name", name, "datatype", datatype, "expression", Map.of("dialects", List.of(
+ Map.of("dialect", "ANSI_SQL", "expression", "physical_column__c"))));
+ }
+
+ @SafeVarargs
+ private static Map<String, Object> targetDataset(String name, Map<String, Object>... fields) {
+ return Map.of("apiName", name, "semanticMeasurements", List.of(fields));
+ }
+
+ private static Map<String, Object> targetField(String name, String datatype) {
+ return Map.of("apiName", name, "dataType", datatype, "dataObjectFieldName", "physical_column__c");
+ }
+
+ private static void assertError(MetricFieldResolver resolver, List<Identifier> parts,
+ boolean tableau, String expected) {
+ IllegalArgumentException error = assertThrows(IllegalArgumentException.class,
+ () -> resolver.resolve(parts, tableau));
+ assertTrue(error.getMessage().contains(expected), error.getMessage());
+ }
+}