[PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv opened a new pull request, #247:
URL: https://github.com/apache/ossie/pull/247

   ## Summary
   
   Adds an offline, bidirectional converter between Apache Ossie YAML and NVIDIA GSF semantic-model YAML.
   
   - `ossie-gsf export`: Ossie → NVIDIA GSF
   - `ossie-gsf import`: NVIDIA GSF → Ossie
   - Python string-in/string-out API
   - No GSF, Neo4j, database, or network connection required for conversion
   - NVIDIA-specific metadata uses the `NVIDIA_GSF` custom-extension identifier
   
   ## Why
   
   GSF stores its active semantic layer in Neo4j. Its standalone semantic-model YAML provides a portable representation that can be versioned, reviewed, and converted without connecting directly to a GSF deployment.
   
   ## Mapping
   
   - Semantic model → GSF model
   - Dataset → Term
   - Physical field → ColumnAttribute
   - Computed field or metric → SqlAttribute
   - Relationship → semantic foreign key
   - Expression dialects and vendor metadata are preserved for round trips
   
   ## Testing
   
   - 19 converter tests
   - Checked-in Ossie/GSF fixture pair
   - Bidirectional semantic round-trip coverage
   - Generated Ossie YAML validated with the repository’s official schema and SQL validator
   - Manually verified through the full CLI and live Neo4j import/export path
   
   ## Notes
   
   The physical catalog must already be ingested before the generated model is loaded into GSF.
   
   The corresponding GSF import/export implementation is available on:
   https://github.com/NVIDIA/GSF/tree/feature/ossie-import-export


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666615579


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:

Review Comment:
   Agreed, that was fragile. Fixed in 972e486: the check now compares major and minor only, so any 0.2.x is accepted including .dev releases, and the series is derived from OSSIE_VERSION itself rather than a second constant that could drift out of sync. Output is still written as the version the converter targets. Tests cover 0.2.0, 0.2.1 and 0.2.7.dev3 as accepted, and 0.1.9, 0.3.0 and 1.0.0 as rejected.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["johntomcat7408-cmyk (via GitHub)" <gi...@apache.org>]

johntomcat7408-cmyk commented on PR #247:
URL: https://github.com/apache/ossie/pull/247#issuecomment-5090248978

   Superseded by #275 (https://github.com/apache/ossie/pull/275). The replacement preserves the original contribution context, consolidates the change into one commit, and includes the review fixes from this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3663033581


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(

Review Comment:
   **Date-part filtering is a load-bearing denylist for Ossie-origin catalogs** (columns are synthesized from expression identifiers). Two failure modes: (a) a real column literally named `day`/`month`/`q`/`us` used as a genuine argument to a recognized date function is dropped; (b) a unit argument to a date function *not* in `_DATE_FUNCTIONS` gets added as a fictional physical column. The single-letter/abbrev entries (`q`, `dd`, `mm`, `us`, `ns`) widen the false-drop surface. Partly inherent to synthesizing a catalog from SQL text — mostly flagging for awareness.



##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:

Review Comment:
   **Exact version pin is fragile.** This rejects anything but `0.2.0.dev0` via exact equality. Since the spec is actively on a `.dev0` line, the next version bump silently breaks every real model until this constant is hand-edited. Consider accepting a compatible range (e.g. major/minor) rather than exact match.



##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:
+        raise GSFConversionError(
+            f"Unsupported Ossie version {root.get('version')!r}; "
+            f"supported version is {OSSIE_VERSION!r}"
+        )
+    models = root.get("semantic_model")
+    if not isinstance(models, list) or len(models) != 1:
+        raise GSFConversionError("Ossie input must contain exactly one semantic model")
+    model = models[0]
+    if not isinstance(model, dict) or not model.get("name"):
+        raise GSFConversionError("Ossie semantic model requires a name")
+    return root, model
+
+
+def _load_yaml(value: str, label: str) -> dict[str, Any]:
+    try:
+        root = yaml.safe_load(value)
+    except yaml.YAMLError as exc:
+        raise GSFConversionError(f"Invalid {label} YAML: {exc}") from exc
+    if not isinstance(root, dict):
+        raise GSFConversionError(f"Invalid {label} YAML: expected a root mapping")
+    return root
+
+
+def _validate_relationship(
+    relationship: Any,
+    datasets: Mapping[str, Any],
+) -> dict[str, Any]:
+    if not isinstance(relationship, dict) or not relationship.get("name"):
+        raise GSFConversionError("Every Ossie relationship needs a name")
+    from_name = relationship.get("from")
+    to_name = relationship.get("to")
+    if from_name not in datasets or to_name not in datasets:
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} references an unknown dataset"
+        )
+    from_columns = relationship.get("from_columns") or []
+    to_columns = relationship.get("to_columns") or []
+    if not from_columns or len(from_columns) != len(to_columns):
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} must have equal, "
+            "non-empty column lists"
+        )
+    return relationship
+
+
+def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
+    if not isinstance(value, dict):
+        raise GSFConversionError(f"{name!r} has no valid expression")
+    dialects = value.get("dialects")
+    if not isinstance(dialects, list) or not dialects:
+        raise GSFConversionError(f"{name!r} requires at least one expression dialect")
+    result = [
+        {
+            "dialect": str(item["dialect"]),
+            "expression": str(item["expression"]),
+        }
+        for item in dialects
+        if isinstance(item, dict)
+        and item.get("dialect")
+        and item.get("expression") is not None
+    ]
+    if not result:
+        raise GSFConversionError(f"{name!r} has no usable expression dialect")
+    return result
+
+
+def _pick_expression(expressions: list[dict[str, str]], name: str) -> str:
+    for expression in expressions:
+        if expression["dialect"].upper() == "ANSI_SQL":
+            return expression["expression"]
+    if expressions:
+        return expressions[0]["expression"]
+    raise GSFConversionError(f"{name!r} has no usable expression")
+
+
+def _simple_source_column(
+    expression: str,
+    dataset_name: str,
+    table_name: str,
+) -> str | None:
+    match = _SIMPLE_COLUMN.fullmatch(expression.strip())
+    if not match:
+        return None
+    qualifier = match.group("qualifier")
+    if qualifier and qualifier not in (dataset_name, table_name):
+        return None
+    return match.group("column")
+
+
+def _field_table_refs(
+    field: Mapping[str, Any],
+    expression: str,
+    owner: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(field)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(field.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"SQL attribute {field.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    result = list(dict.fromkeys([owner, *refs]))
+    _validate_single_database_refs(
+        result,
+        datasets,
+        f"SQL attribute {field.get('name')!r}",
+        sql=reference_sql,
+    )
+    return result
+
+
+def _metric_table_refs(
+    metric: Mapping[str, Any],
+    expression: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(metric)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(metric.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    if refs:
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if len(datasets) == 1:
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if extension.get("entity") == "custom_analysis" and extension.get("sql"):
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    raise GSFConversionError(
+        f"Metric {metric.get('name')!r} does not identify a source dataset; "
+        "qualify a referenced column or add NVIDIA_GSF table_refs"
+    )
+
+
+def _validate_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    name: str,
+) -> None:
+    unknown = [ref for ref in refs if ref not in datasets]
+    if unknown:
+        raise GSFConversionError(
+            f"{name!r} has unknown NVIDIA_GSF table_refs: {', '.join(unknown)}"
+        )
+
+
+def _validate_single_database_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    context: str,
+    *,
+    sql: str,
+) -> None:
+    databases = {
+        str(datasets[ref]["source"]["database"]) for ref in refs if ref in datasets
+    }
+    databases.update(
+        table.catalog for table in _parse_sql(sql).find_all(exp.Table) if table.catalog
+    )
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _referenced_datasets(
+    sql: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    references: list[str] = []
+    parsed = _parse_sql(sql)
+    for table in parsed.find_all(exp.Table):
+        matches = [
+            name
+            for name, context in datasets.items()
+            if table.name == str(context["source"]["table"])
+            and (not table.db or table.db == str(context["source"]["schema"]))
+            and (
+                not table.catalog or table.catalog == str(context["source"]["database"])
+            )
+        ]
+        if matches:
+            match = matches[0]
+            if match not in references:
+                references.append(match)
+    for column in parsed.find_all(exp.Column):
+        qualifier = column.table
+        if not qualifier:
+            continue
+        matches = [
+            name
+            for name, context in datasets.items()
+            if qualifier in (name, str(context["source"]["table"]))
+        ]
+        if len(matches) == 1 and matches[0] not in references:
+            references.append(matches[0])
+    return references
+
+
+def _collect_expression_columns(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> None:
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        if not dataset_name:
+            continue
+        context = datasets[dataset_name]
+        source_key = _source_key(context["source"])
+        known_table = source_key in preserved["tables"]
+        if known_table and (*source_key, column.name) not in preserved["columns"]:
+            # A GSF-sourced catalog is authoritative: an identifier that is not
+            # already a column of the table is SQL syntax, not physical data.
+            continue
+        _add_catalog_column(context, column.name)
+
+
+def _sql_column_ids(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    result: list[str] = []
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        column_id = (
+            column_ids.get((dataset_name, column.name)) if dataset_name else None
+        )
+        if column_id and column_id not in result:
+            result.append(column_id)
+    return result
+
+
+def _column_dataset(
+    column: exp.Column,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, Any],
+) -> str | None:
+    qualifier = column.table
+    if qualifier:
+        matches = [
+            name
+            for name, context in datasets.items()
+            if name in refs and qualifier in (name, str(context["source"]["table"]))
+        ]
+        return matches[0] if len(matches) == 1 else None
+    if owner:
+        return owner
+    return refs[0] if len(refs) == 1 else None
+
+
+def _sql_columns(sql: str) -> list[exp.Column]:
+    return [
+        column
+        for column in _parse_sql(sql).find_all(exp.Column)
+        if not _is_date_part_unit(column)
+    ]
+
+
+def _is_date_part_unit(column: exp.Column) -> bool:
+    """Report whether a parsed column is really a date-part keyword.
+
+    ``DATEDIFF(day, a, b)`` and friends put the unit in an argument slot that
+    sqlglot parses as an unqualified column, which would otherwise be mistaken
+    for a physical catalog column.
+    """
+    parent = column.parent
+    if column.table or column.name.lower() not in _DATE_PART_UNITS:
+        return False
+    if not isinstance(parent, exp.Func):
+        return False
+    if isinstance(parent, exp.Anonymous):
+        return str(parent.this).lower() in _DATE_FUNCTIONS
+    try:
+        names = parent.sql_names()
+    except (AttributeError, IndexError):
+        return False
+    return any(name.lower() in _DATE_FUNCTIONS for name in names)
+
+
+def _parse_sql(sql: str) -> exp.Expression:

Review Comment:
   **SQL parsed with no dialect, and any `ParseError` aborts the whole conversion.** Converter-emitted SQL is ANSI-ish so that's fine, but *preserved* native SQL on a GSF→Ossie import can be dialect-specific (the README notes per-connection dialects). A model that imported and validated cleanly could then fail to re-export. Worth either parsing preserved SQL more leniently or documenting the assumption.



##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:
+        raise GSFConversionError(
+            f"Unsupported Ossie version {root.get('version')!r}; "
+            f"supported version is {OSSIE_VERSION!r}"
+        )
+    models = root.get("semantic_model")
+    if not isinstance(models, list) or len(models) != 1:
+        raise GSFConversionError("Ossie input must contain exactly one semantic model")
+    model = models[0]
+    if not isinstance(model, dict) or not model.get("name"):
+        raise GSFConversionError("Ossie semantic model requires a name")
+    return root, model
+
+
+def _load_yaml(value: str, label: str) -> dict[str, Any]:
+    try:
+        root = yaml.safe_load(value)
+    except yaml.YAMLError as exc:
+        raise GSFConversionError(f"Invalid {label} YAML: {exc}") from exc
+    if not isinstance(root, dict):
+        raise GSFConversionError(f"Invalid {label} YAML: expected a root mapping")
+    return root
+
+
+def _validate_relationship(
+    relationship: Any,
+    datasets: Mapping[str, Any],
+) -> dict[str, Any]:
+    if not isinstance(relationship, dict) or not relationship.get("name"):
+        raise GSFConversionError("Every Ossie relationship needs a name")
+    from_name = relationship.get("from")
+    to_name = relationship.get("to")
+    if from_name not in datasets or to_name not in datasets:
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} references an unknown dataset"
+        )
+    from_columns = relationship.get("from_columns") or []
+    to_columns = relationship.get("to_columns") or []
+    if not from_columns or len(from_columns) != len(to_columns):
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} must have equal, "
+            "non-empty column lists"
+        )
+    return relationship
+
+
+def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
+    if not isinstance(value, dict):
+        raise GSFConversionError(f"{name!r} has no valid expression")
+    dialects = value.get("dialects")
+    if not isinstance(dialects, list) or not dialects:
+        raise GSFConversionError(f"{name!r} requires at least one expression dialect")
+    result = [
+        {
+            "dialect": str(item["dialect"]),
+            "expression": str(item["expression"]),
+        }
+        for item in dialects
+        if isinstance(item, dict)
+        and item.get("dialect")
+        and item.get("expression") is not None
+    ]
+    if not result:
+        raise GSFConversionError(f"{name!r} has no usable expression dialect")
+    return result
+
+
+def _pick_expression(expressions: list[dict[str, str]], name: str) -> str:
+    for expression in expressions:
+        if expression["dialect"].upper() == "ANSI_SQL":
+            return expression["expression"]
+    if expressions:
+        return expressions[0]["expression"]
+    raise GSFConversionError(f"{name!r} has no usable expression")
+
+
+def _simple_source_column(
+    expression: str,
+    dataset_name: str,
+    table_name: str,
+) -> str | None:
+    match = _SIMPLE_COLUMN.fullmatch(expression.strip())
+    if not match:
+        return None
+    qualifier = match.group("qualifier")
+    if qualifier and qualifier not in (dataset_name, table_name):
+        return None
+    return match.group("column")
+
+
+def _field_table_refs(
+    field: Mapping[str, Any],
+    expression: str,
+    owner: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(field)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(field.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"SQL attribute {field.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    result = list(dict.fromkeys([owner, *refs]))
+    _validate_single_database_refs(
+        result,
+        datasets,
+        f"SQL attribute {field.get('name')!r}",
+        sql=reference_sql,
+    )
+    return result
+
+
+def _metric_table_refs(
+    metric: Mapping[str, Any],
+    expression: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(metric)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(metric.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    if refs:
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if len(datasets) == 1:
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if extension.get("entity") == "custom_analysis" and extension.get("sql"):
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    raise GSFConversionError(
+        f"Metric {metric.get('name')!r} does not identify a source dataset; "
+        "qualify a referenced column or add NVIDIA_GSF table_refs"
+    )
+
+
+def _validate_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    name: str,
+) -> None:
+    unknown = [ref for ref in refs if ref not in datasets]
+    if unknown:
+        raise GSFConversionError(
+            f"{name!r} has unknown NVIDIA_GSF table_refs: {', '.join(unknown)}"
+        )
+
+
+def _validate_single_database_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    context: str,
+    *,
+    sql: str,
+) -> None:
+    databases = {
+        str(datasets[ref]["source"]["database"]) for ref in refs if ref in datasets
+    }
+    databases.update(
+        table.catalog for table in _parse_sql(sql).find_all(exp.Table) if table.catalog
+    )
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _referenced_datasets(
+    sql: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    references: list[str] = []
+    parsed = _parse_sql(sql)
+    for table in parsed.find_all(exp.Table):
+        matches = [
+            name
+            for name, context in datasets.items()
+            if table.name == str(context["source"]["table"])
+            and (not table.db or table.db == str(context["source"]["schema"]))
+            and (
+                not table.catalog or table.catalog == str(context["source"]["database"])
+            )
+        ]
+        if matches:
+            match = matches[0]
+            if match not in references:
+                references.append(match)
+    for column in parsed.find_all(exp.Column):
+        qualifier = column.table
+        if not qualifier:
+            continue
+        matches = [
+            name
+            for name, context in datasets.items()
+            if qualifier in (name, str(context["source"]["table"]))
+        ]
+        if len(matches) == 1 and matches[0] not in references:
+            references.append(matches[0])
+    return references
+
+
+def _collect_expression_columns(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> None:
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        if not dataset_name:
+            continue
+        context = datasets[dataset_name]
+        source_key = _source_key(context["source"])
+        known_table = source_key in preserved["tables"]
+        if known_table and (*source_key, column.name) not in preserved["columns"]:
+            # A GSF-sourced catalog is authoritative: an identifier that is not
+            # already a column of the table is SQL syntax, not physical data.
+            continue
+        _add_catalog_column(context, column.name)
+
+
+def _sql_column_ids(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    result: list[str] = []
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        column_id = (
+            column_ids.get((dataset_name, column.name)) if dataset_name else None
+        )
+        if column_id and column_id not in result:
+            result.append(column_id)
+    return result
+
+
+def _column_dataset(
+    column: exp.Column,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, Any],
+) -> str | None:
+    qualifier = column.table
+    if qualifier:
+        matches = [
+            name
+            for name, context in datasets.items()
+            if name in refs and qualifier in (name, str(context["source"]["table"]))
+        ]
+        return matches[0] if len(matches) == 1 else None
+    if owner:
+        return owner
+    return refs[0] if len(refs) == 1 else None
+
+
+def _sql_columns(sql: str) -> list[exp.Column]:
+    return [
+        column
+        for column in _parse_sql(sql).find_all(exp.Column)
+        if not _is_date_part_unit(column)
+    ]
+
+
+def _is_date_part_unit(column: exp.Column) -> bool:
+    """Report whether a parsed column is really a date-part keyword.
+
+    ``DATEDIFF(day, a, b)`` and friends put the unit in an argument slot that
+    sqlglot parses as an unqualified column, which would otherwise be mistaken
+    for a physical catalog column.
+    """
+    parent = column.parent
+    if column.table or column.name.lower() not in _DATE_PART_UNITS:
+        return False
+    if not isinstance(parent, exp.Func):
+        return False
+    if isinstance(parent, exp.Anonymous):
+        return str(parent.this).lower() in _DATE_FUNCTIONS
+    try:
+        names = parent.sql_names()
+    except (AttributeError, IndexError):
+        return False
+    return any(name.lower() in _DATE_FUNCTIONS for name in names)
+
+
+def _parse_sql(sql: str) -> exp.Expression:
+    try:
+        return parse_one(sql)
+    except (ParseError, ValueError) as exc:
+        raise GSFConversionError(
+            f"Unable to parse SQL expression {sql!r}: {exc}"
+        ) from exc
+
+
+def _first_resolvable_column_ids(
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    for ref in refs:
+        for column in datasets[ref]["columns"]:
+            column_id = column_ids.get((ref, column))
+            if column_id:
+                return [column_id]
+    return []
+
+
+def _wrap_expression(
+    expression: str,
+    name: str,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    relationships: list[dict[str, Any]],
+) -> str:
+    stripped = expression.strip()
+    if stripped.upper().startswith(("SELECT ", "SELECT\n", "WITH ", "WITH\n")):
+        return stripped
+    if not refs:
+        raise GSFConversionError(f"Cannot determine a source table for {name!r}")
+    anchor_name = refs[0]
+    from_sql = (
+        f"{_qualified_table(datasets[anchor_name]['source'])} "
+        f"AS {_quote_identifier(anchor_name)}"
+    )
+    joined = {anchor_name}
+    remaining = set(refs[1:])
+    joins: list[str] = []
+    while remaining:
+        matched = False
+        for relationship in relationships:
+            left = str(relationship["from"])
+            right = str(relationship["to"])
+            if left in joined and right in remaining:
+                new_name = right
+            elif right in joined and left in remaining:
+                new_name = left
+            else:
+                continue
+            conditions = [
+                f"{_quote_identifier(left)}.{_quote_identifier(str(left_column))} = "
+                f"{_quote_identifier(right)}.{_quote_identifier(str(right_column))}"
+                for left_column, right_column in zip(
+                    relationship["from_columns"],
+                    relationship["to_columns"],
+                    strict=True,
+                )
+            ]
+            joins.append(
+                f"JOIN {_qualified_table(datasets[new_name]['source'])} "
+                f"AS {_quote_identifier(new_name)} ON {' AND '.join(conditions)}"
+            )
+            joined.add(new_name)
+            remaining.remove(new_name)
+            matched = True
+            break
+        if not matched:
+            missing = min(remaining)
+            raise GSFConversionError(
+                f"{name!r} references disconnected dataset {missing!r}; "
+                "declare a relationship connecting all referenced datasets"
+            )
+    suffix = f" {' '.join(joins)}" if joins else ""
+    return f"SELECT {stripped} AS {_quote_identifier(name)} FROM {from_sql}{suffix}"
+
+
+def _expression_from_sql(sql: str) -> str:
+    try:
+        parsed = parse_one(sql)
+    except (ParseError, ValueError):
+        return sql
+    select = parsed.find(exp.Select)
+    if select is None or not select.expressions:
+        return sql
+    expression = select.expressions[0]
+    if isinstance(expression, exp.Alias):
+        expression = expression.this
+    return expression.sql()
+
+
+def _expression_matches(current: str, emitted: Any) -> bool:
+    if not isinstance(emitted, str) or not emitted.strip():
+        return False
+    try:
+        return parse_one(current).sql() == parse_one(emitted).sql()
+    except (ParseError, ValueError):
+        return current.strip() == emitted.strip()
+
+
+def _ossie_expression(expression: str) -> dict[str, Any]:

Review Comment:
   Minor: on import this always labels expressions `ANSI_SQL` even when the source GSF SQL was dialect-specific — a potential mislabel, though harmless for validation.



##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,

Review Comment:
   **Design note (not a blocker):** embedding the entire native document here — plus per-field/metric extensions carrying `sql`/`sql_column_is` — means the Ossie output contains a full duplicate of the GSF catalog alongside the derived model. Great for round-trip fidelity, but large and hard to hand-review. Worth calling out explicitly as a tradeoff in the README.



##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:

Review Comment:
   **Inconsistent handling of a malformed native snapshot.** Here a `GSFConversionError` from `_read_catalog` is swallowed and an empty index returned, but `_reconcile_native_relationships` (line 1784) calls `_read_catalog(native)` without catching — so the same bad snapshot silently degrades in one path and hard-fails in the other. Only reachable via a hand-edited extension, but the two paths should agree.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre commented on PR #247:
URL: https://github.com/apache/ossie/pull/247#issuecomment-5127145449

   I added CI for this converter (on the PR directly).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on PR #247:
URL: https://github.com/apache/ossie/pull/247#issuecomment-5106122166

   Also rebased onto current main and picked up #113: datatype now maps to and from the physical type on the GSF column behind a field, with the two directions as inverses so a declared type survives a full cycle. A type outside Ossie's vocabulary becomes Opaque, and a type preserved from a live GSF catalog always wins over the logical one. Computed fields and metrics have no column and GSF stores no type for them, so theirs isn't carried, documented in the README. There's a test that reads the DataType enum from osi-schema.json so a future member can't be added without the mapping noticing.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666672665


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:

Review Comment:
   Agreed, I made both fail rather than both degrade: silently ignoring a bad snapshot means quietly discarding the live GSF identifiers and emitting a document with regenerated ones that won't match the instance it came from, which felt like the worse outcome for a hand-edit to cause. There's a test asserting both entry points raise alike on the same bad snapshot.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666630394


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:
+        raise GSFConversionError(
+            f"Unsupported Ossie version {root.get('version')!r}; "
+            f"supported version is {OSSIE_VERSION!r}"
+        )
+    models = root.get("semantic_model")
+    if not isinstance(models, list) or len(models) != 1:
+        raise GSFConversionError("Ossie input must contain exactly one semantic model")
+    model = models[0]
+    if not isinstance(model, dict) or not model.get("name"):
+        raise GSFConversionError("Ossie semantic model requires a name")
+    return root, model
+
+
+def _load_yaml(value: str, label: str) -> dict[str, Any]:
+    try:
+        root = yaml.safe_load(value)
+    except yaml.YAMLError as exc:
+        raise GSFConversionError(f"Invalid {label} YAML: {exc}") from exc
+    if not isinstance(root, dict):
+        raise GSFConversionError(f"Invalid {label} YAML: expected a root mapping")
+    return root
+
+
+def _validate_relationship(
+    relationship: Any,
+    datasets: Mapping[str, Any],
+) -> dict[str, Any]:
+    if not isinstance(relationship, dict) or not relationship.get("name"):
+        raise GSFConversionError("Every Ossie relationship needs a name")
+    from_name = relationship.get("from")
+    to_name = relationship.get("to")
+    if from_name not in datasets or to_name not in datasets:
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} references an unknown dataset"
+        )
+    from_columns = relationship.get("from_columns") or []
+    to_columns = relationship.get("to_columns") or []
+    if not from_columns or len(from_columns) != len(to_columns):
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} must have equal, "
+            "non-empty column lists"
+        )
+    return relationship
+
+
+def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
+    if not isinstance(value, dict):
+        raise GSFConversionError(f"{name!r} has no valid expression")
+    dialects = value.get("dialects")
+    if not isinstance(dialects, list) or not dialects:
+        raise GSFConversionError(f"{name!r} requires at least one expression dialect")
+    result = [
+        {
+            "dialect": str(item["dialect"]),
+            "expression": str(item["expression"]),
+        }
+        for item in dialects
+        if isinstance(item, dict)
+        and item.get("dialect")
+        and item.get("expression") is not None
+    ]
+    if not result:
+        raise GSFConversionError(f"{name!r} has no usable expression dialect")
+    return result
+
+
+def _pick_expression(expressions: list[dict[str, str]], name: str) -> str:
+    for expression in expressions:
+        if expression["dialect"].upper() == "ANSI_SQL":
+            return expression["expression"]
+    if expressions:
+        return expressions[0]["expression"]
+    raise GSFConversionError(f"{name!r} has no usable expression")
+
+
+def _simple_source_column(
+    expression: str,
+    dataset_name: str,
+    table_name: str,
+) -> str | None:
+    match = _SIMPLE_COLUMN.fullmatch(expression.strip())
+    if not match:
+        return None
+    qualifier = match.group("qualifier")
+    if qualifier and qualifier not in (dataset_name, table_name):
+        return None
+    return match.group("column")
+
+
+def _field_table_refs(
+    field: Mapping[str, Any],
+    expression: str,
+    owner: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(field)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(field.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"SQL attribute {field.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    result = list(dict.fromkeys([owner, *refs]))
+    _validate_single_database_refs(
+        result,
+        datasets,
+        f"SQL attribute {field.get('name')!r}",
+        sql=reference_sql,
+    )
+    return result
+
+
+def _metric_table_refs(
+    metric: Mapping[str, Any],
+    expression: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(metric)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(metric.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    if refs:
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if len(datasets) == 1:
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if extension.get("entity") == "custom_analysis" and extension.get("sql"):
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    raise GSFConversionError(
+        f"Metric {metric.get('name')!r} does not identify a source dataset; "
+        "qualify a referenced column or add NVIDIA_GSF table_refs"
+    )
+
+
+def _validate_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    name: str,
+) -> None:
+    unknown = [ref for ref in refs if ref not in datasets]
+    if unknown:
+        raise GSFConversionError(
+            f"{name!r} has unknown NVIDIA_GSF table_refs: {', '.join(unknown)}"
+        )
+
+
+def _validate_single_database_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    context: str,
+    *,
+    sql: str,
+) -> None:
+    databases = {
+        str(datasets[ref]["source"]["database"]) for ref in refs if ref in datasets
+    }
+    databases.update(
+        table.catalog for table in _parse_sql(sql).find_all(exp.Table) if table.catalog
+    )
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _referenced_datasets(
+    sql: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    references: list[str] = []
+    parsed = _parse_sql(sql)
+    for table in parsed.find_all(exp.Table):
+        matches = [
+            name
+            for name, context in datasets.items()
+            if table.name == str(context["source"]["table"])
+            and (not table.db or table.db == str(context["source"]["schema"]))
+            and (
+                not table.catalog or table.catalog == str(context["source"]["database"])
+            )
+        ]
+        if matches:
+            match = matches[0]
+            if match not in references:
+                references.append(match)
+    for column in parsed.find_all(exp.Column):
+        qualifier = column.table
+        if not qualifier:
+            continue
+        matches = [
+            name
+            for name, context in datasets.items()
+            if qualifier in (name, str(context["source"]["table"]))
+        ]
+        if len(matches) == 1 and matches[0] not in references:
+            references.append(matches[0])
+    return references
+
+
+def _collect_expression_columns(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> None:
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        if not dataset_name:
+            continue
+        context = datasets[dataset_name]
+        source_key = _source_key(context["source"])
+        known_table = source_key in preserved["tables"]
+        if known_table and (*source_key, column.name) not in preserved["columns"]:
+            # A GSF-sourced catalog is authoritative: an identifier that is not
+            # already a column of the table is SQL syntax, not physical data.
+            continue
+        _add_catalog_column(context, column.name)
+
+
+def _sql_column_ids(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    result: list[str] = []
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        column_id = (
+            column_ids.get((dataset_name, column.name)) if dataset_name else None
+        )
+        if column_id and column_id not in result:
+            result.append(column_id)
+    return result
+
+
+def _column_dataset(
+    column: exp.Column,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, Any],
+) -> str | None:
+    qualifier = column.table
+    if qualifier:
+        matches = [
+            name
+            for name, context in datasets.items()
+            if name in refs and qualifier in (name, str(context["source"]["table"]))
+        ]
+        return matches[0] if len(matches) == 1 else None
+    if owner:
+        return owner
+    return refs[0] if len(refs) == 1 else None
+
+
+def _sql_columns(sql: str) -> list[exp.Column]:
+    return [
+        column
+        for column in _parse_sql(sql).find_all(exp.Column)
+        if not _is_date_part_unit(column)
+    ]
+
+
+def _is_date_part_unit(column: exp.Column) -> bool:
+    """Report whether a parsed column is really a date-part keyword.
+
+    ``DATEDIFF(day, a, b)`` and friends put the unit in an argument slot that
+    sqlglot parses as an unqualified column, which would otherwise be mistaken
+    for a physical catalog column.
+    """
+    parent = column.parent
+    if column.table or column.name.lower() not in _DATE_PART_UNITS:
+        return False
+    if not isinstance(parent, exp.Func):
+        return False
+    if isinstance(parent, exp.Anonymous):
+        return str(parent.this).lower() in _DATE_FUNCTIONS
+    try:
+        names = parent.sql_names()
+    except (AttributeError, IndexError):
+        return False
+    return any(name.lower() in _DATE_FUNCTIONS for name in names)
+
+
+def _parse_sql(sql: str) -> exp.Expression:

Review Comment:
   Good catch, i fixed it now, both the dialect list and the opaque-SQL behavior are now documented in the README



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3663033581


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(

Review Comment:
   **Date-part filtering is a load-bearing denylist for Ossie-origin catalogs** (columns are synthesized from expression identifiers). Two failure modes: (a) a real column literally named `day`/`month`/`q`/`us` used as a genuine argument to a recognized date function is dropped; (b) a unit argument to a date function *not* in `_DATE_FUNCTIONS` gets added as a fictional physical column. The single-letter/abbrev entries (`q`, `dd`, `mm`, `us`, `ns`) widen the false-drop surface. Partly inherent to synthesizing a catalog from SQL text, mostly flagging for awareness.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre merged PR #247:
URL: https://github.com/apache/ossie/pull/247


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3663033585


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:

Review Comment:
   **Inconsistent handling of a malformed native snapshot.** Here a `GSFConversionError` from `_read_catalog` is swallowed and an empty index returned, but `_reconcile_native_relationships` (line 1784) calls `_read_catalog(native)` without catching (so the same bad snapshot silently degrades in one path and hard-fails in the other. Only reachable via a hand-edited extension, but the two paths should agree).



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666688969


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,

Review Comment:
   fixed.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on PR #247:
URL: https://github.com/apache/ossie/pull/247#issuecomment-5096759301

   #275 is not a supersession of this PR. It's a copy of this branch's code re-committed under different authorship, based on an earlier state of this PR.
   
   This PR was updated in 791b440 to target GSF's actual model interchange contract (`GsfModelDocument`). #275 predates that change and still emits the earlier provisional shape, which current GSF cannot import. I've verified the code here end to end against a running GSF instance: a real model exports, converts in both directions, and re-imports with every record and identifier intact.
   
   This remains the active contribution and I'll keep it current. Happy to fold in the CI workflow idea from #275.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["jbonofre (via GitHub)" <gi...@apache.org>]

jbonofre commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3663033590


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,

Review Comment:
   **Design note (not a blocker):** embedding the entire native document here (plus per-field/metric extensions carrying `sql`/`sql_column_is`) means the Ossie output contains a full duplicate of the GSF catalog alongside the derived model. Great for round-trip fidelity, but large and hard to hand-review. Worth calling out explicitly as a tradeoff in the README.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["johntomcat7408-cmyk (via GitHub)" <gi...@apache.org>]

johntomcat7408-cmyk commented on PR #247:
URL: https://github.com/apache/ossie/pull/247#issuecomment-5090197790

   Superseded by # (). The replacement preserves the original contribution context, consolidates the change into one commit, and includes the review fixes from this PR.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666658790


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(

Review Comment:
   Reworked rather than trimmed. A keyword now counts as a unit only when it sits in the unit slot itself, so LAST_DAY(day), TRUNC(day) and DATEDIFF(month, day, x) all keep day as a real column, that's the false-drop case gone. I left q/dd/mm in deliberately, with the position check they're harmless, and removing them would turn the common T-SQL DATEDIFF(dd, ...) into a fictional column, trading your first failure mode for your second. The second one is still possible for a date function the converter doesn't recognize, which is inherent to deriving a catalog from SQL text, so I documented it rather than implying it's solved.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

Re: [PR] [OSSIE][NVIDIA_GSF] Add bidirectional GSF semantic model converter [ossie]

["liofnv (via GitHub)" <gi...@apache.org>]

liofnv commented on code in PR #247:
URL: https://github.com/apache/ossie/pull/247#discussion_r3666694456


##########
converters/gsf/src/ossie_gsf/native_converter.py:
##########
@@ -0,0 +1,1937 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+
+from __future__ import annotations
+
+import argparse
+import json
+import re
+import sys
+from collections import defaultdict
+from collections.abc import Iterable, Mapping
+from copy import deepcopy
+from pathlib import Path
+from typing import Any
+from uuid import UUID, uuid5
+
+import yaml
+from sqlglot import exp, parse_one
+from sqlglot.errors import ParseError
+
+OSSIE_VERSION = "0.2.0.dev0"
+NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
+GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
+_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
+_SIMPLE_COLUMN = re.compile(
+    r"^(?:(?P<qualifier>[A-Za-z_][A-Za-z0-9_]*)\.)?"
+    r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
+)
+_DATE_PART_UNITS = frozenset(
+    {
+        "year",
+        "years",
+        "yy",
+        "yyyy",
+        "quarter",
+        "quarters",
+        "qq",
+        "q",
+        "month",
+        "months",
+        "mm",
+        "mon",
+        "week",
+        "weeks",
+        "wk",
+        "ww",
+        "isoweek",
+        "day",
+        "days",
+        "dd",
+        "dayofyear",
+        "doy",
+        "dy",
+        "dayofweek",
+        "dow",
+        "weekday",
+        "hour",
+        "hours",
+        "hh",
+        "minute",
+        "minutes",
+        "mi",
+        "second",
+        "seconds",
+        "ss",
+        "millisecond",
+        "milliseconds",
+        "ms",
+        "microsecond",
+        "microseconds",
+        "us",
+        "nanosecond",
+        "nanoseconds",
+        "ns",
+        "epoch",
+    }
+)
+_DATE_FUNCTIONS = frozenset(
+    {
+        "datediff",
+        "date_diff",
+        "datetime_diff",
+        "timestampdiff",
+        "timestamp_diff",
+        "timediff",
+        "time_diff",
+        "dateadd",
+        "date_add",
+        "datetime_add",
+        "timestampadd",
+        "timestamp_add",
+        "datesub",
+        "date_sub",
+        "timestampsub",
+        "timestamp_sub",
+        "date_trunc",
+        "datetrunc",
+        "datetime_trunc",
+        "timestamp_trunc",
+        "time_trunc",
+        "date_part",
+        "datepart",
+        "datename",
+        "extract",
+        "last_day",
+        "trunc",
+    }
+)
+
+
+class GSFConversionError(Exception):
+    """Raised when a document cannot be converted safely."""
+
+
+def convert_ossie_to_gsf(
+    ossie_yaml: str,
+    *,
+    database_name: str | None = None,
+) -> str:
+    """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+    _, model = _parse_ossie(ossie_yaml)
+    source_datasets = model.get("datasets") or []
+    if not isinstance(source_datasets, list) or not source_datasets:
+        raise GSFConversionError(
+            "The Ossie semantic model must contain at least one dataset"
+        )
+
+    native = _native_snapshot(model)
+    preserved = _index_native_document(native)
+    datasets: dict[str, dict[str, Any]] = {}
+    for item in source_datasets:
+        if not isinstance(item, dict) or not item.get("name"):
+            raise GSFConversionError(
+                "Every Ossie dataset must be a mapping with a name"
+            )
+        name = str(item["name"])
+        if name in datasets:
+            raise GSFConversionError(f"Duplicate dataset name {name!r}")
+        source = _parse_source(item.get("source"), database_name)
+        if not source["database"] or not source["schema"]:
+            raise GSFConversionError(
+                f"Dataset {name!r} source must resolve to database.schema.table"
+            )
+        datasets[name] = {
+            "source": source,
+            "original": item,
+            "columns": [],
+            "column_set": set(),
+            "simple_fields": [],
+            "computed_fields": [],
+        }
+        preserved_table = preserved["tables"].get(_source_key(source), {})
+        for column in preserved_table.get("columns") or []:
+            if isinstance(column, dict) and column.get("name"):
+                _add_catalog_column(datasets[name], str(column["name"]))
+
+    relationships = [
+        _validate_relationship(item, datasets)
+        for item in model.get("relationships") or []
+    ]
+    for relationship in relationships:
+        for side, key in (("from", "from_columns"), ("to", "to_columns")):
+            for column in relationship[key]:
+                _add_catalog_column(datasets[str(relationship[side])], str(column))
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        for column in dataset.get("primary_key") or []:
+            _add_catalog_column(context, str(column))
+        for key in dataset.get("unique_keys") or []:
+            for column in key:
+                _add_catalog_column(context, str(column))
+
+        field_names: set[str] = set()
+        for field in dataset.get("fields") or []:
+            if not isinstance(field, dict) or not field.get("name"):
+                raise GSFConversionError(
+                    f"Every field in dataset {name!r} needs a name"
+                )
+            field_name = str(field["name"])
+            if field_name in field_names:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in dataset {name!r}"
+                )
+            field_names.add(field_name)
+            expressions = _normalize_expressions(field.get("expression"), field_name)
+            selected = _pick_expression(expressions, field_name)
+            source_column = _simple_source_column(
+                selected, name, str(context["source"]["table"])
+            )
+            if source_column:
+                _add_catalog_column(context, source_column)
+                context["simple_fields"].append((field, source_column))
+            else:
+                refs = _field_table_refs(field, selected, name, datasets)
+                context["computed_fields"].append((field, expressions, selected, refs))
+                _collect_expression_columns(selected, name, refs, datasets, preserved)
+
+    metrics: list[tuple[dict[str, Any], list[dict[str, str]], str, list[str]]] = []
+    metric_names: set[str] = set()
+    for metric in model.get("metrics") or []:
+        if not isinstance(metric, dict) or not metric.get("name"):
+            raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+        name = str(metric["name"])
+        if name in metric_names:
+            raise GSFConversionError(f"Duplicate metric name {name!r}")
+        metric_names.add(name)
+        expressions = _normalize_expressions(metric.get("expression"), name)
+        selected = _pick_expression(expressions, name)
+        refs = _metric_table_refs(metric, selected, datasets)
+        _collect_expression_columns(selected, None, refs, datasets, preserved)
+        metrics.append((metric, expressions, selected, refs))
+
+    databases, table_ids, column_ids = _build_catalog(datasets, preserved)
+    if native:
+        databases = _merge_catalog_databases(
+            native.get("data_layer", {}).get("databases"), databases
+        )
+        for column_id in _catalog_column_ids(databases):
+            column_ids.setdefault(("__native__", column_id), column_id)
+    terms: list[dict[str, Any]] = []
+    term_ids: dict[str, str] = {}
+    attribute_ids: dict[tuple[str, str], str] = {}
+
+    for name, context in datasets.items():
+        dataset = context["original"]
+        source_key = _source_key(context["source"])
+        preserved_term = preserved["terms"].get((name, source_key), {})
+        term_id = str(preserved_term.get("id") or _stable_id("term", name, *source_key))
+        term_ids[name] = term_id
+        term: dict[str, Any] = {
+            "id": term_id,
+            "name": name,
+            "description": str(dataset.get("description") or ""),
+            "represents": [table_ids[name]],
+            "columns_attributes": [],
+        }
+        for field, source_column in context["simple_fields"]:
+            field_name = str(field["name"])
+            preserved_attr = preserved["column_attributes"].get(
+                (name, field_name, source_key, source_column), {}
+            )
+            attr_id = str(
+                preserved_attr.get("id")
+                or _stable_id(
+                    "column-attribute", name, field_name, *source_key, source_column
+                )
+            )
+            attribute_ids[(name, source_column)] = attr_id
+            term["columns_attributes"].append(
+                {
+                    "id": attr_id,
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "column_id": column_ids[(name, source_column)],
+                }
+            )
+        terms.append(term)
+
+    joins: list[dict[str, Any]] = []
+    foreign_keys: list[dict[str, str]] = []
+    semantic_fks: list[dict[str, str]] = []
+    for relationship in relationships:
+        from_name = str(relationship["from"])
+        to_name = str(relationship["to"])
+        joins.append(
+            {
+                "source_table_id": table_ids[from_name],
+                "target_table_id": table_ids[to_name],
+                "join_columns": [
+                    {"source": str(source), "target": str(target)}
+                    for source, target in zip(
+                        relationship["from_columns"],
+                        relationship["to_columns"],
+                        strict=True,
+                    )
+                ],
+            }
+        )
+        for source, target in zip(
+            relationship["from_columns"],
+            relationship["to_columns"],
+            strict=True,
+        ):
+            source_name = str(source)
+            target_name = str(target)
+            foreign_keys.append(
+                {
+                    "source_column_id": column_ids[(from_name, source_name)],
+                    "target_column_id": column_ids[(to_name, target_name)],
+                }
+            )
+            target_attribute_id = attribute_ids.get((to_name, target_name))
+            if target_attribute_id:
+                semantic_fks.append(
+                    {
+                        "column_attribute_id": target_attribute_id,
+                        "column_id": column_ids[(from_name, source_name)],
+                    }
+                )
+
+    sql_groups: dict[str, list[dict[str, Any]]] = {key: [] for key in _SQL_GROUPS}
+    for name, context in datasets.items():
+        for field, _, selected, refs in context["computed_fields"]:
+            field_name = str(field["name"])
+            extension = _gsf_extension_data(field)
+            preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
+            expression_unchanged = _expression_matches(
+                selected, extension.get("ossie_expression")
+            )
+            source_group = str(
+                extension.get("sql_source")
+                or preserved_attr.get("source_group")
+                or "manual"
+            )
+            if source_group not in sql_groups:
+                source_group = "manual"
+            full_sql = str(
+                extension.get("sql")
+                if expression_unchanged and extension.get("sql")
+                else _wrap_expression(
+                    selected,
+                    field_name,
+                    refs,
+                    datasets,
+                    relationships,
+                )
+            )
+            resolved_column_ids = _sql_column_ids(
+                selected, name, refs, datasets, column_ids
+            )
+            preserved_column_ids = extension.get(
+                "sql_column_is", preserved_attr.get("sql_column_is")
+            )
+            if expression_unchanged and _all_resolvable_ids(
+                preserved_column_ids, column_ids
+            ):
+                resolved_column_ids = list(preserved_column_ids)
+            sql_groups[source_group].append(
+                {
+                    "id": str(
+                        extension.get("id")
+                        or preserved_attr.get("id")
+                        or _stable_id("sql-attribute", name, field_name)
+                    ),
+                    "name": field_name,
+                    "description": str(field.get("description") or ""),
+                    "sql": full_sql,
+                    "sql_column_is": resolved_column_ids,
+                    "term_id": term_ids[name],
+                }
+            )
+
+    custom_analyses: list[dict[str, Any]] = []
+    for metric, _, selected, refs in metrics:
+        name = str(metric["name"])
+        extension = _gsf_extension_data(metric)
+        preserved_analysis = preserved["custom_analyses"].get(name, {})
+        expression_unchanged = _expression_matches(
+            selected, extension.get("ossie_expression")
+        )
+        full_sql = str(
+            extension.get("sql")
+            if expression_unchanged and extension.get("sql")
+            else _wrap_expression(selected, name, refs, datasets, relationships)
+        )
+        referenced_column_ids = _sql_column_ids(
+            selected, None, refs, datasets, column_ids
+        )
+        preserved_column_ids = extension.get(
+            "sql_column_is", preserved_analysis.get("sql_column_is")
+        )
+        if expression_unchanged and _all_resolvable_ids(
+            preserved_column_ids, column_ids
+        ):
+            referenced_column_ids = list(preserved_column_ids)
+        if not referenced_column_ids:
+            referenced_column_ids = _first_resolvable_column_ids(
+                refs, datasets, column_ids
+            )
+        if not referenced_column_ids:
+            raise GSFConversionError(
+                f"Metric {name!r} has no resolvable catalog column for "
+                "custom-analysis SQL validation"
+            )
+        custom_analyses.append(
+            {
+                "id": str(
+                    extension.get("id")
+                    or preserved_analysis.get("id")
+                    or _stable_id("custom-analysis", str(model["name"]), name)
+                ),
+                "name": name,
+                "description": str(metric.get("description") or ""),
+                "sql": full_sql,
+                "sql_column_is": referenced_column_ids,
+            }
+        )
+
+    if native:
+        foreign_keys, joins, semantic_fks = _reconcile_native_relationships(
+            native,
+            represented_table_ids=set(table_ids.values()),
+            foreign_keys=foreign_keys,
+            joins=joins,
+            semantic_fks=semantic_fks,
+        )
+
+    output = {
+        "data_layer": {
+            "databases": databases,
+            "foreign_keys": foreign_keys,
+            "joins": joins,
+        },
+        "semantic_layer": {
+            "terms": terms,
+            "semantic_fks": semantic_fks,
+            "sql_attributes": sql_groups,
+            "custom_analyses": custom_analyses,
+        },
+        "zones": deepcopy(native.get("zones") or []) if native else [],
+    }
+    return _dump_yaml(output)
+
+
+def convert_gsf_to_ossie(
+    gsf_yaml: str,
+    *,
+    model_name: str | None = None,
+) -> str:
+    """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
+    root = _parse_gsf(gsf_yaml)
+    catalog = _read_catalog(root)
+    semantic = root["semantic_layer"]
+    terms = semantic["terms"]
+    if not terms:
+        raise GSFConversionError(
+            "GSF document has no representable terms; Ossie requires at least "
+            "one dataset"
+        )
+
+    term_by_id: dict[str, dict[str, Any]] = {}
+    term_id_by_name: dict[str, str] = {}
+    datasets_by_table: dict[str, list[str]] = defaultdict(list)
+    datasets: list[dict[str, Any]] = []
+    fields_by_term: dict[str, list[dict[str, Any]]] = defaultdict(list)
+    field_names_by_term: dict[str, set[str]] = defaultdict(set)
+    term_columns: dict[str, set[str]] = defaultdict(set)
+    attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
+
+    for term in terms:
+        term_id = _required_id(term, "GSF term")
+        if term_id in term_by_id:
+            raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+        represents = term.get("represents") or []
+        if len(represents) != 1:
+            raise GSFConversionError(
+                f"Converter supports only GSF terms that represent exactly one "
+                f"table; term {term.get('name')!r} represents {len(represents)}"
+            )
+        table_id = str(represents[0])
+        table = catalog["tables"].get(table_id)
+        if table is None:
+            raise GSFConversionError(
+                f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+            )
+        name = str(term.get("name") or "")
+        if not name:
+            raise GSFConversionError("Every GSF term needs a non-empty name")
+        if name in term_id_by_name:
+            raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+        term_id_by_name[name] = term_id
+        datasets_by_table[table_id].append(name)
+        term_by_id[term_id] = term
+        dataset: dict[str, Any] = {
+            "name": name,
+            "source": ".".join(table["source"]),
+        }
+        if term.get("description"):
+            dataset["description"] = str(term["description"])
+        if table["item"].get("pk"):
+            dataset["primary_key"] = list(table["item"]["pk"])
+        unique_columns = [
+            str(column.get("name"))
+            for column in table["item"].get("columns") or []
+            if column.get("is_unique") and column.get("name")
+        ]
+        if unique_columns:
+            dataset["unique_keys"] = [[column] for column in unique_columns]
+
+        for attribute in term.get("columns_attributes") or []:
+            if not isinstance(attribute, dict) or not attribute.get("id"):
+                raise GSFConversionError(
+                    f"Term {name!r} contains a column attribute without an id"
+                )
+            column_id = str(attribute.get("column_id") or "")
+            column = catalog["columns"].get(column_id)
+            if column is None:
+                raise GSFConversionError(
+                    f"Column attribute {attribute.get('name')!r} references "
+                    f"unknown column {column_id!r}"
+                )
+            field_name = str(attribute.get("name") or column["name"])
+            if field_name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {field_name!r} in GSF term {name!r}"
+                )
+            field_names_by_term[term_id].add(field_name)
+            term_columns[name].add(str(column["name"]))
+            field: dict[str, Any] = {
+                "name": field_name,
+                "expression": _ossie_expression(str(column["name"])),
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+            attr_owner[str(attribute["id"])] = (term_id, attribute)
+
+        datasets.append(dataset)
+
+    for source_group in _SQL_GROUPS:
+        for attribute in semantic["sql_attributes"][source_group]:
+            attribute_id = _required_id(attribute, "GSF SQL attribute")
+            term_id = str(attribute.get("term_id") or "")
+            if term_id not in term_by_id:
+                raise GSFConversionError(
+                    f"SQL attribute {attribute.get('name')!r} references "
+                    f"unknown term {term_id!r}"
+                )
+            sql = str(attribute.get("sql") or "")
+            name = str(attribute.get("name") or "")
+            if not name or not sql:
+                raise GSFConversionError(
+                    "Every GSF SQL attribute requires non-empty name and sql"
+                )
+            term_name = str(term_by_id[term_id].get("name") or "")
+            if name in field_names_by_term[term_id]:
+                raise GSFConversionError(
+                    f"Duplicate field name {name!r} in GSF term {term_name!r}"
+                )
+            field_names_by_term[term_id].add(name)
+            represented_table_id = str(term_by_id[term_id]["represents"][0])
+            _validate_gsf_sql_databases(
+                f"SQL attribute {name!r}",
+                sql,
+                attribute.get("sql_column_is") or [],
+                catalog,
+                attached_table_id=represented_table_id,
+            )
+            ossie_expression = _expression_from_sql(sql)
+            field = {
+                "name": name,
+                "expression": _ossie_expression(ossie_expression),
+                "custom_extensions": [
+                    _gsf_extension(
+                        {
+                            "entity": "sql_attribute",
+                            "id": attribute_id,
+                            "sql": sql,
+                            "sql_source": source_group,
+                            "sql_column_is": list(attribute.get("sql_column_is") or []),
+                            "term_id": term_id,
+                            "ossie_expression": ossie_expression,
+                        }
+                    )
+                ],
+            }
+            if attribute.get("description"):
+                field["description"] = str(attribute["description"])
+            fields_by_term[term_id].append(field)
+
+    for dataset in datasets:
+        term_id = term_id_by_name[dataset["name"]]
+        if fields_by_term[term_id]:
+            dataset["fields"] = fields_by_term[term_id]
+
+    metrics: list[dict[str, Any]] = []
+    for analysis in semantic.get("custom_analyses") or []:
+        analysis_id = _required_id(analysis, "GSF custom analysis")
+        sql = str(analysis.get("sql") or "")
+        name = str(analysis.get("name") or "")
+        if not name or not sql:
+            raise GSFConversionError(
+                "Every GSF custom analysis requires non-empty name and sql"
+            )
+        _validate_gsf_sql_databases(
+            f"Custom analysis {name!r}",
+            sql,
+            analysis.get("sql_column_is") or [],
+            catalog,
+        )
+        ossie_expression = _expression_from_sql(sql)
+        metric: dict[str, Any] = {
+            "name": name,
+            "expression": _ossie_expression(ossie_expression),
+            "custom_extensions": [
+                _gsf_extension(
+                    {
+                        "entity": "custom_analysis",
+                        "id": analysis_id,
+                        "sql": sql,
+                        "sql_column_is": list(analysis.get("sql_column_is") or []),
+                        "ossie_expression": ossie_expression,
+                    }
+                )
+            ],
+        }
+        if analysis.get("description"):
+            metric["description"] = str(analysis["description"])
+        metrics.append(metric)
+
+    relationships = _relationships_from_gsf(
+        root,
+        catalog,
+        datasets_by_table,
+        term_columns,
+        attr_owner,
+        term_by_id,
+    )
+    database_names = {
+        source[0]
+        for source in (table["source"] for table in catalog["tables"].values())
+    }
+    inferred_name = (
+        model_name
+        or (next(iter(database_names)) if len(database_names) == 1 else None)
+        or "gsf_model"
+    )
+    semantic_model: dict[str, Any] = {
+        "name": inferred_name,
+        "datasets": datasets,
+        "custom_extensions": [
+            _gsf_extension(
+                {
+                    "model_name": inferred_name,
+                    "native_document": root,
+                }
+            )
+        ],
+    }
+    if relationships:
+        semantic_model["relationships"] = relationships
+    if metrics:
+        semantic_model["metrics"] = metrics
+    return _dump_yaml({"version": OSSIE_VERSION, "semantic_model": [semantic_model]})
+
+
+def _build_catalog(
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> tuple[list[dict[str, Any]], dict[str, str], dict[tuple[str, str], str]]:
+    db_tree: dict[str, dict[str, dict[str, Any]]] = {}
+    table_ids: dict[str, str] = {}
+    column_ids: dict[tuple[str, str], str] = {}
+    source_groups: dict[tuple[str, str, str], list[tuple[str, dict[str, Any]]]] = (
+        defaultdict(list)
+    )
+    for dataset_name, context in datasets.items():
+        source_groups[_source_key(context["source"])].append((dataset_name, context))
+
+    for source_key, contexts in source_groups.items():
+        database, schema, table = source_key
+        preserved_table = preserved["tables"].get(source_key, {})
+        preserved_schema = preserved["schemas"].get((database, schema), {})
+        preserved_db = preserved["databases"].get(database, {})
+        db_entry = db_tree.setdefault(
+            database,
+            {
+                "id": str(preserved_db.get("id") or _stable_id("database", database)),
+                "dialect": str(preserved_db.get("dialect") or ""),
+                "schemas": {},
+            },
+        )
+        schema_entry = db_entry["schemas"].setdefault(
+            schema,
+            {
+                "id": str(
+                    preserved_schema.get("id") or _stable_id("schema", database, schema)
+                ),
+                "name": schema,
+                "database_name": database,
+                "tables": [],
+            },
+        )
+        table_id = str(
+            preserved_table.get("id") or _stable_id("table", database, schema, table)
+        )
+        for dataset_name, _ in contexts:
+            table_ids[dataset_name] = table_id
+        pk = list(
+            dict.fromkeys(
+                str(value)
+                for _, context in contexts
+                for value in context["original"].get("primary_key") or []
+            )
+        )
+        unique_keys = [
+            [str(value) for value in key]
+            for _, context in contexts
+            for key in context["original"].get("unique_keys") or []
+        ]
+        catalog_columns = list(
+            dict.fromkeys(
+                column for _, context in contexts for column in context["columns"]
+            )
+        )
+        columns: list[dict[str, Any]] = []
+        for column_name in catalog_columns:
+            preserved_column = preserved["columns"].get((*source_key, column_name), {})
+            column_id = str(
+                preserved_column.get("id")
+                or _stable_id("column", database, schema, table, column_name)
+            )
+            for dataset_name, _ in contexts:
+                column_ids[(dataset_name, column_name)] = column_id
+            single_unique = [column_name] in unique_keys or (
+                len(pk) == 1 and pk[0] == column_name
+            )
+            columns.append(
+                {
+                    "id": column_id,
+                    "name": column_name,
+                    "description": str(preserved_column.get("description") or ""),
+                    "type": str(preserved_column.get("type") or ""),
+                    "sample_values": list(preserved_column.get("sample_values") or []),
+                    "is_nullable": bool(
+                        preserved_column.get("is_nullable", column_name not in pk)
+                    ),
+                    "is_unique": bool(preserved_column.get("is_unique", single_unique)),
+                }
+            )
+        schema_entry["tables"].append(
+            {
+                "id": table_id,
+                "name": table,
+                "description": str(
+                    preserved_table.get("description")
+                    if preserved_table
+                    else next(
+                        (
+                            context["original"].get("description")
+                            for _, context in contexts
+                            if context["original"].get("description")
+                        ),
+                        "",
+                    )
+                ),
+                "pk": pk,
+                "type": str(preserved_table.get("type") or ""),
+                "columns": columns,
+            }
+        )
+
+    result: list[dict[str, Any]] = []
+    for database in sorted(db_tree):
+        db_entry = db_tree[database]
+        schemas = [db_entry["schemas"][name] for name in sorted(db_entry["schemas"])]
+        for schema in schemas:
+            schema["tables"].sort(key=lambda item: (item["name"], item["id"]))
+        result.append(
+            {
+                "id": db_entry["id"],
+                "dialect": db_entry["dialect"],
+                "schemas": schemas,
+            }
+        )
+    return result, table_ids, column_ids
+
+
+def _merge_catalog_databases(
+    preserved: Any,
+    generated: list[dict[str, Any]],
+) -> list[dict[str, Any]]:
+    """Keep catalog objects that Ossie cannot represent directly."""
+    result = deepcopy(generated)
+    databases_by_id = {str(item["id"]): item for item in result}
+    for preserved_database in preserved or []:
+        if not isinstance(preserved_database, dict):
+            continue
+        database_id = str(preserved_database.get("id") or "")
+        database = databases_by_id.get(database_id)
+        if database is None:
+            copied = deepcopy(preserved_database)
+            result.append(copied)
+            databases_by_id[database_id] = copied
+            continue
+        schemas_by_id = {
+            str(item["id"]): item for item in database.get("schemas") or []
+        }
+        for preserved_schema in preserved_database.get("schemas") or []:
+            schema_id = str(preserved_schema.get("id") or "")
+            schema = schemas_by_id.get(schema_id)
+            if schema is None:
+                database["schemas"].append(deepcopy(preserved_schema))
+                continue
+            tables_by_id = {
+                str(item["id"]): item for item in schema.get("tables") or []
+            }
+            for preserved_table in preserved_schema.get("tables") or []:
+                table_id = str(preserved_table.get("id") or "")
+                table = tables_by_id.get(table_id)
+                if table is None:
+                    schema["tables"].append(deepcopy(preserved_table))
+                    continue
+                known_column_ids = {
+                    str(item["id"]) for item in table.get("columns") or []
+                }
+                table["columns"].extend(
+                    deepcopy(column)
+                    for column in preserved_table.get("columns") or []
+                    if str(column.get("id") or "") not in known_column_ids
+                )
+    return result
+
+
+def _catalog_column_ids(databases: Iterable[Mapping[str, Any]]) -> set[str]:
+    return {
+        str(column["id"])
+        for database in databases
+        for schema in database.get("schemas") or []
+        for table in schema.get("tables") or []
+        for column in table.get("columns") or []
+        if column.get("id")
+    }
+
+
+def _index_native_document(root: dict[str, Any] | None) -> dict[str, Any]:
+    result: dict[str, Any] = {
+        "databases": {},
+        "schemas": {},
+        "tables": {},
+        "columns": {},
+        "terms": {},
+        "column_attributes": {},
+        "sql_attributes": {},
+        "custom_analyses": {},
+    }
+    if not root:
+        return result
+    try:
+        catalog = _read_catalog(root)
+    except GSFConversionError:
+        return result
+    for database in root["data_layer"].get("databases") or []:
+        database_names = {
+            str(schema.get("database_name") or "")
+            for schema in database.get("schemas") or []
+            if schema.get("database_name")
+        }
+        for name in database_names:
+            result["databases"][name] = database
+        for schema in database.get("schemas") or []:
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            result["schemas"][(database_name, schema_name)] = schema
+            for table in schema.get("tables") or []:
+                source = (database_name, schema_name, str(table.get("name") or ""))
+                result["tables"][source] = table
+                for column in table.get("columns") or []:
+                    result["columns"][(*source, str(column.get("name") or ""))] = column
+    table_source = {
+        table_id: tuple(table["source"])
+        for table_id, table in catalog["tables"].items()
+    }
+    for term in root["semantic_layer"].get("terms") or []:
+        represents = term.get("represents") or []
+        if len(represents) != 1 or str(represents[0]) not in table_source:
+            continue
+        source = table_source[str(represents[0])]
+        term_name = str(term.get("name") or "")
+        result["terms"][(term_name, source)] = term
+        for attribute in term.get("columns_attributes") or []:
+            column = catalog["columns"].get(str(attribute.get("column_id") or ""))
+            if column:
+                result["column_attributes"][
+                    (
+                        term_name,
+                        str(attribute.get("name") or ""),
+                        source,
+                        column["name"],
+                    )
+                ] = attribute
+    term_names = {
+        str(term.get("id")): str(term.get("name") or "")
+        for term in root["semantic_layer"].get("terms") or []
+    }
+    for group in _SQL_GROUPS:
+        for attribute in (root["semantic_layer"].get("sql_attributes") or {}).get(
+            group, []
+        ):
+            item = dict(attribute)
+            item["source_group"] = group
+            result["sql_attributes"][
+                (
+                    term_names.get(str(attribute.get("term_id") or ""), ""),
+                    str(attribute.get("name") or ""),
+                )
+            ] = item
+    for analysis in root["semantic_layer"].get("custom_analyses") or []:
+        result["custom_analyses"][str(analysis.get("name") or "")] = analysis
+    return result
+
+
+def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
+    tables: dict[str, dict[str, Any]] = {}
+    columns: dict[str, dict[str, Any]] = {}
+    database_ids: set[str] = set()
+    schema_ids: set[str] = set()
+    for database in root["data_layer"]["databases"]:
+        database_id = _required_id(database, "GSF database")
+        if database_id in database_ids:
+            raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+        database_ids.add(database_id)
+        for schema in database.get("schemas") or []:
+            schema_id = _required_id(schema, "GSF schema")
+            if schema_id in schema_ids:
+                raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+            schema_ids.add(schema_id)
+            database_name = str(schema.get("database_name") or "")
+            schema_name = str(schema.get("name") or "")
+            if not database_name:
+                raise GSFConversionError(
+                    f"GSF schema {schema_name!r} requires database_name"
+                )
+            for table in schema.get("tables") or []:
+                table_id = str(table.get("id") or "")
+                if not table_id or table_id in tables:
+                    raise GSFConversionError(
+                        f"Every GSF table needs a globally unique id; got {table_id!r}"
+                    )
+                table_name = str(table.get("name") or "")
+                tables[table_id] = {
+                    "item": table,
+                    "source": (database_name, schema_name, table_name),
+                }
+                for column in table.get("columns") or []:
+                    column_id = str(column.get("id") or "")
+                    if not column_id or column_id in columns:
+                        raise GSFConversionError(
+                            "Every GSF column needs a globally unique id; "
+                            f"got {column_id!r}"
+                        )
+                    columns[column_id] = {
+                        "item": column,
+                        "name": str(column.get("name") or ""),
+                        "table_id": table_id,
+                    }
+    return {"tables": tables, "columns": columns}
+
+
+def _validate_gsf_sql_databases(
+    context: str,
+    sql: str,
+    sql_column_ids: Iterable[Any],
+    catalog: Mapping[str, Any],
+    *,
+    attached_table_id: str | None = None,
+) -> None:
+    databases: set[str] = set()
+    if attached_table_id:
+        table = catalog["tables"].get(attached_table_id)
+        if table:
+            databases.add(str(table["source"][0]))
+    for column_id in sql_column_ids:
+        column = catalog["columns"].get(str(column_id))
+        if column:
+            databases.add(str(catalog["tables"][column["table_id"]]["source"][0]))
+
+    parsed = _parse_sql(sql)
+    for sql_table in parsed.find_all(exp.Table):
+        if sql_table.catalog:
+            databases.add(sql_table.catalog)
+        matches = [
+            table
+            for table in catalog["tables"].values()
+            if sql_table.name == table["source"][2]
+            and (not sql_table.db or sql_table.db == table["source"][1])
+            and (not sql_table.catalog or sql_table.catalog == table["source"][0])
+            and (sql_table.catalog or not databases or table["source"][0] in databases)
+        ]
+        databases.update(str(table["source"][0]) for table in matches)
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _relationships_from_gsf(
+    root: dict[str, Any],
+    catalog: Mapping[str, Any],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+    attr_owner: Mapping[str, tuple[str, dict[str, Any]]],
+    term_by_id: Mapping[str, dict[str, Any]],
+) -> list[dict[str, Any]]:
+    pairs: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    covered_fk_pairs: set[tuple[str, str]] = set()
+    for join in root["data_layer"].get("joins") or []:
+        source_table_id = str(join.get("source_table_id") or "")
+        target_table_id = str(join.get("target_table_id") or "")
+        if (
+            source_table_id not in datasets_by_table
+            or target_table_id not in datasets_by_table
+        ):
+            continue
+        source_columns: list[tuple[str, str]] = []
+        for item in join.get("join_columns") or []:
+            if not isinstance(item, dict):
+                continue
+            source_name = _join_column_name(
+                item.get("source"), source_table_id, catalog
+            )
+            target_name = _join_column_name(
+                item.get("target"), target_table_id, catalog
+            )
+            if source_name and target_name:
+                source_columns.append((source_name, target_name))
+        if not source_columns:
+            source_columns = _fk_columns_for_tables(
+                root, source_table_id, target_table_id, catalog
+            )
+        if source_columns:
+            key = (
+                _relationship_dataset(
+                    source_table_id,
+                    [source for source, _ in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+                _relationship_dataset(
+                    target_table_id,
+                    [target for _, target in source_columns],
+                    datasets_by_table,
+                    term_columns,
+                ),
+            )
+            pairs[key].extend(source_columns)
+            covered_fk_pairs.add((source_table_id, target_table_id))
+
+    fk_groups: dict[tuple[str, str], list[tuple[str, str]]] = defaultdict(list)
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if not source or not target:
+            continue
+        table_pair = (source["table_id"], target["table_id"])
+        if table_pair in covered_fk_pairs:
+            continue
+        if table_pair[0] in datasets_by_table and table_pair[1] in datasets_by_table:
+            fk_groups[table_pair].append((source["name"], target["name"]))
+    for table_pair, columns in fk_groups.items():
+        source_name = _relationship_dataset(
+            table_pair[0],
+            [source for source, _ in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        target_name = _relationship_dataset(
+            table_pair[1],
+            [target for _, target in columns],
+            datasets_by_table,
+            term_columns,
+        )
+        pairs[(source_name, target_name)].extend(columns)
+
+    for semantic_fk in root["semantic_layer"].get("semantic_fks") or []:
+        source = catalog["columns"].get(str(semantic_fk.get("column_id") or ""))
+        owner = attr_owner.get(str(semantic_fk.get("column_attribute_id") or ""))
+        if not source or not owner:
+            continue
+        target_term_id, target_attr = owner
+        target_column = catalog["columns"].get(str(target_attr.get("column_id") or ""))
+        target_term = term_by_id.get(target_term_id)
+        if not target_column or not target_term:
+            continue
+        if source["table_id"] not in datasets_by_table:
+            continue
+        from_name = _relationship_dataset(
+            source["table_id"],
+            [source["name"]],
+            datasets_by_table,
+            term_columns,
+        )
+        to_name = str(target_term.get("name") or "")
+        if not to_name:
+            continue
+        pair = (source["name"], target_column["name"])
+        if pair not in pairs[(from_name, to_name)]:
+            pairs[(from_name, to_name)].append(pair)
+
+    relationships: list[dict[str, Any]] = []
+    used_names: dict[str, int] = defaultdict(int)
+    for (from_name, to_name), columns in pairs.items():
+        unique_columns = list(dict.fromkeys(columns))
+        base_name = f"{from_name}_to_{to_name}"
+        used_names[base_name] += 1
+        suffix = "" if used_names[base_name] == 1 else f"_{used_names[base_name]}"
+        relationships.append(
+            {
+                "name": base_name + suffix,
+                "from": from_name,
+                "to": to_name,
+                "from_columns": [source for source, _ in unique_columns],
+                "to_columns": [target for _, target in unique_columns],
+            }
+        )
+    return relationships
+
+
+def _relationship_dataset(
+    table_id: str,
+    columns: list[str],
+    datasets_by_table: Mapping[str, list[str]],
+    term_columns: Mapping[str, set[str]],
+) -> str:
+    candidates = datasets_by_table.get(table_id) or []
+    if len(candidates) == 1:
+        return candidates[0]
+    matching = [
+        name for name in candidates if set(columns) <= term_columns.get(name, set())
+    ]
+    if len(matching) == 1:
+        return matching[0]
+    raise GSFConversionError(
+        f"Cannot map relationship on table {table_id!r} and columns "
+        f"{', '.join(columns)} to exactly one represented term; candidates: "
+        f"{', '.join(candidates) or 'none'}"
+    )
+
+
+def _fk_columns_for_tables(
+    root: Mapping[str, Any],
+    source_table_id: str,
+    target_table_id: str,
+    catalog: Mapping[str, Any],
+) -> list[tuple[str, str]]:
+    result: list[tuple[str, str]] = []
+    for foreign_key in root["data_layer"].get("foreign_keys") or []:
+        source = catalog["columns"].get(str(foreign_key.get("source_column_id") or ""))
+        target = catalog["columns"].get(str(foreign_key.get("target_column_id") or ""))
+        if (
+            source
+            and target
+            and source["table_id"] == source_table_id
+            and target["table_id"] == target_table_id
+        ):
+            result.append((source["name"], target["name"]))
+    return result
+
+
+def _join_column_name(
+    value: Any,
+    table_id: str,
+    catalog: Mapping[str, Any],
+) -> str | None:
+    text = str(value or "")
+    column = catalog["columns"].get(text)
+    if column and column["table_id"] == table_id:
+        return str(column["name"])
+    for item in catalog["columns"].values():
+        if item["table_id"] == table_id and item["name"] == text:
+            return text
+    return None
+
+
+def _required_id(item: Any, context: str) -> str:
+    if not isinstance(item, dict) or not item.get("id"):
+        raise GSFConversionError(f"{context} requires a non-empty id")
+    return str(item["id"])
+
+
+def _parse_gsf(value: str) -> dict[str, Any]:
+    root = _load_yaml(value, "GSF")
+    expected = {"data_layer", "semantic_layer", "zones"}
+    unknown = sorted(set(root) - expected)
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported GSF root properties: " + ", ".join(unknown)
+        )
+    root.setdefault("data_layer", {})
+    root.setdefault("semantic_layer", {})
+    root.setdefault("zones", [])
+    for key in ("data_layer", "semantic_layer"):
+        if not isinstance(root[key], dict):
+            raise GSFConversionError(f"GSF {key!r} must be a mapping")
+    if not isinstance(root["zones"], list):
+        raise GSFConversionError("GSF 'zones' must be a list")
+    data_layer = root["data_layer"]
+    semantic_layer = root["semantic_layer"]
+    for key in ("databases", "foreign_keys", "joins"):
+        data_layer.setdefault(key, [])
+        if not isinstance(data_layer.get(key), list):
+            raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+    for key in ("terms", "semantic_fks", "custom_analyses"):
+        semantic_layer.setdefault(key, [])
+        if not isinstance(semantic_layer.get(key), list):
+            raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+    semantic_layer.setdefault("sql_attributes", {})
+    sql_attributes = semantic_layer["sql_attributes"]
+    if not isinstance(sql_attributes, dict):
+        raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+    for key in _SQL_GROUPS:
+        sql_attributes.setdefault(key, [])
+        if not isinstance(sql_attributes.get(key), list):
+            raise GSFConversionError(
+                f"GSF semantic_layer.sql_attributes.{key} must be a list"
+            )
+    return root
+
+
+def _parse_ossie(value: str) -> tuple[dict[str, Any], dict[str, Any]]:
+    root = _load_yaml(value, "Ossie")
+    unknown = sorted(set(root) - {"version", "semantic_model"})
+    if unknown:
+        raise GSFConversionError(
+            "Unsupported Ossie root properties: " + ", ".join(unknown)
+        )
+    if str(root.get("version", "")) != OSSIE_VERSION:
+        raise GSFConversionError(
+            f"Unsupported Ossie version {root.get('version')!r}; "
+            f"supported version is {OSSIE_VERSION!r}"
+        )
+    models = root.get("semantic_model")
+    if not isinstance(models, list) or len(models) != 1:
+        raise GSFConversionError("Ossie input must contain exactly one semantic model")
+    model = models[0]
+    if not isinstance(model, dict) or not model.get("name"):
+        raise GSFConversionError("Ossie semantic model requires a name")
+    return root, model
+
+
+def _load_yaml(value: str, label: str) -> dict[str, Any]:
+    try:
+        root = yaml.safe_load(value)
+    except yaml.YAMLError as exc:
+        raise GSFConversionError(f"Invalid {label} YAML: {exc}") from exc
+    if not isinstance(root, dict):
+        raise GSFConversionError(f"Invalid {label} YAML: expected a root mapping")
+    return root
+
+
+def _validate_relationship(
+    relationship: Any,
+    datasets: Mapping[str, Any],
+) -> dict[str, Any]:
+    if not isinstance(relationship, dict) or not relationship.get("name"):
+        raise GSFConversionError("Every Ossie relationship needs a name")
+    from_name = relationship.get("from")
+    to_name = relationship.get("to")
+    if from_name not in datasets or to_name not in datasets:
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} references an unknown dataset"
+        )
+    from_columns = relationship.get("from_columns") or []
+    to_columns = relationship.get("to_columns") or []
+    if not from_columns or len(from_columns) != len(to_columns):
+        raise GSFConversionError(
+            f"Relationship {relationship['name']!r} must have equal, "
+            "non-empty column lists"
+        )
+    return relationship
+
+
+def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
+    if not isinstance(value, dict):
+        raise GSFConversionError(f"{name!r} has no valid expression")
+    dialects = value.get("dialects")
+    if not isinstance(dialects, list) or not dialects:
+        raise GSFConversionError(f"{name!r} requires at least one expression dialect")
+    result = [
+        {
+            "dialect": str(item["dialect"]),
+            "expression": str(item["expression"]),
+        }
+        for item in dialects
+        if isinstance(item, dict)
+        and item.get("dialect")
+        and item.get("expression") is not None
+    ]
+    if not result:
+        raise GSFConversionError(f"{name!r} has no usable expression dialect")
+    return result
+
+
+def _pick_expression(expressions: list[dict[str, str]], name: str) -> str:
+    for expression in expressions:
+        if expression["dialect"].upper() == "ANSI_SQL":
+            return expression["expression"]
+    if expressions:
+        return expressions[0]["expression"]
+    raise GSFConversionError(f"{name!r} has no usable expression")
+
+
+def _simple_source_column(
+    expression: str,
+    dataset_name: str,
+    table_name: str,
+) -> str | None:
+    match = _SIMPLE_COLUMN.fullmatch(expression.strip())
+    if not match:
+        return None
+    qualifier = match.group("qualifier")
+    if qualifier and qualifier not in (dataset_name, table_name):
+        return None
+    return match.group("column")
+
+
+def _field_table_refs(
+    field: Mapping[str, Any],
+    expression: str,
+    owner: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(field)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(field.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"SQL attribute {field.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    result = list(dict.fromkeys([owner, *refs]))
+    _validate_single_database_refs(
+        result,
+        datasets,
+        f"SQL attribute {field.get('name')!r}",
+        sql=reference_sql,
+    )
+    return result
+
+
+def _metric_table_refs(
+    metric: Mapping[str, Any],
+    expression: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    extension = _gsf_extension_data(metric)
+    extension_refs = extension.get("table_refs")
+    expression_unchanged = _expression_matches(
+        expression, extension.get("ossie_expression")
+    )
+    reference_sql = (
+        str(extension["sql"])
+        if extension.get("sql") and expression_unchanged
+        else expression
+    )
+    use_extension_refs = not extension.get("entity") or expression_unchanged
+    if use_extension_refs and isinstance(extension_refs, list) and extension_refs:
+        refs = [str(item) for item in extension_refs]
+        _validate_refs(refs, datasets, str(metric.get("name")))
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    refs = _referenced_datasets(reference_sql, datasets)
+    if refs:
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if len(datasets) == 1:
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    if extension.get("entity") == "custom_analysis" and extension.get("sql"):
+        refs = [next(iter(datasets))]
+        _validate_single_database_refs(
+            refs,
+            datasets,
+            f"Custom analysis {metric.get('name')!r}",
+            sql=reference_sql,
+        )
+        return refs
+    raise GSFConversionError(
+        f"Metric {metric.get('name')!r} does not identify a source dataset; "
+        "qualify a referenced column or add NVIDIA_GSF table_refs"
+    )
+
+
+def _validate_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    name: str,
+) -> None:
+    unknown = [ref for ref in refs if ref not in datasets]
+    if unknown:
+        raise GSFConversionError(
+            f"{name!r} has unknown NVIDIA_GSF table_refs: {', '.join(unknown)}"
+        )
+
+
+def _validate_single_database_refs(
+    refs: Iterable[str],
+    datasets: Mapping[str, Any],
+    context: str,
+    *,
+    sql: str,
+) -> None:
+    databases = {
+        str(datasets[ref]["source"]["database"]) for ref in refs if ref in datasets
+    }
+    databases.update(
+        table.catalog for table in _parse_sql(sql).find_all(exp.Table) if table.catalog
+    )
+    if len(databases) > 1:
+        raise GSFConversionError(
+            f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
+            "the GSF importer validates each SQL object against one database"
+        )
+
+
+def _referenced_datasets(
+    sql: str,
+    datasets: Mapping[str, Any],
+) -> list[str]:
+    references: list[str] = []
+    parsed = _parse_sql(sql)
+    for table in parsed.find_all(exp.Table):
+        matches = [
+            name
+            for name, context in datasets.items()
+            if table.name == str(context["source"]["table"])
+            and (not table.db or table.db == str(context["source"]["schema"]))
+            and (
+                not table.catalog or table.catalog == str(context["source"]["database"])
+            )
+        ]
+        if matches:
+            match = matches[0]
+            if match not in references:
+                references.append(match)
+    for column in parsed.find_all(exp.Column):
+        qualifier = column.table
+        if not qualifier:
+            continue
+        matches = [
+            name
+            for name, context in datasets.items()
+            if qualifier in (name, str(context["source"]["table"]))
+        ]
+        if len(matches) == 1 and matches[0] not in references:
+            references.append(matches[0])
+    return references
+
+
+def _collect_expression_columns(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    preserved: Mapping[str, Any],
+) -> None:
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        if not dataset_name:
+            continue
+        context = datasets[dataset_name]
+        source_key = _source_key(context["source"])
+        known_table = source_key in preserved["tables"]
+        if known_table and (*source_key, column.name) not in preserved["columns"]:
+            # A GSF-sourced catalog is authoritative: an identifier that is not
+            # already a column of the table is SQL syntax, not physical data.
+            continue
+        _add_catalog_column(context, column.name)
+
+
+def _sql_column_ids(
+    sql: str,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    result: list[str] = []
+    for column in _sql_columns(sql):
+        dataset_name = _column_dataset(column, owner, refs, datasets)
+        column_id = (
+            column_ids.get((dataset_name, column.name)) if dataset_name else None
+        )
+        if column_id and column_id not in result:
+            result.append(column_id)
+    return result
+
+
+def _column_dataset(
+    column: exp.Column,
+    owner: str | None,
+    refs: list[str],
+    datasets: Mapping[str, Any],
+) -> str | None:
+    qualifier = column.table
+    if qualifier:
+        matches = [
+            name
+            for name, context in datasets.items()
+            if name in refs and qualifier in (name, str(context["source"]["table"]))
+        ]
+        return matches[0] if len(matches) == 1 else None
+    if owner:
+        return owner
+    return refs[0] if len(refs) == 1 else None
+
+
+def _sql_columns(sql: str) -> list[exp.Column]:
+    return [
+        column
+        for column in _parse_sql(sql).find_all(exp.Column)
+        if not _is_date_part_unit(column)
+    ]
+
+
+def _is_date_part_unit(column: exp.Column) -> bool:
+    """Report whether a parsed column is really a date-part keyword.
+
+    ``DATEDIFF(day, a, b)`` and friends put the unit in an argument slot that
+    sqlglot parses as an unqualified column, which would otherwise be mistaken
+    for a physical catalog column.
+    """
+    parent = column.parent
+    if column.table or column.name.lower() not in _DATE_PART_UNITS:
+        return False
+    if not isinstance(parent, exp.Func):
+        return False
+    if isinstance(parent, exp.Anonymous):
+        return str(parent.this).lower() in _DATE_FUNCTIONS
+    try:
+        names = parent.sql_names()
+    except (AttributeError, IndexError):
+        return False
+    return any(name.lower() in _DATE_FUNCTIONS for name in names)
+
+
+def _parse_sql(sql: str) -> exp.Expression:
+    try:
+        return parse_one(sql)
+    except (ParseError, ValueError) as exc:
+        raise GSFConversionError(
+            f"Unable to parse SQL expression {sql!r}: {exc}"
+        ) from exc
+
+
+def _first_resolvable_column_ids(
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    column_ids: Mapping[tuple[str, str], str],
+) -> list[str]:
+    for ref in refs:
+        for column in datasets[ref]["columns"]:
+            column_id = column_ids.get((ref, column))
+            if column_id:
+                return [column_id]
+    return []
+
+
+def _wrap_expression(
+    expression: str,
+    name: str,
+    refs: list[str],
+    datasets: Mapping[str, dict[str, Any]],
+    relationships: list[dict[str, Any]],
+) -> str:
+    stripped = expression.strip()
+    if stripped.upper().startswith(("SELECT ", "SELECT\n", "WITH ", "WITH\n")):
+        return stripped
+    if not refs:
+        raise GSFConversionError(f"Cannot determine a source table for {name!r}")
+    anchor_name = refs[0]
+    from_sql = (
+        f"{_qualified_table(datasets[anchor_name]['source'])} "
+        f"AS {_quote_identifier(anchor_name)}"
+    )
+    joined = {anchor_name}
+    remaining = set(refs[1:])
+    joins: list[str] = []
+    while remaining:
+        matched = False
+        for relationship in relationships:
+            left = str(relationship["from"])
+            right = str(relationship["to"])
+            if left in joined and right in remaining:
+                new_name = right
+            elif right in joined and left in remaining:
+                new_name = left
+            else:
+                continue
+            conditions = [
+                f"{_quote_identifier(left)}.{_quote_identifier(str(left_column))} = "
+                f"{_quote_identifier(right)}.{_quote_identifier(str(right_column))}"
+                for left_column, right_column in zip(
+                    relationship["from_columns"],
+                    relationship["to_columns"],
+                    strict=True,
+                )
+            ]
+            joins.append(
+                f"JOIN {_qualified_table(datasets[new_name]['source'])} "
+                f"AS {_quote_identifier(new_name)} ON {' AND '.join(conditions)}"
+            )
+            joined.add(new_name)
+            remaining.remove(new_name)
+            matched = True
+            break
+        if not matched:
+            missing = min(remaining)
+            raise GSFConversionError(
+                f"{name!r} references disconnected dataset {missing!r}; "
+                "declare a relationship connecting all referenced datasets"
+            )
+    suffix = f" {' '.join(joins)}" if joins else ""
+    return f"SELECT {stripped} AS {_quote_identifier(name)} FROM {from_sql}{suffix}"
+
+
+def _expression_from_sql(sql: str) -> str:
+    try:
+        parsed = parse_one(sql)
+    except (ParseError, ValueError):
+        return sql
+    select = parsed.find(exp.Select)
+    if select is None or not select.expressions:
+        return sql
+    expression = select.expressions[0]
+    if isinstance(expression, exp.Alias):
+        expression = expression.this
+    return expression.sql()
+
+
+def _expression_matches(current: str, emitted: Any) -> bool:
+    if not isinstance(emitted, str) or not emitted.strip():
+        return False
+    try:
+        return parse_one(current).sql() == parse_one(emitted).sql()
+    except (ParseError, ValueError):
+        return current.strip() == emitted.strip()
+
+
+def _ossie_expression(expression: str) -> dict[str, Any]:

Review Comment:
   Fixed too!



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscribe@ossie.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org