You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@ossie.apache.org by jb...@apache.org on 2026/10/02 15:24:53 UTC
(ossie) branch main updated: fix(nvidia): rename the GSF converter to Auto Ontology (#499)
This is an automated email from the ASF dual-hosted git repository.
jbonofre pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/ossie.git
The following commit(s) were added to refs/heads/main by this push:
new 3cb21f24 fix(nvidia): rename the GSF converter to Auto Ontology (#499)
3cb21f24 is described below
commit 3cb21f240b1f0be3151a0ea57bd5b5eb91792f04
Author: liofnv <lf...@nvidia.com>
AuthorDate: Fri Oct 2 10:24:48 2026 -0500
fix(nvidia): rename the GSF converter to Auto Ontology (#499)
NVIDIA renamed Generative Semantic Fabric (GSF) to Auto Ontology, and its
native model document is now AutoOntologyModelDocument. The document shape
itself is unchanged, so this is a rename with no mapping changes.
Rename the distribution to apache-ossie-nvidia-auto-ontology, the import
package to ossie_nvidia_auto_ontology, the console script to
ossie-nvidia-auto-ontology, and the public API to
AutoOntologyConversionError, convert_auto_ontology_to_ossie and
convert_ossie_to_auto_ontology. The converters/nvidia directory is kept, as
it is named after the vendor rather than the product.
The custom extension vendor code becomes NVIDIA_AUTO_ONTOLOGY. Extensions
written under NVIDIA_GSF or GSF are still read, so Ossie files exported
before the rename keep their round-trip fidelity.
Also drop the README's stale Neo4j mention (Auto Ontology now runs on
Postgres), update the vendor table and cross-converter prose, and point the
Cube interop matrix at converters/nvidia, which it had been missing since
the directory moved from converters/gsf.
---
.github/workflows/converter-nvidia-ci.yml | 2 +-
converters/README.md | 2 +-
converters/cube/README.md | 12 +-
converters/cube/pyproject.toml | 2 +-
converters/cube/src/ossie_cube/cube_to_osi.py | 14 +-
converters/cube/src/ossie_cube/expressions.py | 2 +-
converters/cube/tests/test_edge_cases.py | 4 +-
converters/cube/tools/interop_matrix.py | 2 +-
converters/nvidia/README.md | 169 +++++------
converters/nvidia/pyproject.toml | 10 +-
.../__init__.py | 14 +-
.../converter.py | 14 +-
.../native_converter.py | 309 ++++++++++++---------
.../{sales.gsf.yaml => sales.auto_ontology.yaml} | 0
converters/nvidia/tests/test_converter.py | 246 +++++++++-------
converters/nvidia/uv.lock | 2 +-
converters/sigma/src/ossie_sigma/sigma_formula.py | 4 +-
17 files changed, 448 insertions(+), 360 deletions(-)
diff --git a/.github/workflows/converter-nvidia-ci.yml b/.github/workflows/converter-nvidia-ci.yml
index 32ef91fa..d0d419b0 100644
--- a/.github/workflows/converter-nvidia-ci.yml
+++ b/.github/workflows/converter-nvidia-ci.yml
@@ -17,7 +17,7 @@
# under the License.
#
-name: Converters NVIDIA GSF CI
+name: Converters NVIDIA Auto Ontology CI
on:
push:
diff --git a/converters/README.md b/converters/README.md
index 19ea72d6..fa0a8cfc 100644
--- a/converters/README.md
+++ b/converters/README.md
@@ -85,7 +85,7 @@ The Ossie specification currently defines extensions for the following vendors:
| `DATABRICKS` | Databricks semantic layer |
| `OMNI` | Omni semantic model |
| `WISDOM` | WisdomAI domain |
-| `NVIDIA_GSF` | NVIDIA Generative Semantic Fabric standalone YAML |
+| `NVIDIA_AUTO_ONTOLOGY` | NVIDIA Auto Ontology model document YAML (formerly `NVIDIA_GSF`) |
| `SIGMA` | Sigma Computing data model |
| `THOUGHTSPOT` | ThoughtSpot TML (Model + Table/SQL View) |
| `CUBE` | Cube data model |
diff --git a/converters/cube/README.md b/converters/cube/README.md
index 9c3ff4f4..4a095195 100644
--- a/converters/cube/README.md
+++ b/converters/cube/README.md
@@ -60,8 +60,8 @@ pip install -e .
```
Runtime dependencies are `PyYAML` and `sqlglot` (already a runtime dependency of
-the dbt and NVIDIA GSF converters, used here to locate the aggregate calls inside a
-composite metric). Python 3.11+.
+the dbt and NVIDIA Auto Ontology converters, used here to locate the aggregate
+calls inside a composite metric). Python 3.11+.
## Usage
@@ -441,8 +441,8 @@ metric belongs on) is derived rather than recorded. On the TPC-DS model that is
stash entries rather than 41, and 2 Databricks warnings rather than 32.
**Qualify your `sql_table`.** Cube accepts `orders` or `public.orders`, but the
-Databricks, Snowflake and NVIDIA GSF converters all require a three-part
-`catalog.schema.table` and reject anything shorter:
+Databricks, Snowflake and NVIDIA Auto Ontology converters all require a
+three-part `catalog.schema.table` and reject anything shorter:
```
Error: Dataset 'orders': source 'public.orders' must be a 3-part catalog.schema.table
@@ -478,8 +478,8 @@ spoke result warns foreign note
databricks OK 22 2
dbt FAIL 0 0 AttributeError: 'PydanticSemanticManifes
gooddata OK 0 0
-gsf OK 0 0
honeydew OK 0 0
+nvidia OK 0 0
omni OK 15 7
orionbelt OK 2 0
snowflake OK 7 7
@@ -513,7 +513,7 @@ detail string.
| `GEO_DIMENSION_SPLIT` | A `type: geo` dimension became two Ossie fields |
| `TEMPLATED_FILE_SKIPPED` | Jinja templating anywhere in a file, or a `.js`/`.ts` model file. Detected per file, as Cube's own tooling does, so the file is preserved whole rather than half-converted |
| `NO_USABLE_DIALECT` | Export: no `ANSI_SQL` or preferred-dialect expression |
-| `SOURCE_NOT_FULLY_QUALIFIED` | A `sql_table` shorter than `catalog.schema.table`. Valid Cube and nothing is lost, but the Databricks, Snowflake and NVIDIA GSF converters reject such a source, so the model cannot convert onward — see [Onward conversion](#onward-conversion) |
+| `SOURCE_NOT_FULLY_QUALIFIED` | A `sql_table` shorter than `catalog.schema.table`. Valid Cube and nothing is lost, but the Databricks, Snowflake and NVIDIA Auto Ontology converters reject such a source, so the model cannot convert onward — see [Onward conversion](#onward-conversion) |
| `PARKED_IN_META` | Preserved in the stash or under `meta.ossie` — invisible to Cube, but intact through a round trip |
| `DROPPED_NO_CUBE_EQUIVALENT` | **Gone from the output.** Cube has nowhere to hold it and it cannot be parked: relationship `ai_context` (a Cube join entry has no `meta`) and a `dimension.is_time` role or opt-out that Cube expresses only through `type` |
| `DROPPED_FROM_PROJECTION` | View projection only. **Gone from the output**: a Cube-only property of the view, a projected cube or a published member -- an access policy, pre-aggregations, drill members, extra `meta` -- that the round-trip import would preserve, but a projection of the public surface does not carry |
diff --git a/converters/cube/pyproject.toml b/converters/cube/pyproject.toml
index 72a087d4..7c587e2f 100644
--- a/converters/cube/pyproject.toml
+++ b/converters/cube/pyproject.toml
@@ -40,7 +40,7 @@ dependencies = [
"PyYAML>=6.0",
# Expression handling: locating the aggregate calls inside a composite metric so
# each can become its own Cube measure. Already a runtime dependency of the dbt
- # and NVIDIA GSF converters, which use it for the same purpose.
+ # and NVIDIA Auto Ontology converters, which use it for the same purpose.
"sqlglot>=20.0",
]
diff --git a/converters/cube/src/ossie_cube/cube_to_osi.py b/converters/cube/src/ossie_cube/cube_to_osi.py
index 5eaa84b6..7ff3fa01 100644
--- a/converters/cube/src/ossie_cube/cube_to_osi.py
+++ b/converters/cube/src/ossie_cube/cube_to_osi.py
@@ -680,15 +680,15 @@ def _convert_cube(cname, cube, plain, extra_joins, extra_measures, issues):
parts = source_part_count(ds["source"])
if parts is not None and parts < 3:
# Cube accepts a one- or two-part `sql_table`, but the Ossie spec describes
- # `source` as `database.schema.table` and the Databricks, Snowflake and NVIDIA
- # GSF converters all reject anything shorter -- so a model that converts
- # cleanly here still cannot reach them. Better to say so at the point the
- # Ossie document is produced than to have it fail three hops later.
+ # `source` as `database.schema.table` and the Databricks, Snowflake and
+ # NVIDIA Auto Ontology converters all reject anything shorter -- so a model
+ # that converts cleanly here still cannot reach them. Better to say so at the
+ # point the Ossie document is produced than to have it fail three hops later.
issues.add(IssueType.SOURCE_NOT_FULLY_QUALIFIED, scope,
f"source '{ds['source']}' has {parts} part(s); several Ossie "
- f"converters (Databricks, Snowflake, NVIDIA GSF) require a "
- f"3-part catalog.schema.table, so qualify the cube's `sql_table` "
- f"if the model needs to convert onward")
+ f"converters (Databricks, Snowflake, NVIDIA Auto Ontology) "
+ f"require a 3-part catalog.schema.table, so qualify the cube's "
+ f"`sql_table` if the model needs to convert onward")
if cube.get("description"):
ds["description"] = unescape_braces_from_cube(cube["description"])
diff --git a/converters/cube/src/ossie_cube/expressions.py b/converters/cube/src/ossie_cube/expressions.py
index 9a2edbdc..62682c7a 100644
--- a/converters/cube/src/ossie_cube/expressions.py
+++ b/converters/cube/src/ossie_cube/expressions.py
@@ -32,7 +32,7 @@ improvement for cross-dataset metrics, not a formatting choice.
Locating the aggregate calls is done with sqlglot rather than a regex, since an
expression can nest them (`SUM(x) / NULLIF(SUM(y), 0)`) and string matching cannot
tell a top-level call from one inside another argument. sqlglot is already a runtime
-dependency of the dbt and NVIDIA GSF converters for the same purpose.
+dependency of the dbt and NVIDIA Auto Ontology converters for the same purpose.
"""
import re
diff --git a/converters/cube/tests/test_edge_cases.py b/converters/cube/tests/test_edge_cases.py
index 5de48d8a..ec68f422 100644
--- a/converters/cube/tests/test_edge_cases.py
+++ b/converters/cube/tests/test_edge_cases.py
@@ -503,8 +503,8 @@ def test_field_and_metric_foreign_extensions_survive_the_round_trip():
])
def test_a_source_that_other_converters_reject_is_reported(sql_table, parts, warns):
"""Cube is happy with a one- or two-part `sql_table`, but the Databricks,
- Snowflake and NVIDIA GSF converters all reject a source shorter than
- `catalog.schema.table` -- so a model that converts cleanly here still cannot
+ Snowflake and NVIDIA Auto Ontology converters all reject a source shorter
+ than `catalog.schema.table` -- so a model that converts cleanly here still cannot
travel. Reported at the point the Ossie document is produced, rather than being
discovered three hops later."""
files = _files(orders=(
diff --git a/converters/cube/tools/interop_matrix.py b/converters/cube/tools/interop_matrix.py
index bb194087..fda9ae3b 100644
--- a/converters/cube/tools/interop_matrix.py
+++ b/converters/cube/tools/interop_matrix.py
@@ -67,8 +67,8 @@ SPOKES = [
("databricks", ["ossie-databricks", "export"], False),
("dbt", ["ossie-dbt", "osi-to-msi"], False),
("gooddata", None, False), # API-only; see _run_gooddata
- ("gsf", ["ossie-gsf", "export"], False),
("honeydew", ["honeydew-osi", "osi-to-honeydew"], True),
+ ("nvidia", ["ossie-nvidia-auto-ontology", "export"], False),
("omni", ["osi-omni", "export"], True),
("orionbelt", ["ossie-orionbelt", "osi-to-obml"], False),
("snowflake", ["ossie-snowflake"], False),
diff --git a/converters/nvidia/README.md b/converters/nvidia/README.md
index 1351c587..62d93b32 100644
--- a/converters/nvidia/README.md
+++ b/converters/nvidia/README.md
@@ -17,11 +17,15 @@
under the License.
-->
-# Apache Ossie ↔ NVIDIA GSF Converter
+# Apache Ossie ↔ NVIDIA Auto Ontology Converter
-Offline conversion between Apache Ossie YAML and NVIDIA GSF's native
-`GsfModelDocument` YAML contract. Conversion itself does not require GSF,
-Neo4j, a database, or network access.
+Offline conversion between Apache Ossie YAML and NVIDIA Auto Ontology's native
+`AutoOntologyModelDocument` YAML contract. Conversion itself does not require
+Auto Ontology, a database, or network access.
+
+Auto Ontology was previously named Generative Semantic Fabric (GSF). Ossie
+files written under the old name carry an `NVIDIA_GSF` custom extension; the
+converter still reads it, and writes `NVIDIA_AUTO_ONTOLOGY` from now on.
Ossie input and output use one model per document, with `name`, `datasets`,
`relationships`, and `metrics` directly at the root beside `version`. Migrate
@@ -30,7 +34,7 @@ legacy `semantic_model` wrappers before conversion; see the
## Mapping
-| Apache Ossie | Native GSF model document |
+| Apache Ossie | Native Auto Ontology model document |
|---|---|
| Dataset source | `data_layer.databases[].schemas[].tables[]` |
| Dataset field backed by one column | `semantic_layer.terms[].columns_attributes[]` |
@@ -55,12 +59,12 @@ cd converters/nvidia
uv sync
```
-## Ossie → GSF
+## Ossie → Auto Ontology
```bash
-uv run ossie-nvidia-gsf export \
+uv run ossie-nvidia-auto-ontology export \
--input ../../examples/tpcds_semantic_model.yaml \
- --output tpcds.gsf.yaml \
+ --output tpcds.auto_ontology.yaml \
--database-name tpcds
```
@@ -69,118 +73,123 @@ qualified `database.schema.table` sources do not require it. One document may
contain multiple databases.
```python
-from ossie_nvidia_gsf import convert_ossie_to_gsf
+from ossie_nvidia_auto_ontology import convert_ossie_to_auto_ontology
-gsf_yaml = convert_ossie_to_gsf(ossie_yaml, database_name="tpcds")
+auto_ontology_yaml = convert_ossie_to_auto_ontology(ossie_yaml, database_name="tpcds")
```
-## GSF → Ossie
+## Auto Ontology → Ossie
```bash
-uv run ossie-nvidia-gsf import \
- --input tpcds.gsf.yaml \
+uv run ossie-nvidia-auto-ontology import \
+ --input tpcds.auto_ontology.yaml \
--output semantic_model.yaml \
--name tpcds
```
`--name` overrides the Ossie model name. Without it, the converter uses the
-single catalog database name when there is one, otherwise `gsf_model`.
+single catalog database name when there is one, otherwise
+`auto_ontology_model`.
```python
-from ossie_nvidia_gsf import convert_gsf_to_ossie
+from ossie_nvidia_auto_ontology import convert_auto_ontology_to_ossie
-ossie_yaml = convert_gsf_to_ossie(gsf_yaml, model_name="tpcds")
+ossie_yaml = convert_auto_ontology_to_ossie(auto_ontology_yaml, model_name="tpcds")
```
-This converter subset currently accepts only GSF terms that represent exactly
-one table, because one Ossie dataset cannot represent several physical tables.
-Several terms may represent the same table and become distinct Ossie datasets
-sharing one source. SQL attributes become Ossie fields regardless of their GSF
-source group. Custom analyses remain global by becoming model-level Ossie
-metrics. Relationships are recovered from joins, then physical foreign keys,
-then semantic foreign keys.
+This converter subset currently accepts only Auto Ontology terms that represent
+exactly one table, because one Ossie dataset cannot represent several physical
+tables. Several terms may represent the same table and become distinct Ossie
+datasets sharing one source. SQL attributes become Ossie fields regardless of
+their Auto Ontology source group. Custom analyses remain global by becoming
+model-level Ossie metrics. Relationships are recovered from joins, then
+physical foreign keys, then semantic foreign keys.
-## Importing the model into GSF
+## Importing the model into Auto Ontology
-Start GSF, then send the native document to its REST API:
+Start Auto Ontology, then send the native document to its REST API:
```bash
curl --fail-with-body \
-X POST \
'http://127.0.0.1:3001/api/model/import?replace=true&embed=true' \
-H 'Content-Type: application/x-yaml' \
- --data-binary @tpcds.gsf.yaml
+ --data-binary @tpcds.auto_ontology.yaml
```
The endpoint also accepts a multipart upload in a `file` field.
-The target GSF instance must already have a connection configured for each
-database named in the document. GSF validates every imported SQL attribute
-against that connection's dialect, so importing into an instance with no
-matching connection fails. A database's `dialect` is likewise derived from the
-live connection rather than stored on import, so it is exported for information
-only and does not survive a GSF → GSF cycle.
+The target Auto Ontology instance must already have a connection configured for
+each database named in the document. Auto Ontology validates every imported SQL
+attribute against that connection's dialect, so importing into an instance with
+no matching connection fails. A database's `dialect` is likewise derived from
+the live connection rather than stored on import, so it is exported for
+information only and does not survive an Auto Ontology → Auto Ontology cycle.
## Fidelity and unavoidable losses
-When converting GSF to Ossie, the converter records the native document in an
-`NVIDIA_GSF` custom extension. A direct GSF → Ossie → GSF cycle can therefore
-reuse live identifiers and preserve catalog properties, SQL source groups,
-SQL text, `sql_column_is`, relationships, and zones. Ossie-origin entities use
+When converting Auto Ontology to Ossie, the converter records the native
+document in an `NVIDIA_AUTO_ONTOLOGY` custom extension. A direct
+Auto Ontology → Ossie → Auto Ontology cycle can therefore reuse live
+identifiers and preserve catalog properties, SQL source groups, SQL text,
+`sql_column_is`, relationships, and zones. Ossie-origin entities use
deterministic IDs when no preserved native ID is available. Current Ossie
expressions and relationships remain authoritative: preserved SQL and native
relationship records are reused only when they still correspond to the Ossie
entities or are outside the represented Ossie catalog scope.
That extension holds the whole native document, so an Ossie file produced from
-GSF carries a full copy of the GSF catalog alongside the model derived from it.
-This is a deliberate trade of size for round-trip fidelity: it is what lets a
-GSF → Ossie → GSF cycle keep live identifiers, and it means the Ossie output of
-a large catalog is bulky and not meant to be reviewed by hand. Converting
-Ossie → GSF from a hand-written Ossie file, which has no such extension, is
-unaffected.
+Auto Ontology carries a full copy of the Auto Ontology catalog alongside the
+model derived from it. This is a deliberate trade of size for round-trip
+fidelity: it is what lets an Auto Ontology → Ossie → Auto Ontology cycle keep
+live identifiers, and it means the Ossie output of a large catalog is bulky and
+not meant to be reviewed by hand. Converting Ossie → Auto Ontology from a
+hand-written Ossie file, which has no such extension, is unaffected.
SQL is parsed with sqlglot across a list of candidate dialects rather than a
-single one, because preserved GSF SQL carries whatever dialect its connection
-reported. SQL that no candidate can parse is treated as opaque: it is still
-carried through verbatim, and only the parse-derived enrichment (discovering
-which tables and columns an expression touches) is skipped, so a model that
-imported cleanly can always be exported again. On GSF → Ossie, expressions are
-labelled with the source connection's dialect when Ossie names it
-(`SNOWFLAKE`, `DATABRICKS`, `BIGQUERY`) and `ANSI_SQL` otherwise.
-
-For an Ossie-origin model there is no GSF catalog to check against, so physical
-columns are synthesized from the identifiers in each expression. Date-part
-keywords are excluded, but only in the unit argument of a recognized date
-function, so a column genuinely named `day` or `month` is kept everywhere else.
-A unit passed to a date function the converter does not recognize is still
-synthesized as a column; this is inherent to deriving a catalog from SQL text
-and does not apply once a GSF catalog is present, since a GSF-sourced catalog
-is authoritative and never widened.
-
-The GSF contract has no semantic-model envelope, `ai_context`, dimensions,
-synonyms, Ossie custom-extension storage, or expression-dialect variants.
-Those values cannot be represented in a native GSF document and are
-unavoidably lost on Ossie → GSF. GSF joins also have no relationship name, so
-GSF → Ossie synthesizes a stable `<from>_to_<to>` name. The converter never
-adds fictional fields to the GSF schema. GSF records uniqueness per column, so
-Ossie composite unique keys cannot be reconstructed after GSF → Ossie; only
+single one, because preserved Auto Ontology SQL carries whatever dialect its
+connection reported. SQL that no candidate can parse is treated as opaque: it
+is still carried through verbatim, and only the parse-derived enrichment
+(discovering which tables and columns an expression touches) is skipped, so a
+model that imported cleanly can always be exported again. On
+Auto Ontology → Ossie, expressions are labelled with the source connection's
+dialect when Ossie names it (`SNOWFLAKE`, `DATABRICKS`, `BIGQUERY`) and
+`ANSI_SQL` otherwise.
+
+For an Ossie-origin model there is no Auto Ontology catalog to check against,
+so physical columns are synthesized from the identifiers in each expression.
+Date-part keywords are excluded, but only in the unit argument of a recognized
+date function, so a column genuinely named `day` or `month` is kept everywhere
+else. A unit passed to a date function the converter does not recognize is
+still synthesized as a column; this is inherent to deriving a catalog from SQL
+text and does not apply once an Auto Ontology catalog is present, since a
+catalog sourced from Auto Ontology is authoritative and never widened.
+
+The Auto Ontology contract has no semantic-model envelope, `ai_context`,
+dimensions, synonyms, Ossie custom-extension storage, or expression-dialect
+variants. Those values cannot be represented in a native Auto Ontology document
+and are unavoidably lost on Ossie → Auto Ontology. Auto Ontology joins also
+have no relationship name, so Auto Ontology → Ossie synthesizes a stable
+`<from>_to_<to>` name. The converter never adds fictional fields to the
+Auto Ontology schema. Auto Ontology records uniqueness per column, so Ossie
+composite unique keys cannot be reconstructed after Auto Ontology → Ossie; only
single-column unique keys survive.
-Ossie's `datatype` maps to and from the physical type on a GSF catalog column,
-for fields backed by a single column. GSF → Ossie reduces the physical type to
-Ossie's logical vocabulary, so `NUMBER(38,0)` becomes `Integer`, `NUMBER(12,2)`
-becomes `Decimal`, and a type Ossie cannot name, such as Snowflake's `VARIANT`,
-becomes `Opaque` as the spec prescribes. Ossie → GSF writes a canonical physical
-type for a column the Ossie model introduces, and never overrides a type
-preserved from a real GSF catalog, since GSF reports what the connection
-actually holds. The two mappings are inverses, so a declared `datatype` survives
-a full cycle.
-
-A computed field or a metric has no single column behind it and GSF stores no
-type for either, so their `datatype` is not carried. `Opaque` is not written
-back, because it names a type outside the vocabulary and there is no physical
-type worth inventing from it.
+Ossie's `datatype` maps to and from the physical type on an Auto Ontology
+catalog column, for fields backed by a single column. Auto Ontology → Ossie
+reduces the physical type to Ossie's logical vocabulary, so `NUMBER(38,0)`
+becomes `Integer`, `NUMBER(12,2)` becomes `Decimal`, and a type Ossie cannot
+name, such as Snowflake's `VARIANT`, becomes `Opaque` as the spec prescribes.
+Ossie → Auto Ontology writes a canonical physical type for a column the Ossie
+model introduces, and never overrides a type preserved from a real
+Auto Ontology catalog, since Auto Ontology reports what the connection actually
+holds. The two mappings are inverses, so a declared `datatype` survives a full
+cycle.
+
+A computed field or a metric has no single column behind it and Auto Ontology
+stores no type for either, so their `datatype` is not carried. `Opaque` is not
+written back, because it names a type outside the vocabulary and there is no
+physical type worth inventing from it.
## Tests
diff --git a/converters/nvidia/pyproject.toml b/converters/nvidia/pyproject.toml
index 68b70a81..6758eaf8 100644
--- a/converters/nvidia/pyproject.toml
+++ b/converters/nvidia/pyproject.toml
@@ -26,9 +26,9 @@ dev = [
]
[project]
-name = "apache-ossie-nvidia-gsf"
+name = "apache-ossie-nvidia-auto-ontology"
version = "0.1.0.dev0"
-description = "NVIDIA GSF <> Apache Ossie offline YAML converter"
+description = "NVIDIA Auto Ontology <> Apache Ossie offline YAML converter"
authors = [{ name = "Apache Software Foundation", email = "dev@ossie.apache.org" }]
requires-python = ">=3.11"
readme = "README.md"
@@ -36,7 +36,7 @@ license = "Apache-2.0"
keywords = [
"Apache Ossie",
"Ossie",
- "NVIDIA GSF",
+ "NVIDIA Auto Ontology",
"semantic model",
]
dependencies = [
@@ -45,14 +45,14 @@ dependencies = [
]
[project.scripts]
-ossie-nvidia-gsf = "ossie_nvidia_gsf.converter:main"
+ossie-nvidia-auto-ontology = "ossie_nvidia_auto_ontology.converter:main"
[project.urls]
homepage = "https://ossie.apache.org/"
repository = "https://github.com/apache/ossie/"
[tool.hatch.build.targets.wheel]
-packages = ["src/ossie_nvidia_gsf"]
+packages = ["src/ossie_nvidia_auto_ontology"]
[tool.pytest.ini_options]
testpaths = ["tests"]
diff --git a/converters/nvidia/src/ossie_nvidia_gsf/__init__.py b/converters/nvidia/src/ossie_nvidia_auto_ontology/__init__.py
similarity index 74%
rename from converters/nvidia/src/ossie_nvidia_gsf/__init__.py
rename to converters/nvidia/src/ossie_nvidia_auto_ontology/__init__.py
index 96522e87..f6524b2d 100644
--- a/converters/nvidia/src/ossie_nvidia_gsf/__init__.py
+++ b/converters/nvidia/src/ossie_nvidia_auto_ontology/__init__.py
@@ -15,16 +15,16 @@
# specific language governing permissions and limitations
# under the License.
-"""Bidirectional Apache Ossie and NVIDIA GSF converter."""
+"""Bidirectional Apache Ossie and NVIDIA Auto Ontology converter."""
from .converter import (
- GSFConversionError,
- convert_gsf_to_ossie,
- convert_ossie_to_gsf,
+ AutoOntologyConversionError,
+ convert_auto_ontology_to_ossie,
+ convert_ossie_to_auto_ontology,
)
__all__ = [
- "GSFConversionError",
- "convert_gsf_to_ossie",
- "convert_ossie_to_gsf",
+ "AutoOntologyConversionError",
+ "convert_auto_ontology_to_ossie",
+ "convert_ossie_to_auto_ontology",
]
diff --git a/converters/nvidia/src/ossie_nvidia_gsf/converter.py b/converters/nvidia/src/ossie_nvidia_auto_ontology/converter.py
similarity index 75%
rename from converters/nvidia/src/ossie_nvidia_gsf/converter.py
rename to converters/nvidia/src/ossie_nvidia_auto_ontology/converter.py
index 0b0e4811..7f0074fd 100644
--- a/converters/nvidia/src/ossie_nvidia_gsf/converter.py
+++ b/converters/nvidia/src/ossie_nvidia_auto_ontology/converter.py
@@ -15,19 +15,19 @@
# specific language governing permissions and limitations
# under the License.
-"""Public API and CLI for native NVIDIA GSF model conversion."""
+"""Public API and CLI for native NVIDIA Auto Ontology model conversion."""
from .native_converter import (
- GSFConversionError,
- convert_gsf_to_ossie,
- convert_ossie_to_gsf,
+ AutoOntologyConversionError,
+ convert_auto_ontology_to_ossie,
+ convert_ossie_to_auto_ontology,
main,
)
__all__ = [
- "GSFConversionError",
- "convert_gsf_to_ossie",
- "convert_ossie_to_gsf",
+ "AutoOntologyConversionError",
+ "convert_auto_ontology_to_ossie",
+ "convert_ossie_to_auto_ontology",
"main",
]
diff --git a/converters/nvidia/src/ossie_nvidia_gsf/native_converter.py b/converters/nvidia/src/ossie_nvidia_auto_ontology/native_converter.py
similarity index 88%
rename from converters/nvidia/src/ossie_nvidia_gsf/native_converter.py
rename to converters/nvidia/src/ossie_nvidia_auto_ontology/native_converter.py
index 3cc13992..accfa820 100644
--- a/converters/nvidia/src/ossie_nvidia_gsf/native_converter.py
+++ b/converters/nvidia/src/ossie_nvidia_auto_ontology/native_converter.py
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-"""Apache Ossie ↔ native NVIDIA GSF model-document conversion."""
+"""Apache Ossie ↔ native NVIDIA Auto Ontology model-document conversion."""
from __future__ import annotations
@@ -39,8 +39,10 @@ OSSIE_VERSION = "0.2.0.dev0"
# still on a .dev line, so pinning the exact string would reject every real
# model as soon as the patch or dev suffix moves.
OSSIE_SERIES = tuple(int(part) for part in OSSIE_VERSION.split(".")[:2])
-NVIDIA_GSF_VENDOR = "NVIDIA_GSF"
-GSF_VENDOR_ALIASES = {NVIDIA_GSF_VENDOR, "GSF"}
+NVIDIA_AUTO_ONTOLOGY_VENDOR = "NVIDIA_AUTO_ONTOLOGY"
+# Ossie files exported before the product was renamed from GSF carry these.
+LEGACY_VENDOR_NAMES = {"NVIDIA_GSF", "GSF"}
+AUTO_ONTOLOGY_VENDOR_ALIASES = {NVIDIA_AUTO_ONTOLOGY_VENDOR, *LEGACY_VENDOR_NAMES}
_ID_NAMESPACE = UUID("03d14261-6432-50fe-b099-77e8061af4f9")
_SQL_GROUPS = ("manual", "table", "sql", "bridge_table")
_SIMPLE_COLUMN = re.compile(
@@ -48,7 +50,7 @@ _SIMPLE_COLUMN = re.compile(
r"(?P<column>[A-Za-z_][A-Za-z0-9_]*)$"
)
# sqlglot's default parser first, since it is closest to ANSI, then the
-# dialects GSF connections commonly report.
+# dialects Auto Ontology connections commonly report.
_SQL_DIALECTS = (
"",
"snowflake",
@@ -153,14 +155,15 @@ _UNIT_IN_THIS_FUNCTIONS = tuple(
if isinstance(node, type)
)
# Ossie names only a few dialects; everything else has no equivalent.
-_GSF_TO_OSSIE_DIALECT = {
+_AUTO_ONTOLOGY_TO_OSSIE_DIALECT = {
"snowflake": "SNOWFLAKE",
"databricks": "DATABRICKS",
"bigquery": "BIGQUERY",
}
-# A GSF column carries the physical type its connection reports. Ossie names ten
-# logical types, so the mapping is deliberately coarse in that direction and
-# canonical in the other, which is what lets a datatype survive a full cycle.
+# An Auto Ontology column carries the physical type its connection reports.
+# Ossie names ten logical types, so the mapping is deliberately coarse in that
+# direction and canonical in the other, which is what lets a datatype survive a
+# full cycle.
_OSSIE_DATATYPE_BY_SQL_TYPE = {
"VARCHAR": "String",
"VARCHAR2": "String",
@@ -229,20 +232,20 @@ _SQL_TYPE_BY_OSSIE_DATATYPE = {
}
-class GSFConversionError(Exception):
+class AutoOntologyConversionError(Exception):
"""Raised when a document cannot be converted safely."""
-def convert_ossie_to_gsf(
+def convert_ossie_to_auto_ontology(
ossie_yaml: str,
*,
database_name: str | None = None,
) -> str:
- """Convert one Apache Ossie model to a native ``GsfModelDocument``."""
+ """Convert one Apache Ossie model to a native ``AutoOntologyModelDocument``."""
model = _parse_ossie(ossie_yaml)
source_datasets = model.get("datasets") or []
if not isinstance(source_datasets, list) or not source_datasets:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
"The Ossie semantic model must contain at least one dataset"
)
@@ -251,15 +254,15 @@ def convert_ossie_to_gsf(
datasets: dict[str, dict[str, Any]] = {}
for item in source_datasets:
if not isinstance(item, dict) or not item.get("name"):
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
"Every Ossie dataset must be a mapping with a name"
)
name = str(item["name"])
if name in datasets:
- raise GSFConversionError(f"Duplicate dataset name {name!r}")
+ raise AutoOntologyConversionError(f"Duplicate dataset name {name!r}")
source = _parse_source(item.get("source"), database_name)
if not source["database"] or not source["schema"]:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Dataset {name!r} source must resolve to database.schema.table"
)
datasets[name] = {
@@ -295,12 +298,12 @@ def convert_ossie_to_gsf(
field_names: set[str] = set()
for field in dataset.get("fields") or []:
if not isinstance(field, dict) or not field.get("name"):
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Every field in dataset {name!r} needs a name"
)
field_name = str(field["name"])
if field_name in field_names:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Duplicate field name {field_name!r} in dataset {name!r}"
)
field_names.add(field_name)
@@ -321,10 +324,12 @@ def convert_ossie_to_gsf(
metric_names: set[str] = set()
for metric in model.get("metrics") or []:
if not isinstance(metric, dict) or not metric.get("name"):
- raise GSFConversionError("Every Ossie metric must be a mapping with a name")
+ raise AutoOntologyConversionError(
+ "Every Ossie metric must be a mapping with a name"
+ )
name = str(metric["name"])
if name in metric_names:
- raise GSFConversionError(f"Duplicate metric name {name!r}")
+ raise AutoOntologyConversionError(f"Duplicate metric name {name!r}")
metric_names.add(name)
expressions = _normalize_expressions(metric.get("expression"), name)
selected = _pick_expression(expressions, name)
@@ -424,7 +429,7 @@ def convert_ossie_to_gsf(
for name, context in datasets.items():
for field, _, selected, refs in context["computed_fields"]:
field_name = str(field["name"])
- extension = _gsf_extension_data(field)
+ extension = _auto_ontology_extension_data(field)
preserved_attr = preserved["sql_attributes"].get((name, field_name), {})
expression_unchanged = _expression_matches(
selected, extension.get("ossie_expression")
@@ -475,7 +480,7 @@ def convert_ossie_to_gsf(
custom_analyses: list[dict[str, Any]] = []
for metric, _, selected, refs in metrics:
name = str(metric["name"])
- extension = _gsf_extension_data(metric)
+ extension = _auto_ontology_extension_data(metric)
preserved_analysis = preserved["custom_analyses"].get(name, {})
expression_unchanged = _expression_matches(
selected, extension.get("ossie_expression")
@@ -500,7 +505,7 @@ def convert_ossie_to_gsf(
refs, datasets, column_ids
)
if not referenced_column_ids:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Metric {name!r} has no resolvable catalog column for "
"custom-analysis SQL validation"
)
@@ -544,21 +549,21 @@ def convert_ossie_to_gsf(
return _dump_yaml(output)
-def convert_gsf_to_ossie(
- gsf_yaml: str,
+def convert_auto_ontology_to_ossie(
+ auto_ontology_yaml: str,
*,
model_name: str | None = None,
) -> str:
- """Convert a native ``GsfModelDocument`` to one Apache Ossie model."""
- root = _parse_gsf(gsf_yaml)
+ """Convert a native ``AutoOntologyModelDocument`` to one Apache Ossie model."""
+ root = _parse_auto_ontology(auto_ontology_yaml)
catalog = _read_catalog(root)
dialects = _dialects_by_database(root)
semantic = root["semantic_layer"]
terms = semantic["terms"]
if not terms:
- raise GSFConversionError(
- "GSF document has no representable terms; Ossie requires at least "
- "one dataset"
+ raise AutoOntologyConversionError(
+ "Auto Ontology document has no representable terms; Ossie requires "
+ "at least one dataset"
)
term_by_id: dict[str, dict[str, Any]] = {}
@@ -571,26 +576,33 @@ def convert_gsf_to_ossie(
attr_owner: dict[str, tuple[str, dict[str, Any]]] = {}
for term in terms:
- term_id = _required_id(term, "GSF term")
+ term_id = _required_id(term, "Auto Ontology term")
if term_id in term_by_id:
- raise GSFConversionError(f"Duplicate GSF term id {term_id!r}")
+ raise AutoOntologyConversionError(
+ f"Duplicate Auto Ontology term id {term_id!r}"
+ )
represents = term.get("represents") or []
if len(represents) != 1:
- raise GSFConversionError(
- f"Converter supports only GSF terms that represent exactly one "
- f"table; term {term.get('name')!r} represents {len(represents)}"
+ raise AutoOntologyConversionError(
+ f"Converter supports only Auto Ontology terms that represent exactly "
+ f"one table; term {term.get('name')!r} represents {len(represents)}"
)
table_id = str(represents[0])
table = catalog["tables"].get(table_id)
if table is None:
- raise GSFConversionError(
- f"GSF term {term.get('name')!r} represents unknown table {table_id!r}"
+ raise AutoOntologyConversionError(
+ f"Auto Ontology term {term.get('name')!r} represents unknown "
+ f"table {table_id!r}"
)
name = str(term.get("name") or "")
if not name:
- raise GSFConversionError("Every GSF term needs a non-empty name")
+ raise AutoOntologyConversionError(
+ "Every Auto Ontology term needs a non-empty name"
+ )
if name in term_id_by_name:
- raise GSFConversionError(f"Duplicate GSF term name {name!r}")
+ raise AutoOntologyConversionError(
+ f"Duplicate Auto Ontology term name {name!r}"
+ )
term_id_by_name[name] = term_id
datasets_by_table[table_id].append(name)
term_by_id[term_id] = term
@@ -612,20 +624,21 @@ def convert_gsf_to_ossie(
for attribute in term.get("columns_attributes") or []:
if not isinstance(attribute, dict) or not attribute.get("id"):
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Term {name!r} contains a column attribute without an id"
)
column_id = str(attribute.get("column_id") or "")
column = catalog["columns"].get(column_id)
if column is None:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Column attribute {attribute.get('name')!r} references "
f"unknown column {column_id!r}"
)
field_name = str(attribute.get("name") or column["name"])
if field_name in field_names_by_term[term_id]:
- raise GSFConversionError(
- f"Duplicate field name {field_name!r} in GSF term {name!r}"
+ raise AutoOntologyConversionError(
+ f"Duplicate field name {field_name!r} in Auto Ontology "
+ f"term {name!r}"
)
field_names_by_term[term_id].add(field_name)
term_columns[name].add(str(column["name"]))
@@ -645,27 +658,27 @@ def convert_gsf_to_ossie(
for source_group in _SQL_GROUPS:
for attribute in semantic["sql_attributes"][source_group]:
- attribute_id = _required_id(attribute, "GSF SQL attribute")
+ attribute_id = _required_id(attribute, "Auto Ontology SQL attribute")
term_id = str(attribute.get("term_id") or "")
if term_id not in term_by_id:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"SQL attribute {attribute.get('name')!r} references "
f"unknown term {term_id!r}"
)
sql = str(attribute.get("sql") or "")
name = str(attribute.get("name") or "")
if not name or not sql:
- raise GSFConversionError(
- "Every GSF SQL attribute requires non-empty name and sql"
+ raise AutoOntologyConversionError(
+ "Every Auto Ontology SQL attribute requires non-empty name and sql"
)
term_name = str(term_by_id[term_id].get("name") or "")
if name in field_names_by_term[term_id]:
- raise GSFConversionError(
- f"Duplicate field name {name!r} in GSF term {term_name!r}"
+ raise AutoOntologyConversionError(
+ f"Duplicate field name {name!r} in Auto Ontology term {term_name!r}"
)
field_names_by_term[term_id].add(name)
represented_table_id = str(term_by_id[term_id]["represents"][0])
- sql_databases = _validate_gsf_sql_databases(
+ sql_databases = _validate_auto_ontology_sql_databases(
f"SQL attribute {name!r}",
sql,
attribute.get("sql_column_is") or [],
@@ -680,7 +693,7 @@ def convert_gsf_to_ossie(
_ossie_dialect(dialects, sql_databases),
),
"custom_extensions": [
- _gsf_extension(
+ _auto_ontology_extension(
{
"entity": "sql_attribute",
"id": attribute_id,
@@ -704,14 +717,14 @@ def convert_gsf_to_ossie(
metrics: list[dict[str, Any]] = []
for analysis in semantic.get("custom_analyses") or []:
- analysis_id = _required_id(analysis, "GSF custom analysis")
+ analysis_id = _required_id(analysis, "Auto Ontology custom analysis")
sql = str(analysis.get("sql") or "")
name = str(analysis.get("name") or "")
if not name or not sql:
- raise GSFConversionError(
- "Every GSF custom analysis requires non-empty name and sql"
+ raise AutoOntologyConversionError(
+ "Every Auto Ontology custom analysis requires non-empty name and sql"
)
- sql_databases = _validate_gsf_sql_databases(
+ sql_databases = _validate_auto_ontology_sql_databases(
f"Custom analysis {name!r}",
sql,
analysis.get("sql_column_is") or [],
@@ -725,7 +738,7 @@ def convert_gsf_to_ossie(
_ossie_dialect(dialects, sql_databases),
),
"custom_extensions": [
- _gsf_extension(
+ _auto_ontology_extension(
{
"entity": "custom_analysis",
"id": analysis_id,
@@ -740,7 +753,7 @@ def convert_gsf_to_ossie(
metric["description"] = str(analysis["description"])
metrics.append(metric)
- relationships = _relationships_from_gsf(
+ relationships = _relationships_from_auto_ontology(
root,
catalog,
datasets_by_table,
@@ -755,13 +768,13 @@ def convert_gsf_to_ossie(
inferred_name = (
model_name
or (next(iter(database_names)) if len(database_names) == 1 else None)
- or "gsf_model"
+ or "auto_ontology_model"
)
semantic_model: dict[str, Any] = {
"name": inferred_name,
"datasets": datasets,
"custom_extensions": [
- _gsf_extension(
+ _auto_ontology_extension(
{
"model_name": inferred_name,
"native_document": root,
@@ -840,7 +853,7 @@ def _build_catalog(
declared_types: dict[str, str] = {}
for _, context in contexts:
for field, column_name in context["simple_fields"]:
- sql_type = _gsf_column_type(field.get("datatype"))
+ sql_type = _auto_ontology_column_type(field.get("datatype"))
if sql_type:
declared_types.setdefault(column_name, sql_type)
@@ -1050,9 +1063,10 @@ def _read_native_catalog(root: dict[str, Any]) -> dict[str, Any]:
"""
try:
return _read_catalog(root)
- except GSFConversionError as exc:
- raise GSFConversionError(
- f"Malformed {NVIDIA_GSF_VENDOR} 'native_document' extension: {exc}"
+ except AutoOntologyConversionError as exc:
+ raise AutoOntologyConversionError(
+ f"Malformed {NVIDIA_AUTO_ONTOLOGY_VENDOR} 'native_document' "
+ f"extension: {exc}"
) from exc
@@ -1062,26 +1076,31 @@ def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
database_ids: set[str] = set()
schema_ids: set[str] = set()
for database in root["data_layer"]["databases"]:
- database_id = _required_id(database, "GSF database")
+ database_id = _required_id(database, "Auto Ontology database")
if database_id in database_ids:
- raise GSFConversionError(f"Duplicate GSF database id {database_id!r}")
+ raise AutoOntologyConversionError(
+ f"Duplicate Auto Ontology database id {database_id!r}"
+ )
database_ids.add(database_id)
for schema in database.get("schemas") or []:
- schema_id = _required_id(schema, "GSF schema")
+ schema_id = _required_id(schema, "Auto Ontology schema")
if schema_id in schema_ids:
- raise GSFConversionError(f"Duplicate GSF schema id {schema_id!r}")
+ raise AutoOntologyConversionError(
+ f"Duplicate Auto Ontology schema id {schema_id!r}"
+ )
schema_ids.add(schema_id)
database_name = str(schema.get("database_name") or "")
schema_name = str(schema.get("name") or "")
if not database_name:
- raise GSFConversionError(
- f"GSF schema {schema_name!r} requires database_name"
+ raise AutoOntologyConversionError(
+ f"Auto Ontology schema {schema_name!r} requires database_name"
)
for table in schema.get("tables") or []:
table_id = str(table.get("id") or "")
if not table_id or table_id in tables:
- raise GSFConversionError(
- f"Every GSF table needs a globally unique id; got {table_id!r}"
+ raise AutoOntologyConversionError(
+ "Every Auto Ontology table needs a globally unique id; "
+ f"got {table_id!r}"
)
table_name = str(table.get("name") or "")
tables[table_id] = {
@@ -1091,8 +1110,8 @@ def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
for column in table.get("columns") or []:
column_id = str(column.get("id") or "")
if not column_id or column_id in columns:
- raise GSFConversionError(
- "Every GSF column needs a globally unique id; "
+ raise AutoOntologyConversionError(
+ "Every Auto Ontology column needs a globally unique id; "
f"got {column_id!r}"
)
columns[column_id] = {
@@ -1103,7 +1122,7 @@ def _read_catalog(root: dict[str, Any]) -> dict[str, Any]:
return {"tables": tables, "columns": columns}
-def _validate_gsf_sql_databases(
+def _validate_auto_ontology_sql_databases(
context: str,
sql: str,
sql_column_ids: Iterable[Any],
@@ -1136,14 +1155,14 @@ def _validate_gsf_sql_databases(
]
databases.update(str(table["source"][0]) for table in matches)
if len(databases) > 1:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
- "the GSF importer validates each SQL object against one database"
+ "the Auto Ontology importer validates each SQL object against one database"
)
return databases
-def _relationships_from_gsf(
+def _relationships_from_auto_ontology(
root: dict[str, Any],
catalog: Mapping[str, Any],
datasets_by_table: Mapping[str, list[str]],
@@ -1279,7 +1298,7 @@ def _relationship_dataset(
]
if len(matching) == 1:
return matching[0]
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Cannot map relationship on table {table_id!r} and columns "
f"{', '.join(columns)} to exactly one represented term; candidates: "
f"{', '.join(candidates) or 'none'}"
@@ -1323,45 +1342,53 @@ def _join_column_name(
def _required_id(item: Any, context: str) -> str:
if not isinstance(item, dict) or not item.get("id"):
- raise GSFConversionError(f"{context} requires a non-empty id")
+ raise AutoOntologyConversionError(f"{context} requires a non-empty id")
return str(item["id"])
-def _parse_gsf(value: str) -> dict[str, Any]:
- root = _load_yaml(value, "GSF")
+def _parse_auto_ontology(value: str) -> dict[str, Any]:
+ root = _load_yaml(value, "Auto Ontology")
expected = {"data_layer", "semantic_layer", "zones"}
unknown = sorted(set(root) - expected)
if unknown:
- raise GSFConversionError(
- "Unsupported GSF root properties: " + ", ".join(unknown)
+ raise AutoOntologyConversionError(
+ "Unsupported Auto Ontology root properties: " + ", ".join(unknown)
)
root.setdefault("data_layer", {})
root.setdefault("semantic_layer", {})
root.setdefault("zones", [])
for key in ("data_layer", "semantic_layer"):
if not isinstance(root[key], dict):
- raise GSFConversionError(f"GSF {key!r} must be a mapping")
+ raise AutoOntologyConversionError(
+ f"Auto Ontology {key!r} must be a mapping"
+ )
if not isinstance(root["zones"], list):
- raise GSFConversionError("GSF 'zones' must be a list")
+ raise AutoOntologyConversionError("Auto Ontology 'zones' must be a list")
data_layer = root["data_layer"]
semantic_layer = root["semantic_layer"]
for key in ("databases", "foreign_keys", "joins"):
data_layer.setdefault(key, [])
if not isinstance(data_layer.get(key), list):
- raise GSFConversionError(f"GSF data_layer.{key} must be a list")
+ raise AutoOntologyConversionError(
+ f"Auto Ontology data_layer.{key} must be a list"
+ )
for key in ("terms", "semantic_fks", "custom_analyses"):
semantic_layer.setdefault(key, [])
if not isinstance(semantic_layer.get(key), list):
- raise GSFConversionError(f"GSF semantic_layer.{key} must be a list")
+ raise AutoOntologyConversionError(
+ f"Auto Ontology semantic_layer.{key} must be a list"
+ )
semantic_layer.setdefault("sql_attributes", {})
sql_attributes = semantic_layer["sql_attributes"]
if not isinstance(sql_attributes, dict):
- raise GSFConversionError("GSF semantic_layer.sql_attributes must be a mapping")
+ raise AutoOntologyConversionError(
+ "Auto Ontology semantic_layer.sql_attributes must be a mapping"
+ )
for key in _SQL_GROUPS:
sql_attributes.setdefault(key, [])
if not isinstance(sql_attributes.get(key), list):
- raise GSFConversionError(
- f"GSF semantic_layer.sql_attributes.{key} must be a list"
+ raise AutoOntologyConversionError(
+ f"Auto Ontology semantic_layer.sql_attributes.{key} must be a list"
)
return root
@@ -1369,7 +1396,7 @@ def _parse_gsf(value: str) -> dict[str, Any]:
def _parse_ossie(value: str) -> dict[str, Any]:
root = _load_yaml(value, "Ossie")
if "semantic_model" in root:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
"Ossie model properties must be at the root; semantic_model wrappers are not supported"
)
unknown = sorted(set(root) - {
@@ -1377,12 +1404,12 @@ def _parse_ossie(value: str) -> dict[str, Any]:
"datasets", "relationships", "metrics", "custom_extensions",
})
if unknown:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
"Unsupported Ossie root properties: " + ", ".join(unknown)
)
_check_ossie_version(root.get("version"))
if not isinstance(root.get("name"), str) or not root["name"]:
- raise GSFConversionError("Ossie semantic model requires a name")
+ raise AutoOntologyConversionError("Ossie semantic model requires a name")
return root
@@ -1391,7 +1418,7 @@ def _check_ossie_version(value: Any) -> None:
series = re.match(r"^\s*(\d+)\.(\d+)", str(value or ""))
if not series or (int(series.group(1)), int(series.group(2))) != OSSIE_SERIES:
expected = ".".join(str(part) for part in OSSIE_SERIES)
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Unsupported Ossie version {value!r}; expected {expected}.x"
)
@@ -1400,9 +1427,11 @@ def _load_yaml(value: str, label: str) -> dict[str, Any]:
try:
root = yaml.safe_load(value)
except yaml.YAMLError as exc:
- raise GSFConversionError(f"Invalid {label} YAML: {exc}") from exc
+ raise AutoOntologyConversionError(f"Invalid {label} YAML: {exc}") from exc
if not isinstance(root, dict):
- raise GSFConversionError(f"Invalid {label} YAML: expected a root mapping")
+ raise AutoOntologyConversionError(
+ f"Invalid {label} YAML: expected a root mapping"
+ )
return root
@@ -1411,17 +1440,17 @@ def _validate_relationship(
datasets: Mapping[str, Any],
) -> dict[str, Any]:
if not isinstance(relationship, dict) or not relationship.get("name"):
- raise GSFConversionError("Every Ossie relationship needs a name")
+ raise AutoOntologyConversionError("Every Ossie relationship needs a name")
from_name = relationship.get("from")
to_name = relationship.get("to")
if from_name not in datasets or to_name not in datasets:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Relationship {relationship['name']!r} references an unknown dataset"
)
from_columns = relationship.get("from_columns") or []
to_columns = relationship.get("to_columns") or []
if not from_columns or len(from_columns) != len(to_columns):
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Relationship {relationship['name']!r} must have equal, "
"non-empty column lists"
)
@@ -1430,10 +1459,12 @@ def _validate_relationship(
def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
if not isinstance(value, dict):
- raise GSFConversionError(f"{name!r} has no valid expression")
+ raise AutoOntologyConversionError(f"{name!r} has no valid expression")
dialects = value.get("dialects")
if not isinstance(dialects, list) or not dialects:
- raise GSFConversionError(f"{name!r} requires at least one expression dialect")
+ raise AutoOntologyConversionError(
+ f"{name!r} requires at least one expression dialect"
+ )
result = [
{
"dialect": str(item["dialect"]),
@@ -1445,7 +1476,7 @@ def _normalize_expressions(value: Any, name: str) -> list[dict[str, str]]:
and item.get("expression") is not None
]
if not result:
- raise GSFConversionError(f"{name!r} has no usable expression dialect")
+ raise AutoOntologyConversionError(f"{name!r} has no usable expression dialect")
return result
@@ -1455,7 +1486,7 @@ def _pick_expression(expressions: list[dict[str, str]], name: str) -> str:
return expression["expression"]
if expressions:
return expressions[0]["expression"]
- raise GSFConversionError(f"{name!r} has no usable expression")
+ raise AutoOntologyConversionError(f"{name!r} has no usable expression")
def _simple_source_column(
@@ -1478,7 +1509,7 @@ def _field_table_refs(
owner: str,
datasets: Mapping[str, Any],
) -> list[str]:
- extension = _gsf_extension_data(field)
+ extension = _auto_ontology_extension_data(field)
extension_refs = extension.get("table_refs")
expression_unchanged = _expression_matches(
expression, extension.get("ossie_expression")
@@ -1515,7 +1546,7 @@ def _metric_table_refs(
expression: str,
datasets: Mapping[str, Any],
) -> list[str]:
- extension = _gsf_extension_data(metric)
+ extension = _auto_ontology_extension_data(metric)
extension_refs = extension.get("table_refs")
expression_unchanged = _expression_matches(
expression, extension.get("ossie_expression")
@@ -1563,9 +1594,9 @@ def _metric_table_refs(
sql=reference_sql,
)
return refs
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Metric {metric.get('name')!r} does not identify a source dataset; "
- "qualify a referenced column or add NVIDIA_GSF table_refs"
+ "qualify a referenced column or add NVIDIA_AUTO_ONTOLOGY table_refs"
)
@@ -1576,8 +1607,9 @@ def _validate_refs(
) -> None:
unknown = [ref for ref in refs if ref not in datasets]
if unknown:
- raise GSFConversionError(
- f"{name!r} has unknown NVIDIA_GSF table_refs: {', '.join(unknown)}"
+ raise AutoOntologyConversionError(
+ f"{name!r} has unknown NVIDIA_AUTO_ONTOLOGY table_refs: "
+ f"{', '.join(unknown)}"
)
@@ -1593,9 +1625,9 @@ def _validate_single_database_refs(
}
databases.update(table.catalog for table in _sql_tables(sql) if table.catalog)
if len(databases) > 1:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"{context} spans multiple databases ({', '.join(sorted(databases))}); "
- "the GSF importer validates each SQL object against one database"
+ "the Auto Ontology importer validates each SQL object against one database"
)
@@ -1650,8 +1682,9 @@ def _collect_expression_columns(
source_key = _source_key(context["source"])
known_table = source_key in preserved["tables"]
if known_table and (*source_key, column.name) not in preserved["columns"]:
- # A GSF-sourced catalog is authoritative: an identifier that is not
- # already a column of the table is SQL syntax, not physical data.
+ # A catalog sourced from Auto Ontology is authoritative: an identifier
+ # that is not already a column of the table is SQL syntax, not physical
+ # data.
continue
_add_catalog_column(context, column.name)
@@ -1739,7 +1772,7 @@ def _argument_count(func: exp.Expression) -> int:
def _parse_sql(sql: str) -> exp.Expression | None:
"""Parse *sql*, trying each candidate dialect, or return ``None``.
- Preserved GSF SQL carries whatever dialect its connection reported, so a
+ Preserved Auto Ontology SQL carries whatever dialect its connection reported, so a
single parser is not enough. SQL that no candidate can parse is treated as
opaque: it is still carried through verbatim, and only the parse-derived
enrichment (column and table discovery) is skipped.
@@ -1776,7 +1809,9 @@ def _wrap_expression(
if stripped.upper().startswith(("SELECT ", "SELECT\n", "WITH ", "WITH\n")):
return stripped
if not refs:
- raise GSFConversionError(f"Cannot determine a source table for {name!r}")
+ raise AutoOntologyConversionError(
+ f"Cannot determine a source table for {name!r}"
+ )
anchor_name = refs[0]
from_sql = (
f"{_qualified_table(datasets[anchor_name]['source'])} "
@@ -1815,7 +1850,7 @@ def _wrap_expression(
break
if not matched:
missing = min(remaining)
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"{name!r} references disconnected dataset {missing!r}; "
"declare a relationship connecting all referenced datasets"
)
@@ -1858,7 +1893,7 @@ def _ossie_expression(expression: str, dialect: str = "ANSI_SQL") -> dict[str, A
def _ossie_datatype(sql_type: Any) -> str | None:
- """Map a GSF column's physical type onto Ossie's logical vocabulary.
+ """Map an Auto Ontology column's physical type onto Ossie's logical vocabulary.
A type Ossie cannot name becomes ``Opaque``, as the spec prescribes for a
known type outside the portable vocabulary. An absent type stays unset
@@ -1890,7 +1925,7 @@ def _split_sql_type(sql_type: Any) -> tuple[str, int | None]:
return " ".join(re.sub(r"\([^)]*\)", " ", text).split()), scale
-def _gsf_column_type(datatype: Any) -> str:
+def _auto_ontology_column_type(datatype: Any) -> str:
"""Map an Ossie logical datatype onto a physical type for a new column."""
return _SQL_TYPE_BY_OSSIE_DATATYPE.get(str(datatype or ""), "")
@@ -1909,14 +1944,16 @@ def _dialects_by_database(root: Mapping[str, Any]) -> dict[str, str]:
def _ossie_dialect(dialects: Mapping[str, str], databases: Iterable[str]) -> str:
- """Map a GSF connection dialect onto the Ossie dialect enum.
+ """Map an Auto Ontology connection dialect onto the Ossie dialect enum.
Ossie names only a few dialects, so anything else stays ANSI_SQL rather
than being labelled inaccurately.
"""
names = {str(dialects.get(database, "")).lower() for database in databases}
labels = {
- _GSF_TO_OSSIE_DIALECT[name] for name in names if name in _GSF_TO_OSSIE_DIALECT
+ _AUTO_ONTOLOGY_TO_OSSIE_DIALECT[name]
+ for name in names
+ if name in _AUTO_ONTOLOGY_TO_OSSIE_DIALECT
}
return labels.pop() if len(labels) == 1 else "ANSI_SQL"
@@ -1930,7 +1967,7 @@ def _parse_source(
schema = source.get("schema")
table = source.get("table")
if not table:
- raise GSFConversionError("Source mapping requires 'table'")
+ raise AutoOntologyConversionError("Source mapping requires 'table'")
return {
"database": str(database) if database else None,
"schema": str(schema) if schema else None,
@@ -1938,9 +1975,11 @@ def _parse_source(
}
value = str(source or "").strip()
if not value:
- raise GSFConversionError("Every dataset needs a source")
+ raise AutoOntologyConversionError("Every dataset needs a source")
if value.upper().startswith(("SELECT ", "SELECT\n", "WITH ", "WITH\n")):
- raise GSFConversionError("GSF terms must identify physical tables")
+ raise AutoOntologyConversionError(
+ "Auto Ontology terms must identify physical tables"
+ )
parts = _split_identifier(value)
if len(parts) == 3:
database, schema, table = parts
@@ -1949,7 +1988,7 @@ def _parse_source(
elif len(parts) == 1:
database, schema, table = default_database, None, parts[0]
else:
- raise GSFConversionError(
+ raise AutoOntologyConversionError(
f"Source {value!r} must be table, schema.table, or database.schema.table"
)
return {"database": database, "schema": schema, "table": table}
@@ -2083,11 +2122,11 @@ def _stable_id(kind: str, *parts: str) -> str:
return str(uuid5(_ID_NAMESPACE, "/".join((kind, *map(str, parts)))))
-def _gsf_extension_data(item: Mapping[str, Any]) -> dict[str, Any]:
+def _auto_ontology_extension_data(item: Mapping[str, Any]) -> dict[str, Any]:
for extension in item.get("custom_extensions") or []:
if not isinstance(extension, dict):
continue
- if extension.get("vendor_name") not in GSF_VENDOR_ALIASES:
+ if extension.get("vendor_name") not in AUTO_ONTOLOGY_VENDOR_ALIASES:
continue
try:
data = json.loads(str(extension.get("data") or "{}"))
@@ -2099,13 +2138,13 @@ def _gsf_extension_data(item: Mapping[str, Any]) -> dict[str, Any]:
def _native_snapshot(model: Mapping[str, Any]) -> dict[str, Any] | None:
- snapshot = _gsf_extension_data(model).get("native_document")
+ snapshot = _auto_ontology_extension_data(model).get("native_document")
return snapshot if isinstance(snapshot, dict) else None
-def _gsf_extension(data: Mapping[str, Any]) -> dict[str, str]:
+def _auto_ontology_extension(data: Mapping[str, Any]) -> dict[str, str]:
return {
- "vendor_name": NVIDIA_GSF_VENDOR,
+ "vendor_name": NVIDIA_AUTO_ONTOLOGY_VENDOR,
"data": json.dumps(data, separators=(",", ":"), sort_keys=True),
}
@@ -2121,12 +2160,14 @@ def _dump_yaml(value: dict[str, Any]) -> str:
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
- description="Convert Apache Ossie YAML and native NVIDIA GSF model YAML"
+ description=(
+ "Convert Apache Ossie YAML and native NVIDIA Auto Ontology model YAML"
+ )
)
subparsers = parser.add_subparsers(dest="command", required=True)
export_parser = subparsers.add_parser(
"export",
- help="Convert Ossie YAML to a native GSF model document",
+ help="Convert Ossie YAML to a native Auto Ontology model document",
)
export_parser.add_argument("-i", "--input", type=Path, required=True)
export_parser.add_argument("-o", "--output", type=Path)
@@ -2136,7 +2177,7 @@ def _build_parser() -> argparse.ArgumentParser:
)
import_parser = subparsers.add_parser(
"import",
- help="Convert a native GSF model document to Ossie YAML",
+ help="Convert a native Auto Ontology model document to Ossie YAML",
)
import_parser.add_argument("-i", "--input", type=Path, required=True)
import_parser.add_argument("-o", "--output", type=Path)
@@ -2152,17 +2193,17 @@ def main(argv: list[str] | None = None) -> None:
try:
source = args.input.read_text(encoding="utf-8")
if args.command == "export":
- output = convert_ossie_to_gsf(
+ output = convert_ossie_to_auto_ontology(
source,
database_name=args.database_name,
)
else:
- output = convert_gsf_to_ossie(source, model_name=args.name)
+ output = convert_auto_ontology_to_ossie(source, model_name=args.name)
if args.output is None:
print(output, end="")
else:
args.output.write_text(output, encoding="utf-8")
- except (GSFConversionError, OSError, UnicodeError) as exc:
+ except (AutoOntologyConversionError, OSError, UnicodeError) as exc:
print(f"Error: {exc}", file=sys.stderr)
raise SystemExit(1) from exc
diff --git a/converters/nvidia/tests/fixtures/sales.gsf.yaml b/converters/nvidia/tests/fixtures/sales.auto_ontology.yaml
similarity index 100%
rename from converters/nvidia/tests/fixtures/sales.gsf.yaml
rename to converters/nvidia/tests/fixtures/sales.auto_ontology.yaml
diff --git a/converters/nvidia/tests/test_converter.py b/converters/nvidia/tests/test_converter.py
index 21a8a2cf..2123dad7 100644
--- a/converters/nvidia/tests/test_converter.py
+++ b/converters/nvidia/tests/test_converter.py
@@ -15,7 +15,7 @@
# specific language governing permissions and limitations
# under the License.
-"""Tests for Apache Ossie ↔ native NVIDIA GSF conversion."""
+"""Tests for Apache Ossie ↔ native NVIDIA Auto Ontology conversion."""
from __future__ import annotations
@@ -29,13 +29,13 @@ from typing import Any
import pytest
import yaml
-from ossie_nvidia_gsf.converter import (
- GSFConversionError,
- convert_gsf_to_ossie,
- convert_ossie_to_gsf,
+from ossie_nvidia_auto_ontology.converter import (
+ AutoOntologyConversionError,
+ convert_auto_ontology_to_ossie,
+ convert_ossie_to_auto_ontology,
main,
)
-from ossie_nvidia_gsf.native_converter import (
+from ossie_nvidia_auto_ontology.native_converter import (
_SQL_TYPE_BY_OSSIE_DATATYPE,
_index_native_document,
_ossie_datatype,
@@ -54,15 +54,15 @@ def _ossie_yaml() -> str:
return (FIXTURES / "sales.ossie.yaml").read_text(encoding="utf-8")
-def _gsf_yaml() -> str:
- return (FIXTURES / "sales.gsf.yaml").read_text(encoding="utf-8")
+def _auto_ontology_yaml() -> str:
+ return (FIXTURES / "sales.auto_ontology.yaml").read_text(encoding="utf-8")
def _native_extension(item: dict[str, Any]) -> dict[str, Any]:
extension = next(
value
for value in item.get("custom_extensions") or []
- if value["vendor_name"] == "NVIDIA_GSF"
+ if value["vendor_name"] == "NVIDIA_AUTO_ONTOLOGY"
)
return json.loads(extension["data"])
@@ -90,8 +90,8 @@ def _ids(value: Any) -> set[str]:
def test_checked_in_fixture_is_exact_native_contract() -> None:
- expected = yaml.safe_load(_gsf_yaml())
- actual = yaml.safe_load(convert_ossie_to_gsf(_ossie_yaml()))
+ expected = yaml.safe_load(_auto_ontology_yaml())
+ actual = yaml.safe_load(convert_ossie_to_auto_ontology(_ossie_yaml()))
assert actual == expected
assert set(actual) == {"data_layer", "semantic_layer", "zones"}
@@ -108,8 +108,8 @@ def test_checked_in_fixture_is_exact_native_contract() -> None:
def test_ossie_ids_are_deterministic_and_references_resolve() -> None:
- first = yaml.safe_load(convert_ossie_to_gsf(_ossie_yaml()))
- second = yaml.safe_load(convert_ossie_to_gsf(_ossie_yaml()))
+ first = yaml.safe_load(convert_ossie_to_auto_ontology(_ossie_yaml()))
+ second = yaml.safe_load(convert_ossie_to_auto_ontology(_ossie_yaml()))
assert _ids(first) == _ids(second)
assert first == second
@@ -129,7 +129,9 @@ def test_ossie_ids_are_deterministic_and_references_resolve() -> None:
def test_generated_ossie_passes_official_validation(tmp_path: Path) -> None:
output_path = tmp_path / "converted.ossie.yaml"
- output_path.write_text(convert_gsf_to_ossie(_gsf_yaml()), encoding="utf-8")
+ output_path.write_text(
+ convert_auto_ontology_to_ossie(_auto_ontology_yaml()), encoding="utf-8"
+ )
result = subprocess.run(
[sys.executable, str(VALIDATOR), str(output_path)],
@@ -143,7 +145,9 @@ def test_generated_ossie_passes_official_validation(tmp_path: Path) -> None:
def test_round_trip_preserves_ossie_semantics_and_global_metrics() -> None:
- result = yaml.safe_load(convert_gsf_to_ossie(convert_ossie_to_gsf(_ossie_yaml())))
+ result = yaml.safe_load(
+ convert_auto_ontology_to_ossie(convert_ossie_to_auto_ontology(_ossie_yaml()))
+ )
model = result
datasets = {dataset["name"]: dataset for dataset in model["datasets"]}
order_fields = {field["name"]: field for field in datasets["orders"]["fields"]}
@@ -166,7 +170,7 @@ def test_round_trip_preserves_ossie_semantics_and_global_metrics() -> None:
def test_edited_ossie_expressions_replace_preserved_native_sql() -> None:
- ossie = yaml.safe_load(convert_gsf_to_ossie(_gsf_yaml()))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(_auto_ontology_yaml()))
model = ossie
orders = next(
dataset for dataset in model["datasets"] if dataset["name"] == "orders"
@@ -180,7 +184,7 @@ def test_edited_ossie_expressions_replace_preserved_native_sql() -> None:
)
regenerated = yaml.safe_load(
- convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
)
sql_attribute = regenerated["semantic_layer"]["sql_attributes"]["manual"][0]
analysis = regenerated["semantic_layer"]["custom_analyses"][0]
@@ -192,7 +196,7 @@ def test_edited_ossie_expressions_replace_preserved_native_sql() -> None:
def test_native_round_trip_preserves_ids_catalog_sql_source_and_zones() -> None:
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
database = native["data_layer"]["databases"][0]
database["dialect"] = "snowflake"
first_column = database["schemas"][0]["tables"][0]["columns"][0]
@@ -223,11 +227,11 @@ def test_native_round_trip_preserves_ids_catalog_sql_source_and_zones() -> None:
native["semantic_layer"]["sql_attributes"]["table"] = manual
native["semantic_layer"]["sql_attributes"]["manual"] = []
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
assert _native_extension(ossie)["native_document"] == native
restored = yaml.safe_load(
- convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
)
restored_database = restored["data_layer"]["databases"][0]
restored_first_column = restored_database["schemas"][0]["tables"][0]["columns"][0]
@@ -249,12 +253,12 @@ def test_native_round_trip_preserves_ids_catalog_sql_source_and_zones() -> None:
def test_relationship_edits_replace_preserved_native_records() -> None:
- ossie = yaml.safe_load(convert_gsf_to_ossie(_gsf_yaml()))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(_auto_ontology_yaml()))
relationship = ossie["relationships"][0]
relationship["from_columns"] = ["order_id"]
regenerated = yaml.safe_load(
- convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
)
assert regenerated["data_layer"]["joins"][0]["join_columns"] == [
@@ -274,11 +278,11 @@ def test_relationship_edits_replace_preserved_native_records() -> None:
def test_relationship_deletion_removes_preserved_native_records() -> None:
- ossie = yaml.safe_load(convert_gsf_to_ossie(_gsf_yaml()))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(_auto_ontology_yaml()))
ossie.pop("relationships")
regenerated = yaml.safe_load(
- convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
)
assert regenerated["data_layer"]["joins"] == []
@@ -287,7 +291,7 @@ def test_relationship_deletion_removes_preserved_native_records() -> None:
def test_relationship_reconciliation_preserves_catalog_only_records() -> None:
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
schema = native["data_layer"]["databases"][0]["schemas"][0]
orders = next(table for table in schema["tables"] if table["name"] == "orders")
order_id = next(
@@ -325,10 +329,10 @@ def test_relationship_reconciliation_preserves_catalog_only_records() -> None:
native["data_layer"]["joins"].append(audit_join)
native["data_layer"]["foreign_keys"].append(audit_fk)
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
ossie.pop("relationships")
regenerated = yaml.safe_load(
- convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
)
assert regenerated["data_layer"]["joins"] == [audit_join]
@@ -342,18 +346,18 @@ def test_multiple_databases_are_supported_and_name_falls_back() -> None:
model["relationships"] = []
model["metrics"] = []
- native_yaml = convert_ossie_to_gsf(yaml.safe_dump(ossie))
+ native_yaml = convert_ossie_to_auto_ontology(yaml.safe_dump(ossie))
native = yaml.safe_load(native_yaml)
database_names = {
schema["database_name"]
for database in native["data_layer"]["databases"]
for schema in database["schemas"]
}
- restored = yaml.safe_load(convert_gsf_to_ossie(native_yaml))
+ restored = yaml.safe_load(convert_auto_ontology_to_ossie(native_yaml))
assert database_names == {"analytics", "crm"}
assert len(native["data_layer"]["databases"]) == 2
- assert restored["name"] == "gsf_model"
+ assert restored["name"] == "auto_ontology_model"
def test_shared_physical_source_uses_one_catalog_table_and_valid_ossie(
@@ -376,7 +380,7 @@ def test_shared_physical_source_uses_one_catalog_table_and_valid_ossie(
}
)
- native_yaml = convert_ossie_to_gsf(yaml.safe_dump(ossie, sort_keys=False))
+ native_yaml = convert_ossie_to_auto_ontology(yaml.safe_dump(ossie, sort_keys=False))
native = yaml.safe_load(native_yaml)
order_tables = [
table
@@ -398,7 +402,7 @@ def test_shared_physical_source_uses_one_catalog_table_and_valid_ossie(
}
assert represented_ids["orders"] == represented_ids["order_amounts"]
- restored = yaml.safe_load(convert_gsf_to_ossie(native_yaml))
+ restored = yaml.safe_load(convert_auto_ontology_to_ossie(native_yaml))
assert {
dataset["name"] for dataset in restored["datasets"]
} >= {"orders", "order_amounts"}
@@ -417,8 +421,8 @@ def test_cross_database_ossie_metric_is_rejected() -> None:
ossie = yaml.safe_load(_ossie_yaml())
ossie["datasets"][1]["source"] = "crm.public.customers"
- with pytest.raises(GSFConversionError, match="spans multiple databases"):
- convert_ossie_to_gsf(yaml.safe_dump(ossie))
+ with pytest.raises(AutoOntologyConversionError, match="spans multiple databases"):
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie))
def test_cross_database_full_query_field_is_rejected() -> None:
@@ -444,18 +448,20 @@ def test_cross_database_full_query_field_is_rejected() -> None:
}
)
- with pytest.raises(GSFConversionError, match="SQL attribute.*multiple databases"):
- convert_ossie_to_gsf(yaml.safe_dump(ossie))
+ with pytest.raises(
+ AutoOntologyConversionError, match="SQL attribute.*multiple databases"
+ ):
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie))
@pytest.mark.parametrize("kind", ["sql_attribute", "custom_analysis"])
-def test_cross_database_gsf_sql_objects_are_rejected(kind: str) -> None:
+def test_cross_database_auto_ontology_sql_objects_are_rejected(kind: str) -> None:
ossie = yaml.safe_load(_ossie_yaml())
model = ossie
model["datasets"][1]["source"] = "crm.public.customers"
model["metrics"] = []
model["relationships"] = []
- native = yaml.safe_load(convert_ossie_to_gsf(yaml.safe_dump(ossie)))
+ native = yaml.safe_load(convert_ossie_to_auto_ontology(yaml.safe_dump(ossie)))
terms = {term["name"]: term for term in native["semantic_layer"]["terms"]}
columns = {
(schema["database_name"], table["name"], column["name"]): column["id"]
@@ -496,12 +502,12 @@ def test_cross_database_gsf_sql_objects_are_rejected(kind: str) -> None:
}
)
- with pytest.raises(GSFConversionError, match="spans multiple databases"):
- convert_gsf_to_ossie(yaml.safe_dump(native))
+ with pytest.raises(AutoOntologyConversionError, match="spans multiple databases"):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(native))
def test_relationships_emit_join_physical_fk_and_semantic_fk() -> None:
- native = yaml.safe_load(convert_ossie_to_gsf(_ossie_yaml()))
+ native = yaml.safe_load(convert_ossie_to_auto_ontology(_ossie_yaml()))
assert len(native["data_layer"]["joins"]) == 1
assert native["data_layer"]["joins"][0]["join_columns"] == [
@@ -511,44 +517,48 @@ def test_relationships_emit_join_physical_fk_and_semantic_fk() -> None:
assert len(native["semantic_layer"]["semantic_fks"]) == 1
native["data_layer"]["joins"] = []
- restored = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ restored = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
assert restored["relationships"][0]["from"] == "orders"
assert restored["relationships"][0]["to"] == "customers"
-def test_gsf_requires_one_represented_table_per_term() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_auto_ontology_requires_one_represented_table_per_term() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
term = native["semantic_layer"]["terms"][0]
term["represents"].append(native["semantic_layer"]["terms"][1]["represents"][0])
- with pytest.raises(GSFConversionError, match="exactly one table"):
- convert_gsf_to_ossie(yaml.safe_dump(native))
+ with pytest.raises(AutoOntologyConversionError, match="exactly one table"):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(native))
-def test_duplicate_gsf_term_names_are_rejected() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_duplicate_auto_ontology_term_names_are_rejected() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
native["semantic_layer"]["terms"][1]["name"] = "orders"
- with pytest.raises(GSFConversionError, match="Duplicate GSF term name"):
- convert_gsf_to_ossie(yaml.safe_dump(native))
+ with pytest.raises(
+ AutoOntologyConversionError, match="Duplicate Auto Ontology term name"
+ ):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(native))
-def test_duplicate_gsf_field_names_across_attribute_kinds_are_rejected() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_duplicate_auto_ontology_field_names_across_attribute_kinds_are_rejected() -> (
+ None
+):
+ native = yaml.safe_load(_auto_ontology_yaml())
native["semantic_layer"]["sql_attributes"]["manual"][0]["name"] = "order_id"
- with pytest.raises(GSFConversionError, match="Duplicate field name"):
- convert_gsf_to_ossie(yaml.safe_dump(native))
+ with pytest.raises(AutoOntologyConversionError, match="Duplicate field name"):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(native))
-def test_catalog_only_gsf_has_no_representable_terms() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_catalog_only_auto_ontology_has_no_representable_terms() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
native["semantic_layer"]["terms"] = []
native["semantic_layer"]["sql_attributes"]["manual"] = []
native["semantic_layer"]["custom_analyses"] = []
- with pytest.raises(GSFConversionError, match="no representable terms"):
- convert_gsf_to_ossie(yaml.safe_dump(native))
+ with pytest.raises(AutoOntologyConversionError, match="no representable terms"):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(native))
@pytest.mark.parametrize(
@@ -574,7 +584,7 @@ def test_date_part_keywords_do_not_become_catalog_columns(
}
)
- native = yaml.safe_load(convert_ossie_to_gsf(yaml.safe_dump(ossie)))
+ native = yaml.safe_load(convert_ossie_to_auto_ontology(yaml.safe_dump(ossie)))
orders = next(
table
for database in native["data_layer"]["databases"]
@@ -595,8 +605,8 @@ def test_date_part_keywords_do_not_become_catalog_columns(
assert unit not in {referenced.get(item) for item in attribute["sql_column_is"]}
-def test_gsf_sourced_catalog_is_never_widened_by_sql_identifiers() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_auto_ontology_sourced_catalog_is_never_widened_by_sql_identifiers() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
manual = native["semantic_layer"]["sql_attributes"]["manual"][0]
manual["sql"] = (
"SELECT DATEDIFF(day, order_date, CURRENT_TIMESTAMP()) + not_a_real_column "
@@ -610,8 +620,8 @@ def test_gsf_sourced_catalog_is_never_widened_by_sql_identifiers() -> None:
for column in table["columns"]
}
- ossie = convert_gsf_to_ossie(yaml.safe_dump(native))
- restored = yaml.safe_load(convert_ossie_to_gsf(ossie))
+ ossie = convert_auto_ontology_to_ossie(yaml.safe_dump(native))
+ restored = yaml.safe_load(convert_ossie_to_auto_ontology(ossie))
after_columns = [
column
for database in restored["data_layer"]["databases"]
@@ -629,7 +639,7 @@ def test_any_release_in_the_supported_series_is_accepted(version: str) -> None:
ossie = yaml.safe_load(_ossie_yaml())
ossie["version"] = version
- native = yaml.safe_load(convert_ossie_to_gsf(yaml.safe_dump(ossie)))
+ native = yaml.safe_load(convert_ossie_to_auto_ontology(yaml.safe_dump(ossie)))
assert native["semantic_layer"]["terms"]
@@ -639,36 +649,54 @@ def test_versions_outside_the_supported_series_are_rejected(version: str) -> Non
ossie = yaml.safe_load(_ossie_yaml())
ossie["version"] = version
- with pytest.raises(GSFConversionError, match="Unsupported Ossie version"):
- convert_ossie_to_gsf(yaml.safe_dump(ossie))
+ with pytest.raises(AutoOntologyConversionError, match="Unsupported Ossie version"):
+ convert_ossie_to_auto_ontology(yaml.safe_dump(ossie))
def test_dialect_specific_native_sql_survives_a_round_trip() -> None:
"""``TOP n`` is valid Snowflake but the default parser rejects it."""
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
manual = native["semantic_layer"]["sql_attributes"]["manual"][0]
manual["sql"] = (
'SELECT TOP 1 "orders"."subtotal" AS "net_total" '
'FROM "analytics"."public"."orders" AS "orders"'
)
- ossie = convert_gsf_to_ossie(yaml.safe_dump(native))
- restored = yaml.safe_load(convert_ossie_to_gsf(ossie))
+ ossie = convert_auto_ontology_to_ossie(yaml.safe_dump(native))
+ restored = yaml.safe_load(convert_ossie_to_auto_ontology(ossie))
assert _manual_sql(restored, "net_total") == manual["sql"]
def test_native_sql_no_dialect_can_parse_is_carried_through_verbatim() -> None:
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
manual = native["semantic_layer"]["sql_attributes"]["manual"][0]
manual["sql"] = "SELECT not ((parseable by any dialect"
- ossie = convert_gsf_to_ossie(yaml.safe_dump(native))
- restored = yaml.safe_load(convert_ossie_to_gsf(ossie))
+ ossie = convert_auto_ontology_to_ossie(yaml.safe_dump(native))
+ restored = yaml.safe_load(convert_ossie_to_auto_ontology(ossie))
assert _manual_sql(restored, "net_total") == manual["sql"]
+@pytest.mark.parametrize("legacy_vendor", ["NVIDIA_GSF", "GSF"])
+def test_extensions_written_under_the_gsf_name_are_still_read(
+ legacy_vendor: str,
+) -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
+ native["zones"] = [{"id": "zone-emea", "name": "emea"}]
+ ossie = convert_auto_ontology_to_ossie(yaml.safe_dump(native))
+ legacy_ossie = ossie.replace(
+ "vendor_name: NVIDIA_AUTO_ONTOLOGY", f"vendor_name: {legacy_vendor}"
+ )
+ assert legacy_ossie != ossie
+
+ restored = yaml.safe_load(convert_ossie_to_auto_ontology(legacy_ossie))
+
+ assert restored == yaml.safe_load(convert_ossie_to_auto_ontology(ossie))
+ assert restored["zones"] == native["zones"]
+
+
@pytest.mark.parametrize(
"expression",
[
@@ -693,7 +721,7 @@ def test_columns_named_like_units_survive_outside_the_unit_slot(
}
)
- native = yaml.safe_load(convert_ossie_to_gsf(yaml.safe_dump(ossie)))
+ native = yaml.safe_load(convert_ossie_to_auto_ontology(yaml.safe_dump(ossie)))
orders = next(
table
for database in native["data_layer"]["databases"]
@@ -707,15 +735,19 @@ def test_columns_named_like_units_survive_outside_the_unit_slot(
def test_malformed_native_snapshot_fails_alike_in_both_paths() -> None:
"""Indexing and relationship reconciliation read the same snapshot."""
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
native["data_layer"]["databases"].append(
deepcopy(native["data_layer"]["databases"][0])
)
- with pytest.raises(GSFConversionError, match="Malformed NVIDIA_GSF"):
+ with pytest.raises(
+ AutoOntologyConversionError, match="Malformed NVIDIA_AUTO_ONTOLOGY"
+ ):
_index_native_document(native)
- with pytest.raises(GSFConversionError, match="Malformed NVIDIA_GSF"):
+ with pytest.raises(
+ AutoOntologyConversionError, match="Malformed NVIDIA_AUTO_ONTOLOGY"
+ ):
_reconcile_native_relationships(
native,
represented_table_ids=set(),
@@ -725,11 +757,11 @@ def test_malformed_native_snapshot_fails_alike_in_both_paths() -> None:
)
-def test_expression_dialect_follows_the_gsf_connection() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_expression_dialect_follows_the_auto_ontology_connection() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
native["data_layer"]["databases"][0]["dialect"] = "snowflake"
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
orders = next(
dataset
for dataset in ossie["datasets"]
@@ -746,10 +778,10 @@ def test_expression_dialect_follows_the_gsf_connection() -> None:
def test_dialects_ossie_cannot_name_stay_ansi() -> None:
- native = yaml.safe_load(_gsf_yaml())
+ native = yaml.safe_load(_auto_ontology_yaml())
native["data_layer"]["databases"][0]["dialect"] = "mysql"
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
orders = next(
dataset
for dataset in ossie["datasets"]
@@ -774,8 +806,8 @@ def test_every_mappable_datatype_survives_a_round_trip(datatype: str) -> None:
"datatype"
] = datatype
- native = convert_ossie_to_gsf(yaml.safe_dump(ossie))
- restored = yaml.safe_load(convert_gsf_to_ossie(native))
+ native = convert_ossie_to_auto_ontology(yaml.safe_dump(ossie))
+ restored = yaml.safe_load(convert_auto_ontology_to_ossie(native))
field = next(
item
for dataset in restored["datasets"]
@@ -827,8 +859,8 @@ def test_physical_types_map_onto_the_ossie_vocabulary(
assert _ossie_datatype(sql_type) == expected
-def test_gsf_column_types_reach_the_ossie_field() -> None:
- native = yaml.safe_load(_gsf_yaml())
+def test_auto_ontology_column_types_reach_the_ossie_field() -> None:
+ native = yaml.safe_load(_auto_ontology_yaml())
orders = next(
table
for database in native["data_layer"]["databases"]
@@ -839,7 +871,7 @@ def test_gsf_column_types_reach_the_ossie_field() -> None:
for column in orders["columns"]:
column["type"] = "NUMBER(38,0)" if column["name"] == "order_id" else "TEXT"
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
fields = {
field["name"]: field.get("datatype")
for dataset in ossie["datasets"]
@@ -849,13 +881,13 @@ def test_gsf_column_types_reach_the_ossie_field() -> None:
assert fields["order_id"] == "Integer"
assert fields["customer_id"] == "String"
- # A computed attribute has no column, so GSF holds no type for it.
+ # A computed attribute has no column, so Auto Ontology holds no type for it.
assert fields["net_total"] is None
-def test_a_live_gsf_column_type_outranks_an_ossie_datatype() -> None:
- """GSF reports the physical type; Ossie only names a logical one."""
- native = yaml.safe_load(_gsf_yaml())
+def test_a_live_auto_ontology_column_type_outranks_an_ossie_datatype() -> None:
+ """Auto Ontology reports the physical type; Ossie only names a logical one."""
+ native = yaml.safe_load(_auto_ontology_yaml())
orders = next(
table
for database in native["data_layer"]["databases"]
@@ -867,7 +899,7 @@ def test_a_live_gsf_column_type_outranks_an_ossie_datatype() -> None:
"type"
] = "NUMBER(38,0)"
- ossie = yaml.safe_load(convert_gsf_to_ossie(yaml.safe_dump(native)))
+ ossie = yaml.safe_load(convert_auto_ontology_to_ossie(yaml.safe_dump(native)))
next(
field
for dataset in ossie["datasets"]
@@ -876,7 +908,7 @@ def test_a_live_gsf_column_type_outranks_an_ossie_datatype() -> None:
if field["name"] == "order_id"
)["datatype"] = "String"
- restored = yaml.safe_load(convert_ossie_to_gsf(yaml.safe_dump(ossie)))
+ restored = yaml.safe_load(convert_ossie_to_auto_ontology(yaml.safe_dump(ossie)))
column = next(
column
for database in restored["data_layer"]["databases"]
@@ -889,19 +921,23 @@ def test_a_live_gsf_column_type_outranks_an_ossie_datatype() -> None:
assert column["type"] == "NUMBER(38,0)"
-def test_old_fictional_gsf_root_is_rejected() -> None:
+def test_old_fictional_auto_ontology_root_is_rejected() -> None:
old_shape = {
"version": "1.0",
"model": {"name": "sales"},
"terms": [],
}
- with pytest.raises(GSFConversionError, match="Unsupported GSF root"):
- convert_gsf_to_ossie(yaml.safe_dump(old_shape))
+ with pytest.raises(
+ AutoOntologyConversionError, match="Unsupported Auto Ontology root"
+ ):
+ convert_auto_ontology_to_ossie(yaml.safe_dump(old_shape))
def test_model_name_override() -> None:
- result = yaml.safe_load(convert_gsf_to_ossie(_gsf_yaml(), model_name="sales"))
+ result = yaml.safe_load(
+ convert_auto_ontology_to_ossie(_auto_ontology_yaml(), model_name="sales")
+ )
assert result["name"] == "sales"
@@ -953,17 +989,17 @@ def test_cli_converts_native_files(
capsys: pytest.CaptureFixture[str],
) -> None:
ossie_path = tmp_path / "model.yaml"
- gsf_path = tmp_path / "model.gsf.yaml"
+ auto_ontology_path = tmp_path / "model.auto_ontology.yaml"
ossie_path.write_text(_ossie_yaml(), encoding="utf-8")
- main(["export", "-i", str(ossie_path), "-o", str(gsf_path)])
- assert set(yaml.safe_load(gsf_path.read_text(encoding="utf-8"))) == {
+ main(["export", "-i", str(ossie_path), "-o", str(auto_ontology_path)])
+ assert set(yaml.safe_load(auto_ontology_path.read_text(encoding="utf-8"))) == {
"data_layer",
"semantic_layer",
"zones",
}
- main(["import", "-i", str(gsf_path), "--name", "sales"])
+ main(["import", "-i", str(auto_ontology_path), "--name", "sales"])
output = yaml.safe_load(capsys.readouterr().out)
assert output["version"] == OSSIE_VERSION
assert output["name"] == "sales"
@@ -974,8 +1010,8 @@ def test_rejects_legacy_wrapper_even_with_root_model(wrapper: Any) -> None:
document = yaml.safe_load(_ossie_yaml())
document["semantic_model"] = wrapper
- with pytest.raises(GSFConversionError, match="at the root"):
- convert_ossie_to_gsf(yaml.safe_dump(document))
+ with pytest.raises(AutoOntologyConversionError, match="at the root"):
+ convert_ossie_to_auto_ontology(yaml.safe_dump(document))
@pytest.mark.parametrize("property_name", ["dialects", "vendors"])
@@ -983,5 +1019,7 @@ def test_rejects_removed_root_metadata(property_name: str) -> None:
document = yaml.safe_load(_ossie_yaml())
document[property_name] = []
- with pytest.raises(GSFConversionError, match="Unsupported Ossie root properties"):
- convert_ossie_to_gsf(yaml.safe_dump(document))
+ with pytest.raises(
+ AutoOntologyConversionError, match="Unsupported Ossie root properties"
+ ):
+ convert_ossie_to_auto_ontology(yaml.safe_dump(document))
diff --git a/converters/nvidia/uv.lock b/converters/nvidia/uv.lock
index 87d22904..a6cb2b07 100644
--- a/converters/nvidia/uv.lock
+++ b/converters/nvidia/uv.lock
@@ -3,7 +3,7 @@ revision = 3
requires-python = ">=3.11"
[[package]]
-name = "apache-ossie-nvidia-gsf"
+name = "apache-ossie-nvidia-auto-ontology"
version = "0.1.0.dev0"
source = { editable = "." }
dependencies = [
diff --git a/converters/sigma/src/ossie_sigma/sigma_formula.py b/converters/sigma/src/ossie_sigma/sigma_formula.py
index e7e9db59..430479c2 100644
--- a/converters/sigma/src/ossie_sigma/sigma_formula.py
+++ b/converters/sigma/src/ossie_sigma/sigma_formula.py
@@ -19,8 +19,8 @@
Sigma data model column and metric formulas look like ``Sum([Orders/Amount])`` or
``If([Status] = "closed", 1, 0)``. Sigma's formula language is not SQL, so — unlike
-converters whose native expressions are already SQL (e.g. NVIDIA GSF, which parses
-straight into sqlglot) — a real tokenizer and recursive-descent parser is needed to
+converters whose native expressions are already SQL (e.g. NVIDIA Auto Ontology, which
+parses straight into sqlglot) — a real tokenizer and recursive-descent parser is needed to
get from formula text to a tree (:class:`FormulaNode`).
From there, though, this module does what the other converters do: it translates into