You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@ossie.apache.org by Kyoung Min Kim <ka...@gmail.com> on 2026/09/28 06:23:34 UTC
[DISCUSS] Semantic checks for ontology documents in validate.py
Hi,
validate.py is the tool the repo documents for checking an ontology
document (--schema ontology/ontology.json), and the validation CI runs
it. For core documents it goes beyond the schema: unique names,
relationship references, column arity, SQL parsing. For ontology
documents it stops at the schema, because every semantic check is
guarded by the presence of `datasets`.
I tried a small ontology document with five defects, one at a time, on
current main. All five come back "Validation PASSED":
- two components declaring the same concept name
- `extends` naming a concept that is not declared
- `identify_by` naming a relationship the concept does not have
- a role whose concept is not declared
- an `iri` QName whose prefix is not in `prefixes`, which ontology.md
says is invalid
The reference parser in converters/ontology rejects four of them with
clear messages. The duplicate concept goes through there as well. So
the rules exist, but only for people who run the converter, and CI runs
validate.py.
What I have in mind is the same kind of checks the core side already
has, scoped to the `ontology` section: unique concept and relationship
names, references from `extends`, `identify_by` and roles to declared
concepts and relationships (built-ins like String and Any included),
and QName prefixes against the `prefixes` map. I would leave
`ontology_mappings` alone for now, since #458 is about to change where
that lives. The author of #479 mentioned the prefix gap there and
offered to open an issue for it, so that part can be folded in or
tracked separately, whichever they prefer.
Before I write it I would like a read on two things: whether
validate.py is the right place for these at all, given the parser has
its own copy of most of them, and whether that scope is right or should
wait for the ontology spec to settle further. If it sounds reasonable I
will put the PR together with tests under validation/tests.
Thanks,
Kyoung Min Kim
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Andrew Nepogoda via dev <de...@ossie.apache.org>.
Hi JB, Kyoung,
Thanks for pointing that out.
I've checked the PRs you've mentioned, and I think it makes sense to move
the DTOs from spec.py to model.py once all the mentioned PRs have landed in
main.
My proposal:
1.
Merge the Ossi → RelationalAI <https://github.com/apache/ossie/pull/412>
and LinkML ↔ Ossie <https://github.com/apache/ossie/pull/389> converters.
2.
Finish and merge reshaping <https://github.com/apache/ossie/pull/458>.
3.
Update the ossie_to_spec and spec_to_ossie converters in
https://github.com/apache/ossie/tree/main/converters/ontology/src/ossie_ontology/converter
to follow the new structure.
4.
Move all DTO classes from spec.py
<https://github.com/apache/ossie/blob/main/converters/ontology/src/ossie_ontology/spec.py>
to model.py
<https://github.com/apache/ossie/blob/main/python/src/ossie/models.py>.
5.
Introduce a new validation module within the validation folder.
I'm happy to take on items 3, 4, and 5.
Regards,
Andrew Nepogoda
On Wed, Sep 30, 2026 at 5:52 AM Jean-Baptiste Onofré <jb...@nanthrax.net>
wrote:
> Hi Kyoung, Andrew,
>
> Thanks Kyoung, the plan sounds great! Please go ahead and open the
> issue and PR for validate.py. I will review it. Keeping these checks
> dict-based in validate.py for now is definitely the right move imho:
> it gives us immediate CI coverage without coupling to the upcoming
> type refactoring.
>
> On Andrew's proposal: I'm also +1 on centralizing the models into
> python (that will move to ossie-converters repo), and Kyoung's
> comments are right:
> 1. We need to be intentional here, as changing the default behavior
> could impact the converters. Calling this out clearly in the PR
> description will be essential.
> 2. Providing thin re-exports in ossy_ontology.spec during the
> transition is a great catch to avoid breaking open PRs and downstream
> consumers.
> 3. Leaving the mapping classes untouched unti #458 lands make sense
> and avoids unnecessary churn and merge conflicts.
>
> Looking forward reviewing both PRs :)
>
> Regards
> JB
>
> On Wed, Sep 30, 2026 at 2:01 AM Kyoung Min Kim <ka...@gmail.com> wrote:
> >
> > Hi JB, Andrew,
> >
> > JB, thanks for the read. I will open the issue and put the PR together
> > with the three additions: the built-in concepts in the extends and role
> > resolution, a cycle check on extends, and a validation-ci step running
> > flights.yaml against the ontology schema.
> >
> > Andrew, having one set of pydantic types in python/ sounds right to me.
> Twelve
> > converters already depend on apache-ossie for the core types, and the
> > ontology converter is the only one carrying its own copy, so the move
> > closes that gap as well as preparing the extraction.
> >
> > A few things I noticed while reading both files, in case they help with
> > the plan:
> >
> > - models.py has no ontology types at all, so concepts, relationships and
> > roles are an addition rather than a merge. The core classes do overlap
> > and differ in posture: spec.py is extra="forbid" and mutable, models.py
> > is frozen and ignores extras. Whichever you pick becomes the behaviour
> > for the twelve downstream converters, so it may be worth calling out
> > in the PR description.
> > - ossie_ontology.spec is imported from four places in the converter, and
> > #412 (PyRel) and #389 (LinkML) build on it. A thin re-export from the
> > old path during the transition would keep those from breaking.
> > - The mapping classes are what #458 is reshaping. Leaving them where they
> > are until that settles would avoid moving them twice, the same reason
> JB
> > gave for keeping mapping checks out of validate.py for now.
> >
> > The ontology-section checks I have in progress work on the parsed dict in
> > validate.py and touch neither module, so they should land independently
> > of the move. Once the types are in python/ I am happy to port them onto
> > the new module or hand them over, whichever fits your validation module
> > better.
> >
> > Kyoung Min Kim
> >
> > > 2026. 9. 30. 오전 12:18, Andrew Nepogoda <an...@relational.ai>
> 작성:
> > >
> > > Hi there,
> > >
> > > I've started looking into the validation implementation, and before
> issuing any validation PRs, I'd like to move the OssieSpec objects from
> spec.py <
> https://github.com/apache/ossie/blob/main/converters/ontology/src/ossie_ontology/spec.py>
> into the python <https://github.com/apache/ossie/tree/main/python>
> directory and merge with model.py <
> https://github.com/apache/ossie/blob/main/python/src/ossie/models.py>,
> which seems like the right place for them.
> > >
> > > Once they're moved, we can reuse them in any module where needed,
> particularly in a new validation module. This would also be one of the
> steps toward moving the converters out and reusing common code from the
> OSSIE repository.
> > >
> > > What do you think about this approach?
> > >
> > >
> > > Regards,
> > > Andrew Nepogoda
> > >
> > >
> > > On Tue, Sep 29, 2026 at 2:46 AM Kyoung Min Kim <kayemkim@gmail.com
> <ma...@gmail.com>> wrote:
> > >> Hi Kurt,
> > >>
> > >> Thanks, that matches how I read it. Pulling the ontology validation
> and
> > >> the formula parser into the main repo and having validate.py call it
> is
> > >> the right end state, and I would be glad to review that PR when it
> comes.
> > >>
> > >> Since the parser move depends on where the converter scope discussion
> > >> lands, my thought is to put the ontology-section checks into
> validate.py
> > >> first, as a small PR with tests, written as self-contained functions
> so
> > >> the extracted module can replace them without touching the callers.
> That
> > >> gets CI catching the five defects in the meantime, whatever the
> timing of
> > >> the larger change.
> > >>
> > >> On the mapping side, Shao has offered in the decoupling thread to add
> the
> > >> reference checks for mapping documents once the embedded versus
> standalone
> > >> question is settled, so the three pieces would not overlap: the
> ontology
> > >> section now, mapping references after #458, and the parser extraction
> > >> replacing the internals.
> > >>
> > >> If you would rather fold the checks straight into the extraction PR,
> that
> > >> works for me too and I will take the review side instead.
> > >>
> > >> Kyoung Min Kim
> > >>
> > >> 2026년 9월 28일 (월) 오후 11:35, Kurt Stirewalt <kurt.stirewalt@gmail.com
> <ma...@gmail.com>>님이 작성:
> > >>> Hi Kyoung,
> > >>>
> > >>> Yeah, that's a good observation. The ontology validations that are
> > >>> currently performed in the converters should run in validate.py. In
> fact,
> > >>> this was one of the motivators for my email last Thursday in the
> "Scope and
> > >>> maintenance of Ossie converters" thread. There, we suggested
> isolating into
> > >>> the main Ossie repo:
> > >>>
> > >>> -
> > >>>
> > >>> the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
> > >>> -
> > >>>
> > >>> the DTOs / object model for Ossie documents — a canonical, typed,
> > >>> versioned in-memory representation with parsing and serialization;
> > >>> -
> > >>>
> > >>> the parser for the formula language that is used in the ontology
> spec
> > >>> and the AST it produces;
> > >>> -
> > >>>
> > >>> validation and conformance tooling (cf. the validation/
> directory);
> > >>> -
> > >>>
> > >>> a compatibility/conformance test kit containing example Ossie
> models
> > >>> that cover the Ossie spec and that any converter can run against.
> > >>>
> > >>> If this sounds reasonable, my team and I can put together a PR that
> > >>> extracts all aspects of ontology validation (including the formula
> parser)
> > >>> out of the current converters and extend validate.py to call it.
> > >>>
> > >>>
> > >>> Please advise, and thanks for the suggestion.
> > >>>
> > >>>
> > >>> Kurt
> > >>>
> > >>> On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <kayemkim@gmail.com
> <ma...@gmail.com>> wrote:
> > >>>
> > >>> > Hi,
> > >>> >
> > >>> > validate.py is the tool the repo documents for checking an ontology
> > >>> > document (--schema ontology/ontology.json), and the validation CI
> runs
> > >>> > it. For core documents it goes beyond the schema: unique names,
> > >>> > relationship references, column arity, SQL parsing. For ontology
> > >>> > documents it stops at the schema, because every semantic check is
> > >>> > guarded by the presence of `datasets`.
> > >>> >
> > >>> > I tried a small ontology document with five defects, one at a
> time, on
> > >>> > current main. All five come back "Validation PASSED":
> > >>> >
> > >>> > - two components declaring the same concept name
> > >>> > - `extends` naming a concept that is not declared
> > >>> > - `identify_by` naming a relationship the concept does not have
> > >>> > - a role whose concept is not declared
> > >>> > - an `iri` QName whose prefix is not in `prefixes`, which
> ontology.md
> > >>> > says is invalid
> > >>> >
> > >>> > The reference parser in converters/ontology rejects four of them
> with
> > >>> > clear messages. The duplicate concept goes through there as well.
> So
> > >>> > the rules exist, but only for people who run the converter, and CI
> runs
> > >>> > validate.py.
> > >>> >
> > >>> > What I have in mind is the same kind of checks the core side
> already
> > >>> > has, scoped to the `ontology` section: unique concept and
> relationship
> > >>> > names, references from `extends`, `identify_by` and roles to
> declared
> > >>> > concepts and relationships (built-ins like String and Any
> included),
> > >>> > and QName prefixes against the `prefixes` map. I would leave
> > >>> > `ontology_mappings` alone for now, since #458 is about to change
> where
> > >>> > that lives. The author of #479 mentioned the prefix gap there and
> > >>> > offered to open an issue for it, so that part can be folded in or
> > >>> > tracked separately, whichever they prefer.
> > >>> >
> > >>> > Before I write it I would like a read on two things: whether
> > >>> > validate.py is the right place for these at all, given the parser
> has
> > >>> > its own copy of most of them, and whether that scope is right or
> should
> > >>> > wait for the ontology spec to settle further. If it sounds
> reasonable I
> > >>> > will put the PR together with tests under validation/tests.
> > >>> >
> > >>> > Thanks,
> > >>> > Kyoung Min Kim
> > >>> >
> >
>
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Jean-Baptiste Onofré <jb...@nanthrax.net>.
Hi Kyoung, Andrew,
Thanks Kyoung, the plan sounds great! Please go ahead and open the
issue and PR for validate.py. I will review it. Keeping these checks
dict-based in validate.py for now is definitely the right move imho:
it gives us immediate CI coverage without coupling to the upcoming
type refactoring.
On Andrew's proposal: I'm also +1 on centralizing the models into
python (that will move to ossie-converters repo), and Kyoung's
comments are right:
1. We need to be intentional here, as changing the default behavior
could impact the converters. Calling this out clearly in the PR
description will be essential.
2. Providing thin re-exports in ossy_ontology.spec during the
transition is a great catch to avoid breaking open PRs and downstream
consumers.
3. Leaving the mapping classes untouched unti #458 lands make sense
and avoids unnecessary churn and merge conflicts.
Looking forward reviewing both PRs :)
Regards
JB
On Wed, Sep 30, 2026 at 2:01 AM Kyoung Min Kim <ka...@gmail.com> wrote:
>
> Hi JB, Andrew,
>
> JB, thanks for the read. I will open the issue and put the PR together
> with the three additions: the built-in concepts in the extends and role
> resolution, a cycle check on extends, and a validation-ci step running
> flights.yaml against the ontology schema.
>
> Andrew, having one set of pydantic types in python/ sounds right to me. Twelve
> converters already depend on apache-ossie for the core types, and the
> ontology converter is the only one carrying its own copy, so the move
> closes that gap as well as preparing the extraction.
>
> A few things I noticed while reading both files, in case they help with
> the plan:
>
> - models.py has no ontology types at all, so concepts, relationships and
> roles are an addition rather than a merge. The core classes do overlap
> and differ in posture: spec.py is extra="forbid" and mutable, models.py
> is frozen and ignores extras. Whichever you pick becomes the behaviour
> for the twelve downstream converters, so it may be worth calling out
> in the PR description.
> - ossie_ontology.spec is imported from four places in the converter, and
> #412 (PyRel) and #389 (LinkML) build on it. A thin re-export from the
> old path during the transition would keep those from breaking.
> - The mapping classes are what #458 is reshaping. Leaving them where they
> are until that settles would avoid moving them twice, the same reason JB
> gave for keeping mapping checks out of validate.py for now.
>
> The ontology-section checks I have in progress work on the parsed dict in
> validate.py and touch neither module, so they should land independently
> of the move. Once the types are in python/ I am happy to port them onto
> the new module or hand them over, whichever fits your validation module
> better.
>
> Kyoung Min Kim
>
> > 2026. 9. 30. 오전 12:18, Andrew Nepogoda <an...@relational.ai> 작성:
> >
> > Hi there,
> >
> > I've started looking into the validation implementation, and before issuing any validation PRs, I'd like to move the OssieSpec objects from spec.py <https://github.com/apache/ossie/blob/main/converters/ontology/src/ossie_ontology/spec.py> into the python <https://github.com/apache/ossie/tree/main/python> directory and merge with model.py <https://github.com/apache/ossie/blob/main/python/src/ossie/models.py>, which seems like the right place for them.
> >
> > Once they're moved, we can reuse them in any module where needed, particularly in a new validation module. This would also be one of the steps toward moving the converters out and reusing common code from the OSSIE repository.
> >
> > What do you think about this approach?
> >
> >
> > Regards,
> > Andrew Nepogoda
> >
> >
> > On Tue, Sep 29, 2026 at 2:46 AM Kyoung Min Kim <kayemkim@gmail.com <ma...@gmail.com>> wrote:
> >> Hi Kurt,
> >>
> >> Thanks, that matches how I read it. Pulling the ontology validation and
> >> the formula parser into the main repo and having validate.py call it is
> >> the right end state, and I would be glad to review that PR when it comes.
> >>
> >> Since the parser move depends on where the converter scope discussion
> >> lands, my thought is to put the ontology-section checks into validate.py
> >> first, as a small PR with tests, written as self-contained functions so
> >> the extracted module can replace them without touching the callers. That
> >> gets CI catching the five defects in the meantime, whatever the timing of
> >> the larger change.
> >>
> >> On the mapping side, Shao has offered in the decoupling thread to add the
> >> reference checks for mapping documents once the embedded versus standalone
> >> question is settled, so the three pieces would not overlap: the ontology
> >> section now, mapping references after #458, and the parser extraction
> >> replacing the internals.
> >>
> >> If you would rather fold the checks straight into the extraction PR, that
> >> works for me too and I will take the review side instead.
> >>
> >> Kyoung Min Kim
> >>
> >> 2026년 9월 28일 (월) 오후 11:35, Kurt Stirewalt <kurt.stirewalt@gmail.com <ma...@gmail.com>>님이 작성:
> >>> Hi Kyoung,
> >>>
> >>> Yeah, that's a good observation. The ontology validations that are
> >>> currently performed in the converters should run in validate.py. In fact,
> >>> this was one of the motivators for my email last Thursday in the "Scope and
> >>> maintenance of Ossie converters" thread. There, we suggested isolating into
> >>> the main Ossie repo:
> >>>
> >>> -
> >>>
> >>> the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
> >>> -
> >>>
> >>> the DTOs / object model for Ossie documents — a canonical, typed,
> >>> versioned in-memory representation with parsing and serialization;
> >>> -
> >>>
> >>> the parser for the formula language that is used in the ontology spec
> >>> and the AST it produces;
> >>> -
> >>>
> >>> validation and conformance tooling (cf. the validation/ directory);
> >>> -
> >>>
> >>> a compatibility/conformance test kit containing example Ossie models
> >>> that cover the Ossie spec and that any converter can run against.
> >>>
> >>> If this sounds reasonable, my team and I can put together a PR that
> >>> extracts all aspects of ontology validation (including the formula parser)
> >>> out of the current converters and extend validate.py to call it.
> >>>
> >>>
> >>> Please advise, and thanks for the suggestion.
> >>>
> >>>
> >>> Kurt
> >>>
> >>> On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <kayemkim@gmail.com <ma...@gmail.com>> wrote:
> >>>
> >>> > Hi,
> >>> >
> >>> > validate.py is the tool the repo documents for checking an ontology
> >>> > document (--schema ontology/ontology.json), and the validation CI runs
> >>> > it. For core documents it goes beyond the schema: unique names,
> >>> > relationship references, column arity, SQL parsing. For ontology
> >>> > documents it stops at the schema, because every semantic check is
> >>> > guarded by the presence of `datasets`.
> >>> >
> >>> > I tried a small ontology document with five defects, one at a time, on
> >>> > current main. All five come back "Validation PASSED":
> >>> >
> >>> > - two components declaring the same concept name
> >>> > - `extends` naming a concept that is not declared
> >>> > - `identify_by` naming a relationship the concept does not have
> >>> > - a role whose concept is not declared
> >>> > - an `iri` QName whose prefix is not in `prefixes`, which ontology.md
> >>> > says is invalid
> >>> >
> >>> > The reference parser in converters/ontology rejects four of them with
> >>> > clear messages. The duplicate concept goes through there as well. So
> >>> > the rules exist, but only for people who run the converter, and CI runs
> >>> > validate.py.
> >>> >
> >>> > What I have in mind is the same kind of checks the core side already
> >>> > has, scoped to the `ontology` section: unique concept and relationship
> >>> > names, references from `extends`, `identify_by` and roles to declared
> >>> > concepts and relationships (built-ins like String and Any included),
> >>> > and QName prefixes against the `prefixes` map. I would leave
> >>> > `ontology_mappings` alone for now, since #458 is about to change where
> >>> > that lives. The author of #479 mentioned the prefix gap there and
> >>> > offered to open an issue for it, so that part can be folded in or
> >>> > tracked separately, whichever they prefer.
> >>> >
> >>> > Before I write it I would like a read on two things: whether
> >>> > validate.py is the right place for these at all, given the parser has
> >>> > its own copy of most of them, and whether that scope is right or should
> >>> > wait for the ontology spec to settle further. If it sounds reasonable I
> >>> > will put the PR together with tests under validation/tests.
> >>> >
> >>> > Thanks,
> >>> > Kyoung Min Kim
> >>> >
>
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Kyoung Min Kim <ka...@gmail.com>.
Hi JB, Andrew,
JB, thanks for the read. I will open the issue and put the PR together
with the three additions: the built-in concepts in the extends and role
resolution, a cycle check on extends, and a validation-ci step running
flights.yaml against the ontology schema.
Andrew, having one set of pydantic types in python/ sounds right to me. Twelve
converters already depend on apache-ossie for the core types, and the
ontology converter is the only one carrying its own copy, so the move
closes that gap as well as preparing the extraction.
A few things I noticed while reading both files, in case they help with
the plan:
- models.py has no ontology types at all, so concepts, relationships and
roles are an addition rather than a merge. The core classes do overlap
and differ in posture: spec.py is extra="forbid" and mutable, models.py
is frozen and ignores extras. Whichever you pick becomes the behaviour
for the twelve downstream converters, so it may be worth calling out
in the PR description.
- ossie_ontology.spec is imported from four places in the converter, and
#412 (PyRel) and #389 (LinkML) build on it. A thin re-export from the
old path during the transition would keep those from breaking.
- The mapping classes are what #458 is reshaping. Leaving them where they
are until that settles would avoid moving them twice, the same reason JB
gave for keeping mapping checks out of validate.py for now.
The ontology-section checks I have in progress work on the parsed dict in
validate.py and touch neither module, so they should land independently
of the move. Once the types are in python/ I am happy to port them onto
the new module or hand them over, whichever fits your validation module
better.
Kyoung Min Kim
> 2026. 9. 30. 오전 12:18, Andrew Nepogoda <an...@relational.ai> 작성:
>
> Hi there,
>
> I've started looking into the validation implementation, and before issuing any validation PRs, I'd like to move the OssieSpec objects from spec.py <https://github.com/apache/ossie/blob/main/converters/ontology/src/ossie_ontology/spec.py> into the python <https://github.com/apache/ossie/tree/main/python> directory and merge with model.py <https://github.com/apache/ossie/blob/main/python/src/ossie/models.py>, which seems like the right place for them.
>
> Once they're moved, we can reuse them in any module where needed, particularly in a new validation module. This would also be one of the steps toward moving the converters out and reusing common code from the OSSIE repository.
>
> What do you think about this approach?
>
>
> Regards,
> Andrew Nepogoda
>
>
> On Tue, Sep 29, 2026 at 2:46 AM Kyoung Min Kim <kayemkim@gmail.com <ma...@gmail.com>> wrote:
>> Hi Kurt,
>>
>> Thanks, that matches how I read it. Pulling the ontology validation and
>> the formula parser into the main repo and having validate.py call it is
>> the right end state, and I would be glad to review that PR when it comes.
>>
>> Since the parser move depends on where the converter scope discussion
>> lands, my thought is to put the ontology-section checks into validate.py
>> first, as a small PR with tests, written as self-contained functions so
>> the extracted module can replace them without touching the callers. That
>> gets CI catching the five defects in the meantime, whatever the timing of
>> the larger change.
>>
>> On the mapping side, Shao has offered in the decoupling thread to add the
>> reference checks for mapping documents once the embedded versus standalone
>> question is settled, so the three pieces would not overlap: the ontology
>> section now, mapping references after #458, and the parser extraction
>> replacing the internals.
>>
>> If you would rather fold the checks straight into the extraction PR, that
>> works for me too and I will take the review side instead.
>>
>> Kyoung Min Kim
>>
>> 2026년 9월 28일 (월) 오후 11:35, Kurt Stirewalt <kurt.stirewalt@gmail.com <ma...@gmail.com>>님이 작성:
>>> Hi Kyoung,
>>>
>>> Yeah, that's a good observation. The ontology validations that are
>>> currently performed in the converters should run in validate.py. In fact,
>>> this was one of the motivators for my email last Thursday in the "Scope and
>>> maintenance of Ossie converters" thread. There, we suggested isolating into
>>> the main Ossie repo:
>>>
>>> -
>>>
>>> the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
>>> -
>>>
>>> the DTOs / object model for Ossie documents — a canonical, typed,
>>> versioned in-memory representation with parsing and serialization;
>>> -
>>>
>>> the parser for the formula language that is used in the ontology spec
>>> and the AST it produces;
>>> -
>>>
>>> validation and conformance tooling (cf. the validation/ directory);
>>> -
>>>
>>> a compatibility/conformance test kit containing example Ossie models
>>> that cover the Ossie spec and that any converter can run against.
>>>
>>> If this sounds reasonable, my team and I can put together a PR that
>>> extracts all aspects of ontology validation (including the formula parser)
>>> out of the current converters and extend validate.py to call it.
>>>
>>>
>>> Please advise, and thanks for the suggestion.
>>>
>>>
>>> Kurt
>>>
>>> On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <kayemkim@gmail.com <ma...@gmail.com>> wrote:
>>>
>>> > Hi,
>>> >
>>> > validate.py is the tool the repo documents for checking an ontology
>>> > document (--schema ontology/ontology.json), and the validation CI runs
>>> > it. For core documents it goes beyond the schema: unique names,
>>> > relationship references, column arity, SQL parsing. For ontology
>>> > documents it stops at the schema, because every semantic check is
>>> > guarded by the presence of `datasets`.
>>> >
>>> > I tried a small ontology document with five defects, one at a time, on
>>> > current main. All five come back "Validation PASSED":
>>> >
>>> > - two components declaring the same concept name
>>> > - `extends` naming a concept that is not declared
>>> > - `identify_by` naming a relationship the concept does not have
>>> > - a role whose concept is not declared
>>> > - an `iri` QName whose prefix is not in `prefixes`, which ontology.md
>>> > says is invalid
>>> >
>>> > The reference parser in converters/ontology rejects four of them with
>>> > clear messages. The duplicate concept goes through there as well. So
>>> > the rules exist, but only for people who run the converter, and CI runs
>>> > validate.py.
>>> >
>>> > What I have in mind is the same kind of checks the core side already
>>> > has, scoped to the `ontology` section: unique concept and relationship
>>> > names, references from `extends`, `identify_by` and roles to declared
>>> > concepts and relationships (built-ins like String and Any included),
>>> > and QName prefixes against the `prefixes` map. I would leave
>>> > `ontology_mappings` alone for now, since #458 is about to change where
>>> > that lives. The author of #479 mentioned the prefix gap there and
>>> > offered to open an issue for it, so that part can be folded in or
>>> > tracked separately, whichever they prefer.
>>> >
>>> > Before I write it I would like a read on two things: whether
>>> > validate.py is the right place for these at all, given the parser has
>>> > its own copy of most of them, and whether that scope is right or should
>>> > wait for the ontology spec to settle further. If it sounds reasonable I
>>> > will put the PR together with tests under validation/tests.
>>> >
>>> > Thanks,
>>> > Kyoung Min Kim
>>> >
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Andrew Nepogoda via dev <de...@ossie.apache.org>.
Hi there,
I've started looking into the validation implementation, and before issuing
any validation PRs, I'd like to move the OssieSpec objects from spec.py
<https://github.com/apache/ossie/blob/main/converters/ontology/src/ossie_ontology/spec.py>
into the python <https://github.com/apache/ossie/tree/main/python>
directory and merge with model.py
<https://github.com/apache/ossie/blob/main/python/src/ossie/models.py>,
which seems like the right place for them.
Once they're moved, we can reuse them in any module where needed,
particularly in a new validation module. This would also be one of the
steps toward moving the converters out and reusing common code from the
OSSIE repository.
What do you think about this approach?
Regards,
Andrew Nepogoda
On Tue, Sep 29, 2026 at 2:46 AM Kyoung Min Kim <ka...@gmail.com> wrote:
> Hi Kurt,
>
> Thanks, that matches how I read it. Pulling the ontology validation and
> the formula parser into the main repo and having validate.py call it is
> the right end state, and I would be glad to review that PR when it comes.
>
> Since the parser move depends on where the converter scope discussion
> lands, my thought is to put the ontology-section checks into validate.py
> first, as a small PR with tests, written as self-contained functions so
> the extracted module can replace them without touching the callers. That
> gets CI catching the five defects in the meantime, whatever the timing of
> the larger change.
>
> On the mapping side, Shao has offered in the decoupling thread to add the
> reference checks for mapping documents once the embedded versus standalone
> question is settled, so the three pieces would not overlap: the ontology
> section now, mapping references after #458, and the parser extraction
> replacing the internals.
>
> If you would rather fold the checks straight into the extraction PR, that
> works for me too and I will take the review side instead.
>
> Kyoung Min Kim
>
> 2026년 9월 28일 (월) 오후 11:35, Kurt Stirewalt <ku...@gmail.com>님이 작성:
>
>> Hi Kyoung,
>>
>> Yeah, that's a good observation. The ontology validations that are
>> currently performed in the converters should run in validate.py. In fact,
>> this was one of the motivators for my email last Thursday in the "Scope
>> and
>> maintenance of Ossie converters" thread. There, we suggested isolating
>> into
>> the main Ossie repo:
>>
>> -
>>
>> the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
>> -
>>
>> the DTOs / object model for Ossie documents — a canonical, typed,
>> versioned in-memory representation with parsing and serialization;
>> -
>>
>> the parser for the formula language that is used in the ontology spec
>> and the AST it produces;
>> -
>>
>> validation and conformance tooling (cf. the validation/ directory);
>> -
>>
>> a compatibility/conformance test kit containing example Ossie models
>> that cover the Ossie spec and that any converter can run against.
>>
>> If this sounds reasonable, my team and I can put together a PR that
>> extracts all aspects of ontology validation (including the formula parser)
>> out of the current converters and extend validate.py to call it.
>>
>>
>> Please advise, and thanks for the suggestion.
>>
>>
>> Kurt
>>
>> On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <ka...@gmail.com>
>> wrote:
>>
>> > Hi,
>> >
>> > validate.py is the tool the repo documents for checking an ontology
>> > document (--schema ontology/ontology.json), and the validation CI runs
>> > it. For core documents it goes beyond the schema: unique names,
>> > relationship references, column arity, SQL parsing. For ontology
>> > documents it stops at the schema, because every semantic check is
>> > guarded by the presence of `datasets`.
>> >
>> > I tried a small ontology document with five defects, one at a time, on
>> > current main. All five come back "Validation PASSED":
>> >
>> > - two components declaring the same concept name
>> > - `extends` naming a concept that is not declared
>> > - `identify_by` naming a relationship the concept does not have
>> > - a role whose concept is not declared
>> > - an `iri` QName whose prefix is not in `prefixes`, which ontology.md
>> > says is invalid
>> >
>> > The reference parser in converters/ontology rejects four of them with
>> > clear messages. The duplicate concept goes through there as well. So
>> > the rules exist, but only for people who run the converter, and CI runs
>> > validate.py.
>> >
>> > What I have in mind is the same kind of checks the core side already
>> > has, scoped to the `ontology` section: unique concept and relationship
>> > names, references from `extends`, `identify_by` and roles to declared
>> > concepts and relationships (built-ins like String and Any included),
>> > and QName prefixes against the `prefixes` map. I would leave
>> > `ontology_mappings` alone for now, since #458 is about to change where
>> > that lives. The author of #479 mentioned the prefix gap there and
>> > offered to open an issue for it, so that part can be folded in or
>> > tracked separately, whichever they prefer.
>> >
>> > Before I write it I would like a read on two things: whether
>> > validate.py is the right place for these at all, given the parser has
>> > its own copy of most of them, and whether that scope is right or should
>> > wait for the ontology spec to settle further. If it sounds reasonable I
>> > will put the PR together with tests under validation/tests.
>> >
>> > Thanks,
>> > Kyoung Min Kim
>> >
>>
>
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Kyoung Min Kim <ka...@gmail.com>.
Hi Kurt,
Thanks, that matches how I read it. Pulling the ontology validation and
the formula parser into the main repo and having validate.py call it is
the right end state, and I would be glad to review that PR when it comes.
Since the parser move depends on where the converter scope discussion
lands, my thought is to put the ontology-section checks into validate.py
first, as a small PR with tests, written as self-contained functions so
the extracted module can replace them without touching the callers. That
gets CI catching the five defects in the meantime, whatever the timing of
the larger change.
On the mapping side, Shao has offered in the decoupling thread to add the
reference checks for mapping documents once the embedded versus standalone
question is settled, so the three pieces would not overlap: the ontology
section now, mapping references after #458, and the parser extraction
replacing the internals.
If you would rather fold the checks straight into the extraction PR, that
works for me too and I will take the review side instead.
Kyoung Min Kim
2026년 9월 28일 (월) 오후 11:35, Kurt Stirewalt <ku...@gmail.com>님이 작성:
> Hi Kyoung,
>
> Yeah, that's a good observation. The ontology validations that are
> currently performed in the converters should run in validate.py. In fact,
> this was one of the motivators for my email last Thursday in the "Scope and
> maintenance of Ossie converters" thread. There, we suggested isolating into
> the main Ossie repo:
>
> -
>
> the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
> -
>
> the DTOs / object model for Ossie documents — a canonical, typed,
> versioned in-memory representation with parsing and serialization;
> -
>
> the parser for the formula language that is used in the ontology spec
> and the AST it produces;
> -
>
> validation and conformance tooling (cf. the validation/ directory);
> -
>
> a compatibility/conformance test kit containing example Ossie models
> that cover the Ossie spec and that any converter can run against.
>
> If this sounds reasonable, my team and I can put together a PR that
> extracts all aspects of ontology validation (including the formula parser)
> out of the current converters and extend validate.py to call it.
>
>
> Please advise, and thanks for the suggestion.
>
>
> Kurt
>
> On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <ka...@gmail.com> wrote:
>
> > Hi,
> >
> > validate.py is the tool the repo documents for checking an ontology
> > document (--schema ontology/ontology.json), and the validation CI runs
> > it. For core documents it goes beyond the schema: unique names,
> > relationship references, column arity, SQL parsing. For ontology
> > documents it stops at the schema, because every semantic check is
> > guarded by the presence of `datasets`.
> >
> > I tried a small ontology document with five defects, one at a time, on
> > current main. All five come back "Validation PASSED":
> >
> > - two components declaring the same concept name
> > - `extends` naming a concept that is not declared
> > - `identify_by` naming a relationship the concept does not have
> > - a role whose concept is not declared
> > - an `iri` QName whose prefix is not in `prefixes`, which ontology.md
> > says is invalid
> >
> > The reference parser in converters/ontology rejects four of them with
> > clear messages. The duplicate concept goes through there as well. So
> > the rules exist, but only for people who run the converter, and CI runs
> > validate.py.
> >
> > What I have in mind is the same kind of checks the core side already
> > has, scoped to the `ontology` section: unique concept and relationship
> > names, references from `extends`, `identify_by` and roles to declared
> > concepts and relationships (built-ins like String and Any included),
> > and QName prefixes against the `prefixes` map. I would leave
> > `ontology_mappings` alone for now, since #458 is about to change where
> > that lives. The author of #479 mentioned the prefix gap there and
> > offered to open an issue for it, so that part can be folded in or
> > tracked separately, whichever they prefer.
> >
> > Before I write it I would like a read on two things: whether
> > validate.py is the right place for these at all, given the parser has
> > its own copy of most of them, and whether that scope is right or should
> > wait for the ontology spec to settle further. If it sounds reasonable I
> > will put the PR together with tests under validation/tests.
> >
> > Thanks,
> > Kyoung Min Kim
> >
>
Re: [DISCUSS] Semantic checks for ontology documents in validate.py
Posted by Kurt Stirewalt <ku...@gmail.com>.
Hi Kyoung,
Yeah, that's a good observation. The ontology validations that are
currently performed in the converters should run in validate.py. In fact,
this was one of the motivators for my email last Thursday in the "Scope and
maintenance of Ossie converters" thread. There, we suggested isolating into
the main Ossie repo:
-
the spec itself (core-spec, ontology, JSON Schema / spec.yaml);
-
the DTOs / object model for Ossie documents — a canonical, typed,
versioned in-memory representation with parsing and serialization;
-
the parser for the formula language that is used in the ontology spec
and the AST it produces;
-
validation and conformance tooling (cf. the validation/ directory);
-
a compatibility/conformance test kit containing example Ossie models
that cover the Ossie spec and that any converter can run against.
If this sounds reasonable, my team and I can put together a PR that
extracts all aspects of ontology validation (including the formula parser)
out of the current converters and extend validate.py to call it.
Please advise, and thanks for the suggestion.
Kurt
On Mon, Sep 28, 2026 at 2:23 AM Kyoung Min Kim <ka...@gmail.com> wrote:
> Hi,
>
> validate.py is the tool the repo documents for checking an ontology
> document (--schema ontology/ontology.json), and the validation CI runs
> it. For core documents it goes beyond the schema: unique names,
> relationship references, column arity, SQL parsing. For ontology
> documents it stops at the schema, because every semantic check is
> guarded by the presence of `datasets`.
>
> I tried a small ontology document with five defects, one at a time, on
> current main. All five come back "Validation PASSED":
>
> - two components declaring the same concept name
> - `extends` naming a concept that is not declared
> - `identify_by` naming a relationship the concept does not have
> - a role whose concept is not declared
> - an `iri` QName whose prefix is not in `prefixes`, which ontology.md
> says is invalid
>
> The reference parser in converters/ontology rejects four of them with
> clear messages. The duplicate concept goes through there as well. So
> the rules exist, but only for people who run the converter, and CI runs
> validate.py.
>
> What I have in mind is the same kind of checks the core side already
> has, scoped to the `ontology` section: unique concept and relationship
> names, references from `extends`, `identify_by` and roles to declared
> concepts and relationships (built-ins like String and Any included),
> and QName prefixes against the `prefixes` map. I would leave
> `ontology_mappings` alone for now, since #458 is about to change where
> that lives. The author of #479 mentioned the prefix gap there and
> offered to open an issue for it, so that part can be folded in or
> tracked separately, whichever they prefer.
>
> Before I write it I would like a read on two things: whether
> validate.py is the right place for these at all, given the parser has
> its own copy of most of them, and whether that scope is right or should
> wait for the ontology spec to settle further. If it sounds reasonable I
> will put the PR together with tests under validation/tests.
>
> Thanks,
> Kyoung Min Kim
>