Other elements in Repository.xml

[onno <on...@onnos.com>]

Hiyall!

So far we had quite good stuff with using OJB - but since it's very limited
regarding validation and that stuff we've been
extending the user.xml (of repository) with several attributes so we can
have the whole data definition in 1 central place.
This was going fine until i add a new element which basically causes a
NullPointer to throw and OJB to stop. I though
it might be a DTD issue but it seems somewhere in the code :

getIdByTag(Unknown Source)

I know rigidity is always a plus but this duality of hey you can add
attributes but not elements i find mildly irritating. If you
are parsing and you encounter an unexpected element you should not be
casting fatal errors but throw a warn or ignore instead.

Saying that is there any mechanism which would allow me to do this without
having to change the code myself (which i do not
wish to do) an example would be:

<field-descriptor name="dkitExperience" column="fd_dkitExperience" id="38"
jdbc-type="VARCHAR" data-type="string" label="How would you rate your
experience of studying at DKIT?">
       <add-rule rule="list-rule">
  <item label="Fair" value="fair"/>
  <item label="Good" value="good"/>
  <item label="Excellent" value="excellent"/>
        </add-rule>
 </field-descriptor>

as you can see we use OJB mostly for convenience (save/store/retrieval) but
we in fact use our own mechanism which does quite
a few things extra...

Re: Other elements in Repository.xml

[onno <on...@onnos.com>]

> Now I see what you are asking for! You want to be allowed to add
> additional XML attributes and element tags, that are not part of the
> existing repository.dtd. Right?

Yep :) BTW i can add xml attributes already to all descriptors

 > If we would allow this, proving corretcness of a repository.xml file
> would be much more difficult as we could not rely on the standard
> validation scheme!

I agree 100% strictness is indeed desirable. Yet too much rigidity isn't
good either:
I think you use a SAX parser - approach: ie run down every element and then
do soemthing:
Although this should allow faster parsing for huge files , i tend to 'let'
the program do the sutff:

ie:    Iterator tClass = ooXPath.getChildIt(document, "class-descriptor");

and such strategy: ie the program 'knows' what to work with, rather then the
approach of feeding it.
again this might be not an ideal choice , but rather blandy this completely
circumvents ANY possibility
of getting invalid data. When it does find a "class-descriptor" it extracts
what it needs to run and then
ignores everything it doesnt.

This OJB seems to do as well but only for attributes inside 'known' elements

> Being very strict on the validity of the repository.xml has been *very*
> beneficial in the past. It allows to detect problems with the mapping
> very early. It allows to use standard XML editory (like XmlSpy) to
> validate the mapping file. And don't want to compromise these benefits.

Indeed. Yet if i change the DTD this still cause OJB to buckle: ie OJB
doesn't care DTD .
an 'Unkown' element is always handy (ie what is exactly wrong with ignoring
stuff if you give warnings?) as long
as all the XML that it uses is valid which is the crucial bit.

And why can i add any old attribute (ie <class-desc label="hello"
somehtng="dddd"/>) without errors: ie i'm glad it
does but strictly speaking this should violate the DTD. But i know why: coz
for attributes OJB also uses

Element.getAttribute(). and hence doesn't use the same strcit approach.

Again i fully understand why you do this, but surely you understand that it
isn't consistent. Please understand i'm not
slackign anything off here ok: OJB is one fine piece of work just making
comment that the more 'entrepid' (ok naughty)
users like to use. Also putting validation in OJB would be a nice addition:
ie stuff like enum, range checking could all be done in OJB with minor extra
code but this would prevent faulty data to be entered. The use of PB and
transactions is brilliant though.

> > Also name stuff like: <tag tag-name=""
> > tag-that="" /> is also a perfect example
> > of how NOT to use xml. (ie redundancy).

> We are simply extending OJB descriptors with a mechanism for user
> defined attribute-name / attribute-value mechanism.
> This is pretty much a standard way of using xml. E.G. the JDO.dtd has a
> very similar mechanism for "vendor-extensions". So I don't see what's so
> bad with this solution.

no there is of course nothing wrong with it, but i'm focusing on a few
syntaxis: ie: the use of an attribute ELEMENT rather then adding a a real
attribute baffles me: can OJB do this:

<attribute attribute-name="hello">some value</attribute>

if not that you might as well put the attributes inside the descriptor tags.

But my real sore (eye sore only is:)

<attribute attribute-name="hello" attribute-value="sss" />

why on earth call the attribute : attribute-name. if you are inside an
element it's of course the 'name' attribute of attribute Element (what a
sentence ;)) hence:

<attribute name="hello" value="hello"/> is much more logical also reads
easier and is in fact clearer to use in the while xml-spirit
(XPath /attribute[@name="hello"])

perhaps 'naming' it attribute is a bit awkward as well:

> Once you say yes to validation of XML files against a schema or DTD,
> your idea won't work!

true, but only if the system itself actually is as strict as the DTD.

> You can to this with OJB ReportQueries as well. See
> http://db.apache.org/ojb/query.html#report%20queries.

Yes we tried that, but we couldn't find a 'LIMIT' thingy (ie how to tell it
we want result 2000 -> 2050). I heard there
is a way of change part of the code - but i'm reluctant for such things.
Also the report_queries are quasi undocumented.
this way we are also capable of doing 'small' edits (ie stuff which doesn't
require full blown OJB store stuff) for multiple
'records' at the same time. But i do understand that this isn't OJB's job :)
(example we something need to change 1 'field' for many fields (thousands+)
and loading them all then storing - well you know - is unfeasible. Coz we
are aware of the Mapping we only allow this on 'pure' members (ie when it's
not reference/collection) but quite handy (required) nonetheless.

> Did you have a look at OJB FieldConversions? See
> http://db.apache.org/ojb/jdbc-types.html.

Yes. we use it in some cases (Date -> Calendar mostly) - however to add
'conversion' for each attribute is a bit awkard: ie perhaps in the future
you could add a set of named conversions.
(ie a piece of xml like this:

   <add-conversion from="java.lang.String" to="java.text.Calendar"
convertor="someClass which does that" />

the system is then aware that in the case that a String -> Calendar needs to
occur it just does it. no need to specify this. This
is the approach we use in our system when it finds that the Types don't
match.

Also i'm not aware that the conversions are nested capable:

Date -> Calendar / Calender->String / D->S (possible?)

Again our system creates these 'weird' conversions on the fly , but one can
overwrite them by adding a more specific one, allowing attribute 'Strict
Conversion' we tell the system that it cannot use automated conversions. Coz
we do have some weird generated conversions:

Calender -> String / String -> Boolean / WILL allow Calendar -> Boolean (and
vice versa). It works but of course is ridiculous but using 'strict' this
simply doesn't work and throws an error (ie the validation of the object
fails so that it cannot be stored and faul the Data)

> Using the slick OJB queries could safe you a lot of time! Maybe the
> "Rolling your own" approach is temptating first. But in the long term a
> o/r tool like OJB gives you a lot of added value.

indeed otherwise we wouldn't use it :). There is always room for improvement
of course:)

Again much appreciate your comments...

Re: Other elements in Repository.xml

[Thomas Mahler <th...@web.de>]

Hi onno,

<snip>
> 
> is allowed otherwise using attributes would be pointless. (i mean the
> attribute name-value is EXACTLY adding
> an attribute in the first place :).

Now I see what you are asking for! You want to be allowed to add 
additional XML attributes and element tags, that are not part of the 
existing repository.dtd. Right?

If we would allow this, proving corretcness of a repository.xml file 
would be much more difficult as we could not rely on the standard 
validation scheme!

Being very strict on the validity of the repository.xml has been *very* 
beneficial in the past. It allows to detect problems with the mapping 
very early. It allows to use standard XML editory (like XmlSpy) to 
validate the mapping file. And don't want to compromise these benefits.


> Also name stuff like: <tag tag-name=""
> tag-that="" /> is also a perfect example
> of how NOT to use xml. (ie redundancy).

We are simply extending OJB descriptors with a mechanism for user 
defined attribute-name / attribute-value mechanism.
This is pretty much a standard way of using xml. E.G. the JDO.dtd has a 
very similar mechanism for "vendor-extensions". So I don't see what's so 
bad with this solution.

Once you say yes to validation of XML files against a schema or DTD, 
your idea won't work!

> 
> 
>>There is also OJB api to work with these attributes.
> 
> 
> Well I already parse the user.xml myself (about 5 times faster dunno how
> it's done by OJB ?) so it's not really
> a problem on my part. 

It all depends on what is done in the parse phase. We are doing a lot. 
The metadata layer that is built up in the read phase is the "brain" of 
OJB, so it's no wonder that it takes some time.
As a mitigation to this long read phase we provide a mechanism to 
serialize the metadata. switching on this feature OJB first looks for a 
file containing a serialized version of the metadata. Of course loading 
this serialized structure is *much* faster than parsing!
With this solution parsing of repository.xml only occurs if no 
serialized repository was found.

> Again we just use OJB as a tool for loading/saving and
> because my friend is using it transparantly
> with JxPath/XForms (from Cocoon). Yet on client side we basically redid (not
> all of) all the sql generation, data managment so
> we could really big/fast selects (ie for 'overviews' which just need parts
> of data) without having to retrieve all the objects unnecessarily. 

You can to this with OJB ReportQueries as well. See 
http://db.apache.org/ojb/query.html#report%20queries.

> We use
> OJB cascading stuff for deletion though, if it occurs. Probably if we find
> time we will drop OJB altogether coz at the moment our system is aware of
> all things that OJB is (Except transparent M:N mapping), it's also quite a
> bit better when it comes to handlign to/from the SQL database (ie
> data-conversions are done transparently, and don't cause Fatals when a
> String -> int needs to be done). 

Did you have a look at OJB FieldConversions? See 
http://db.apache.org/ojb/jdbc-types.html.

> I've just been to busy to create the
> slighly more complex queries.

Using the slick OJB queries could safe you a lot of time! Maybe the 
"Rolling your own" approach is temptating first. But in the long term a 
o/r tool like OJB gives you a lot of added value.

cheers,
Thomas

> Thx very much though for again your fast reply - i actually circumvented the
> problem by creating some more data-types (a set of rules) outside the
> user.xml ..., but would be great to put them back where they belong...
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
> 
> 

Re: Other elements in Repository.xml

[onno <on...@onnos.com>]

> Hi agaian onno,

:)

> You can use user defined elements as in the following example.
>        <field-descriptor
>           name="isSelloutArticle"
>           column="Auslaufartikel"
>           jdbc-type="INTEGER"
>        >
>        <attribute attribute-name="color" attribute-value="green" />
>        <attribute attribute-name="size" attribute-value="small" />
>        </field-descriptor>

Yes i saw that - guess i just wondered if i had to use them. I'm just gonna
check if:

 <attribute attribute-name="test" attribute-value="green" >
   <myxml />
</attribute>

is allowed otherwise using attributes would be pointless. (i mean the
attribute name-value is EXACTLY adding
an attribute in the first place :). Also name stuff like: <tag tag-name=""
tag-that="" /> is also a perfect example
of how NOT to use xml. (ie redundancy).

> There is also OJB api to work with these attributes.

Well I already parse the user.xml myself (about 5 times faster dunno how
it's done by OJB ?) so it's not really
a problem on my part. Again we just use OJB as a tool for loading/saving and
because my friend is using it transparantly
with JxPath/XForms (from Cocoon). Yet on client side we basically redid (not
all of) all the sql generation, data managment so
we could really big/fast selects (ie for 'overviews' which just need parts
of data) without having to retrieve all the objects unnecessarily. We use
OJB cascading stuff for deletion though, if it occurs. Probably if we find
time we will drop OJB altogether coz at the moment our system is aware of
all things that OJB is (Except transparent M:N mapping), it's also quite a
bit better when it comes to handlign to/from the SQL database (ie
data-conversions are done transparently, and don't cause Fatals when a
String -> int needs to be done). I've just been to busy to create the
slighly more complex queries.

Thx very much though for again your fast reply - i actually circumvented the
problem by creating some more data-types (a set of rules) outside the
user.xml ..., but would be great to put them back where they belong...

Re: Other elements in Repository.xml

[Thomas Mahler <th...@web.de>]

Hi agaian onno,

You can use user defined elements as in the following example.
       <field-descriptor
          name="isSelloutArticle"
          column="Auslaufartikel"
          jdbc-type="INTEGER"
       >
       	 <attribute attribute-name="color" attribute-value="green" />
       	 <attribute attribute-name="size" attribute-value="small" />
       </field-descriptor>

There is also OJB api to work with these attributes.

cheers,
Thomas

onno wrote:
> Hiyall!
> 
> So far we had quite good stuff with using OJB - but since it's very limited
> regarding validation and that stuff we've been
> extending the user.xml (of repository) with several attributes so we can
> have the whole data definition in 1 central place.
> This was going fine until i add a new element which basically causes a
> NullPointer to throw and OJB to stop. I though
> it might be a DTD issue but it seems somewhere in the code :
> 
> getIdByTag(Unknown Source)
> 
> I know rigidity is always a plus but this duality of hey you can add
> attributes but not elements i find mildly irritating. If you
> are parsing and you encounter an unexpected element you should not be
> casting fatal errors but throw a warn or ignore instead.
> 
> Saying that is there any mechanism which would allow me to do this without
> having to change the code myself (which i do not
> wish to do) an example would be:
> 
> <field-descriptor name="dkitExperience" column="fd_dkitExperience" id="38"
> jdbc-type="VARCHAR" data-type="string" label="How would you rate your
> experience of studying at DKIT?">
>        <add-rule rule="list-rule">
>   <item label="Fair" value="fair"/>
>   <item label="Good" value="good"/>
>   <item label="Excellent" value="excellent"/>
>         </add-rule>
>  </field-descriptor>
> 
> as you can see we use OJB mostly for convenience (save/store/retrieval) but
> we in fact use our own mechanism which does quite
> a few things extra...
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: ojb-user-unsubscribe@db.apache.org
> For additional commands, e-mail: ojb-user-help@db.apache.org
> 
>