Collada implementation strategy?

[Petter Måhlén <pm...@ncsoft.com>]

Hi,

 

I'm working on figuring out the best way to implement a partially complete Collada (a format used for representing 3D model data, primarily used in games, see www.khronos.org/collada) importer. I'm thinking of doing this in two steps:

 

1.       Reading the XML data into memory

2.       Traversing the document tree, instantiating correct 3D entities based on what I find, and adding them to a scenegraph

 

There are some requirements on the implementation:

1.       It shouldn't be complete - Collada is really complex and we won't need all the features, so a complete implementation would be a waste of time.

2.       It needs to be really good at reporting errors - due to the complexity of the format, and the fact that we don't want to be restricted by a poor importer implementation, we are likely not to be able to define up front exactly which features of Collada we want to support. So when the importer encounters something it cannot understand, I want it to be able to say that 'I cannot handle feature X, which is used on line Y, column Z in the input file'.

3.       It needs to be really easy to add support for new features of Collada. Basically, the strategy is that when our great error reporting tells us that an artist tries to use a feature the importer cannot handle, it should be easy for us to add support for the feature without breaking the overall architecture.

4.       It needs to handle XPointers. A common way to reduce the amount of data in a Collada file is to define a library of some kind of graphical entity type, and then refer to that entity using an id (there are multiple ways this is done, unfortunately) when an instance should be created.

 

I've been looking at using some mapping tool like Castor or JiBX to get the data into memory. Castor doesn't seem really suitable (unable to figure out a way to get line numbers), whereas JiBX has pretty nifty support for getting information about the original source. However, JiBX doesn't make it easy to resolve references to other Collada elements, since the most normal way to use JiBX is to destroy the original document structure and create a more easily-used one in Java. Doing so also helps make step 2 much simpler, since the tree structure to navigate can be made far easier than the full XML document structure.

 

I've also been looking at making step 1 be simply reading in the document using a standard parser and getting to a DOM version of it. That makes it really easy to complete step 1, but it seems to complicate the traversal in step 2, and the main drawback is that I haven't been able to figure out a way to keep track of the original line numbers. They are being reported in case there is a parse error, but the situation I want to handle is when the document is syntactically (and semantically) correct, but the importer doesn't know how to handle the Collada feature that is configured in the correctly defined data.

 

Another possible advantage of using a DOM could be that it would help me resolve references to other elements - I think, only I haven't been able to figure out how to do that yet. Should that be possible?

 

Does anybody have any suggestions for how to approach this? Any help would be much appreciated.

 

Cheers,

 

Petter

Re: Collada implementation strategy?

["Eric J. Schwarzenbach" <Er...@wrycan.com>]

If performance, or scalability in terms of memory usage with large files
is at all a concern, I'd recommend using SAX rather than DOM or one of
those binding frameworks. To accommodate requirement #3, you'd need to
do something like create your own system of handler classes for collada
features, and, assuming you can map such features to simple xpaths,
track the xpath to the node currently being handled in the SAX content
handler and delegate to those collaba handlers as appropriate.

If you don't care about those issues (I can imagine the memory needs of
the scenegraph you're building might dwarf that of the DOM structure for
its source anyway), and if (like most OO programmers) you prefer the
convenience of an object tree to getting your head around the SAX event
model, there are other binding frameworks worth looking at also, like
XMLBeans.

Eric


Petter Måhlén wrote:
>
> Hi,
>
>  
>
> I’m working on figuring out the best way to implement a partially
> complete Collada (a format used for representing 3D model data,
> primarily used in games, see www.khronos.org/collada
> <http://www.khronos.org/collada>) importer. I’m thinking of doing this
> in two steps:
>
>  
>
> 1.       Reading the XML data into memory
>
> 2.       Traversing the document tree, instantiating correct 3D
> entities based on what I find, and adding them to a scenegraph
>
>  
>
> There are some requirements on the implementation:
>
> 1.       It shouldn’t be complete – Collada is really complex and we
> won’t need all the features, so a complete implementation would be a
> waste of time.
>
> 2.       It needs to be really good at reporting errors – due to the
> complexity of the format, and the fact that we don’t want to be
> restricted by a poor importer implementation, we are likely not to be
> able to define up front exactly which features of Collada we want to
> support. So when the importer encounters something it cannot
> understand, I want it to be able to say that ‘I cannot handle feature
> X, which is used on line Y, column Z in the input file’.
>
> 3.       It needs to be really easy to add support for new features of
> Collada. Basically, the strategy is that when our great error
> reporting tells us that an artist tries to use a feature the importer
> cannot handle, it should be easy for us to add support for the feature
> without breaking the overall architecture.
>
> 4.       It needs to handle XPointers. A common way to reduce the
> amount of data in a Collada file is to define a library of some kind
> of graphical entity type, and then refer to that entity using an id
> (there are multiple ways this is done, unfortunately) when an instance
> should be created.
>
>  
>
> I’ve been looking at using some mapping tool like Castor or JiBX to
> get the data into memory. Castor doesn’t seem really suitable (unable
> to figure out a way to get line numbers), whereas JiBX has pretty
> nifty support for getting information about the original source.
> However, JiBX doesn’t make it easy to resolve references to other
> Collada elements, since the most normal way to use JiBX is to destroy
> the original document structure and create a more easily-used one in
> Java. Doing so also helps make step 2 much simpler, since the tree
> structure to navigate can be made far easier than the full XML
> document structure.
>
>  
>
> I’ve also been looking at making step 1 be simply reading in the
> document using a standard parser and getting to a DOM version of it.
> That makes it really easy to complete step 1, but it seems to
> complicate the traversal in step 2, and the main drawback is that I
> haven’t been able to figure out a way to keep track of the original
> line numbers. They are being reported in case there is a parse error,
> but the situation I want to handle is when the document is
> syntactically (and semantically) correct, but the importer doesn’t
> know how to handle the Collada feature that is configured in the
> correctly defined data.
>
>  
>
> Another possible advantage of using a DOM could be that it would help
> me resolve references to other elements – I think, only I haven’t been
> able to figure out how to do that yet. Should that be possible?
>
>  
>
> Does anybody have any suggestions for how to approach this? Any help
> would be much appreciated.
>
>  
>
> Cheers,
>
>  
>
> Petter
>

---------------------------------------------------------------------
To unsubscribe, e-mail: j-users-unsubscribe@xerces.apache.org
For additional commands, e-mail: j-users-help@xerces.apache.org

Re: Collada implementation strategy?

[Michael Glavassevich <mr...@ca.ibm.com>]

Hi Petter,

There's a sample that comes with Xerces called dom.DOMAddLines [1] which
shows how you can add line/column information to the DOM using
Node.setUserData() [2] and read it back with Node.getUserData() [3]. Note
that this requires an extension to the DOM parser implementation. There's
no standard way of doing this and if you start making modifications to the
DOM after loading it the line/column information you initially stored
gradually becomes meaningless.

Thanks.

[1] http://xerces.apache.org/xerces2-j/samples-dom.html#DOMAddLines
[2]
http://xerces.apache.org/xerces2-j/javadocs/api/org/w3c/dom/Node.html#setUserData(java.lang.String,%20java.lang.Object,%20org.w3c.dom.UserDataHandler)
[3]
http://xerces.apache.org/xerces2-j/javadocs/api/org/w3c/dom/Node.html#getUserData(java.lang.String)

Michael Glavassevich
XML Parser Development
IBM Toronto Lab
E-mail: mrglavas@ca.ibm.com
E-mail: mrglavas@apache.org

Petter Måhlén <pm...@ncsoft.com> wrote on 09/02/2008 10:33:46 AM:

> Hi,
>
> I?m working on figuring out the best way to implement a partially
> complete Collada (a format used for representing 3D model data,
> primarily used in games, see www.khronos.org/collada) importer. I?m
> thinking of doing this in two steps:
>
> 1.       Reading the XML data into memory
> 2.       Traversing the document tree, instantiating correct 3D
> entities based on what I find, and adding them to a scenegraph
>
> There are some requirements on the implementation:
> 1.       It shouldn?t be complete ? Collada is really complex and we
> won?t need all the features, so a complete implementation would be a
> waste of time.
> 2.       It needs to be really good at reporting errors ? due to the
> complexity of the format, and the fact that we don?t want to be
> restricted by a poor importer implementation, we are likely not to
> be able to define up front exactly which features of Collada we want
> to support. So when the importer encounters something it cannot
> understand, I want it to be able to say that ?I cannot handle
> feature X, which is used on line Y, column Z in the input file?.
> 3.       It needs to be really easy to add support for new features
> of Collada. Basically, the strategy is that when our great error
> reporting tells us that an artist tries to use a feature the
> importer cannot handle, it should be easy for us to add support for
> the feature without breaking the overall architecture.
> 4.       It needs to handle XPointers. A common way to reduce the
> amount of data in a Collada file is to define a library of some kind
> of graphical entity type, and then refer to that entity using an id
> (there are multiple ways this is done, unfortunately) when an
> instance should be created.
>
> I?ve been looking at using some mapping tool like Castor or JiBX to
> get the data into memory. Castor doesn?t seem really suitable
> (unable to figure out a way to get line numbers), whereas JiBX has
> pretty nifty support for getting information about the original
> source. However, JiBX doesn?t make it easy to resolve references to
> other Collada elements, since the most normal way to use JiBX is to
> destroy the original document structure and create a more easily-
> used one in Java. Doing so also helps make step 2 much simpler,
> since the tree structure to navigate can be made far easier than the
> full XML document structure.
>
> I?ve also been looking at making step 1 be simply reading in the
> document using a standard parser and getting to a DOM version of it.
> That makes it really easy to complete step 1, but it seems to
> complicate the traversal in step 2, and the main drawback is that I
> haven?t been able to figure out a way to keep track of the original
> line numbers. They are being reported in case there is a parse
> error, but the situation I want to handle is when the document is
> syntactically (and semantically) correct, but the importer doesn?t
> know how to handle the Collada feature that is configured in the
> correctly defined data.
>
> Another possible advantage of using a DOM could be that it would
> help me resolve references to other elements ? I think, only I
> haven?t been able to figure out how to do that yet. Should that be
possible?
>
> Does anybody have any suggestions for how to approach this? Any help
> would be much appreciated.
>
> Cheers,
>
> Petter