Problem with the embed-version

[Mario Siegenthaler <ms...@inventsoft.ch>]

Hi
We're using CVS-Version (post 1.0) of Xindice in various projects and it
works great (xml-rpc client). Thank you guys!
Now we run into a big problem while switching a project to the embed
version of xindice. It seems to mess up the symbols table since my
documents (randomly) interchange their tagnames. Example:

I've a document document1 like this:

<Person><Name>Mario Siegenthaler</Name><City>Berne</City></Person>

and a document2 like this:
<Lock><Timestamp>221321412</Timestamp><User>MSN</User><Resource>abcde</R
esource></Lock>

When I save those documents into xindice (first document1 into another
collection than document2, but that doesn't seem to matter) and retrieve
them again from xindice, document1 looks like the following (exact look
varies from time to time):

<Lock><Timestame>Mario Siegenthaler</Timestamp><User>Berne</User></Lock>

This is very strange, since the "tomcat"-version works fine (just
changing the xmldb:xindice urls). I switched to the
filesystem-(reference)-implementation (which also works fine btw) for
now, because we have to produce a version.. But I'd like very much to
stay with xindice.

I tried various things like changing some of the embed.CollectionImpl
methods aso, but I never got a better result.. Can anybody help me with
this? I don't (yet) have the insight into the "storing"-part of xindice
so maybe one of you experts have a better idea of the problem.

Regards
Mario Siegenthaler



BTW: I developed an Avalon-component to start/shutdown/configure the
embed-version. If anybody is interested I'd be happy to send it, since
it's more flexible than the 'basic' embed-version (especially
configuration-wise).

Re: Problem with the embed-version

[Terry Rosenbaum <Te...@amicas.com>]

The symbols are stored in a system collection.
I believe there is one system-wide symbol table.

When documents are compressed, the tags are converted
to symbol indexes. If a tag exists in the system symbol
table, the index of that tag value is used to represent the
tag. If the symbol is not present in the table, it is added
and assigned an index. The compressed document form
contains the indexes where the tags would be (thus
compressing by representing variable length tag
strings as integers (or was it shorts -- I don't recall for sure)).
Here is where things can go bad. I have seen at least one
place in the xindice code where the parser was provided with a
new empty symbol table instead of a copy of the
system's existing symbol table. I saw this when I
created a collection with compression="false" and
then tried to index the collection. The indexer was
looking up symbols in an empty symbol table and thus,
not finding them, was making new entries. As I recall,
the system symbol table ended up with multiple entries
per tag afterward. Or, perhaps, it was just that the
indexes in the stored compressed document did not
match what was in the system symbol table resulting
in tag scrambling like you report. e.g. if tag <Lock>
got stored once with index 0 and tag <Person> got stored
later with index 6 but later a <Person> document got compressed
with an empty initial symbol table the <Person> tags in the compressed
document would be stored encoded as 0 and not 6. Later, perhaps
properly decompressed (using the system symbol collection correctly),
the 0 gets translated into <Lock> instead of <Person> because
index 0 in the system symbol table says 0 is <Lock>, not <Person>.

When a compressed document (from storage)
is reconstituted, the decompressor is (should be?) initialized with a copy
of the system symbol table. The integer representation
of the tags is turned back into text (decompression).

Perhaps you're seeing a similar problem. The problem I saw
never occurs as long as you don't try to disable document
compresion. But, I've never tried out your scenario.

-Terry

Mario Siegenthaler wrote:

>Hi
>We're using CVS-Version (post 1.0) of Xindice in various projects and it
>works great (xml-rpc client). Thank you guys!
>Now we run into a big problem while switching a project to the embed
>version of xindice. It seems to mess up the symbols table since my
>documents (randomly) interchange their tagnames. Example:
>
>I've a document document1 like this:
>
><Person><Name>Mario Siegenthaler</Name><City>Berne</City></Person>
>
>and a document2 like this:
><Lock><Timestamp>221321412</Timestamp><User>MSN</User><Resource>abcde</R
>esource></Lock>
>
>When I save those documents into xindice (first document1 into another
>collection than document2, but that doesn't seem to matter) and retrieve
>them again from xindice, document1 looks like the following (exact look
>varies from time to time):
>
><Lock><Timestame>Mario Siegenthaler</Timestamp><User>Berne</User></Lock>
>
>This is very strange, since the "tomcat"-version works fine (just
>changing the xmldb:xindice urls). I switched to the
>filesystem-(reference)-implementation (which also works fine btw) for
>now, because we have to produce a version.. But I'd like very much to
>stay with xindice.
>
>I tried various things like changing some of the embed.CollectionImpl
>methods aso, but I never got a better result.. Can anybody help me with
>this? I don't (yet) have the insight into the "storing"-part of xindice
>so maybe one of you experts have a better idea of the problem.
>
>Regards
>Mario Siegenthaler
>
>
>
>BTW: I developed an Avalon-component to start/shutdown/configure the
>embed-version. If anybody is interested I'd be happy to send it, since
>it's more flexible than the 'basic' embed-version (especially
>configuration-wise).
>  
>