Threading and CollectionManagementService

["David J. Thomson" <dt...@eecs.tufts.edu>]

On Wed, 10 Sep 2003, Vadim Gritsenko wrote:

> David J. Thomson wrote:
>
> >Also, in my performance testing, I'm trying to create subcollections in a
> >nested loop. Is there some reason why the subcollections are becoming
> >corrupted this way? I'm closing the main collection before trying to open
> >it and create a subcollection under it, but it doesn't seem to matter.
> >
> >BTW, this is all with the embedded version, in case it matters.
> >
> >
>
> If you can consistenlty reproduce the problem with some simple java
> code, please go and file a bug with test case to bugzilla. If you know
> how to fix it -- don't go, run and file a patch to bugzilla! :)
>
> Vadim
>
>
>

Well, I think I've *almost* reproduced the problem consistently. I'm
afraid it has something to do with threading and having multiple
concurrent instances of a collection. It always seems to fail when trying
to instantiate a new CollectionManagementService, which returns null if
another open database inside another thread has already gotten a service.

Any ideas at all?

Thank you.

David

Collection numbers: Here's what's going on

["David J. Thomson" <dt...@eecs.tufts.edu>]

Hello all,

First of all, I'm surprised other people haven't run into this kind of
problem. I have one collection with about 95 subcollections, each of which
has four subcollections. It kills my system after a little while because
it runs out of file descriptors. Java gives all sorts of errors about
having too many open files, after I've already increased the number to
Linux's system max of 1048576. Not only that, but on another occasion, it
somehow corrupted the database when I ran out of file descriptors, which
was making it appear as though the problem was something else. I thought
there was a concurrency problem because one of the collections was
corrupted, but it appears as though this is it. Has anyone else dealt with
this? Can I please take a poll of how many collections people have and how
many documents in each. I mean, most databases can handle hundreds of
thousands of records for tables, so I don't know what to do here.

Thanks,
David

Collection numbers: Here's what's going on

["David J. Thomson" <dt...@eecs.tufts.edu>]

Hello all,

First of all, I'm surprised other people haven't run into this kind of
problem. I have one collection with about 95 subcollections, each of which
has four subcollections. It kills my system after a little while because
it runs out of file descriptors. Java gives all sorts of errors about
having too many open files, after I've already increased the number to
Linux's system max of 1048576. Not only that, but on another occasion, it
somehow corrupted the database when I ran out of file descriptors, which
was making it appear as though the problem was something else. I thought
there was a concurrency problem because one of the collections was
corrupted, but it appears as though this is it. Has anyone else dealt with
this? Can I please take a poll of how many collections people have and how
many documents in each. I mean, most databases can handle hundreds of
thousands of records for tables, so I don't know what to do here.

Thanks,
David

Threading and Collections

["David J. Thomson" <dt...@eecs.tufts.edu>]

Hello all,

I've been working on a problem for quite a while now and I would greatly
appreciate any insight. I'm using 1.1 b1 with the embed driver with jdk
1.4.1.

Essentially, I have one class that queries the database. It instantiates
it as such (exception handling removed):

Collection sc = null;
Database maindb = null;

         String driver =
"org.apache.xindice.client.xmldb.embed.DatabaseImpl";
         Class c = Class.forName(driver);

         maindb = (Database) c.newInstance();
         DatabaseManager.registerDatabase(maindb);


	   sc =
DatabaseManager.getCollection("xmldb:xindice-embed:///db/"+[collection_name]);


I have many threads trying to do this at once, and then querying it with
something like sc.listResources();.

Howevever, if one set of threads is using one collection, and another set
of threads is using another, they conflict and I keep getting a
NullPointerException. Do I need to make some sort of mutex locking to
ensure that only one collection is open at once? I was under the
impression that this wasn't so, but it appears otherwise. Thank you for
your advice.

Kind regards,
David




On Thu, 11 Sep 2003, Vadim Gritsenko wrote:

> David J. Thomson wrote:
>
> >Well, I think I've *almost* reproduced the problem consistently. I'm
> >afraid it has something to do with threading and having multiple
> >concurrent instances of a collection. It always seems to fail when trying
> >to instantiate a new CollectionManagementService, which returns null if
> >another open database inside another thread has already gotten a service.
> >
> >
>
> So, if I set off two-three threads simultaneously to get collection
> management service, it should fail? Good candidate for the unit test then.
>
>
> >Any ideas at all?
> >
> >
>
> No, I'm not that deep into xindice internals yet.
>
> Vadim
>
>
>

Threading and Collections

["David J. Thomson" <dt...@eecs.tufts.edu>]

Hello all,

I've been working on a problem for quite a while now and I would greatly
appreciate any insight. I'm using 1.1 b1 with the embed driver with jdk
1.4.1.

Essentially, I have one class that queries the database. It instantiates
it as such (exception handling removed):

Collection sc = null;
Database maindb = null;

         String driver =
"org.apache.xindice.client.xmldb.embed.DatabaseImpl";
         Class c = Class.forName(driver);

         maindb = (Database) c.newInstance();
         DatabaseManager.registerDatabase(maindb);


	   sc =
DatabaseManager.getCollection("xmldb:xindice-embed:///db/"+[collection_name]);


I have many threads trying to do this at once, and then querying it with
something like sc.listResources();.

Howevever, if one set of threads is using one collection, and another set
of threads is using another, they conflict and I keep getting a
NullPointerException. Do I need to make some sort of mutex locking to
ensure that only one collection is open at once? I was under the
impression that this wasn't so, but it appears otherwise. Thank you for
your advice.

Kind regards,
David




On Thu, 11 Sep 2003, Vadim Gritsenko wrote:

> David J. Thomson wrote:
>
> >Well, I think I've *almost* reproduced the problem consistently. I'm
> >afraid it has something to do with threading and having multiple
> >concurrent instances of a collection. It always seems to fail when trying
> >to instantiate a new CollectionManagementService, which returns null if
> >another open database inside another thread has already gotten a service.
> >
> >
>
> So, if I set off two-three threads simultaneously to get collection
> management service, it should fail? Good candidate for the unit test then.
>
>
> >Any ideas at all?
> >
> >
>
> No, I'm not that deep into xindice internals yet.
>
> Vadim
>
>
>

Re: Threading and CollectionManagementService

[Vadim Gritsenko <va...@verizon.net>]

David J. Thomson wrote:

>Well, I think I've *almost* reproduced the problem consistently. I'm
>afraid it has something to do with threading and having multiple
>concurrent instances of a collection. It always seems to fail when trying
>to instantiate a new CollectionManagementService, which returns null if
>another open database inside another thread has already gotten a service.
>  
>

So, if I set off two-three threads simultaneously to get collection 
management service, it should fail? Good candidate for the unit test then.


>Any ideas at all?
>  
>

No, I'm not that deep into xindice internals yet.

Vadim