Introduction followed by Dumb, Obvious Developer Question

[Charles Severance <cs...@umich.edu>]

<blah-blah>
Hello to the JackRabbit community.  My name is Charles Severance, the Chief
Architect of the Sakai project - www.sakaiproject.org -
collab.sakaiproject.org has a sample system up and running.

I am kicking the tires on Jackrabbit for possible adoption as the core of
our storage within Sakai (An open source collaboration and learning
management system).  If we were to jump in here with Jackrabbit, we would
probably write/improve/contribute an open source high performance
implementation to scale to 100000 users at a single site (yikes).   We are
already well underway to building a fast, scalable, hierarchical
authorizaion system using Hibernate.

I have 2-3 developers to dedicate to this effort for the next six months if
we adopt Jackrabbit because our alternative is to write this ourselves from
scratch with our own API.
</blah-blah>

I am trying to do a quick test integration over the weekend and almost have
it running - but all my unit tests are failing.  I checked out the code,
upgraded my maven plugin, and ran "maven jar" just like the README says.
Everything compiles fine except the Unit tests all fail with:

[junit] Running org.apache.jackrabbit.test.observation.TestAll
[junit] Exception in thread "main" java.lang.NoClassDefFoundError:
org/w3c/dom/ranges/DocumentRange
[junit]     at java.lang.ClassLoader.defineClass0(Native Method)
[junit]     at java.lang.ClassLoader.defineClass(ClassLoader.java:539)

[snip]

[junit]     at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestR
unner.java:536)
[junit] [ERROR] TEST org.apache.jackrabbit.test.search.TestAll FAILED

I am betting that this has a really simple solution :)

Thanks in advance.

Charles Severance
www.dr-chuck.com
csev@umich.edu

Re: Introduction followed by Dumb, Obvious Developer Question

[Charles Severance <cs...@umich.edu>]

On 2/7/05 12:31 PM, "Stefano Mazzocchi" <st...@apache.org> wrote:

> Charles, pleased to meet you. Happy to see the Sakai project thinking
> about adopting the JCR API.
> 
> I wear three hats here, one is my apache hat, one is my JSR-170 expert
> group member hat (very similar hats at the end of the day) and one is my
> MIT hat.

Perhaps we could send you a Sakai hat in case your head gets cold.  Are you
in Boston?
 
> At MIT I'm part of the Digital Library Research Group and I've been
> pushing JSR170 with the OKI people (David also gave them a talk last
> year, when we held one of the JSR170 face2face meetings here in
> Cambridge) and I also push it for DSpace 2, even if with moderate
> traction so far, I must say. Having Sakai chosing JackRabbit will give
> me a very strong push for JCR into DSpace2, which might also be a useful
> thing in the global scheme of educational technologies in general.

This is good to know.
 
> So, count on me for help as I'm really interested in having Sakai use
> JackRabbit.

Excellent.  I have been a closet fan of 170 from the time I heard about it
well back in the CHEF project. It has always seemed tantalizingly out of
reach (kind of like JSR-168  *still* is :) ).  Jackrabbit and an active
community around jackrabbit makes it seem much closer...

> As for your problem, have you checked your classpath for weird
> xml-api.jar files? what JVM/OS are you running?

Yes - I am pretty sure that it was an Evil version of Xerxces that got in
front of the good version in the class loader - solved by deleting all and
restarting from the beginning - what else?

/Chuck

Re: Introduction followed by Dumb, Obvious Developer Question

[Stefano Mazzocchi <st...@apache.org>]

Charles Severance wrote:
> <blah-blah>
> Hello to the JackRabbit community.  My name is Charles Severance, the Chief
> Architect of the Sakai project - www.sakaiproject.org -
> collab.sakaiproject.org has a sample system up and running.
> 
> I am kicking the tires on Jackrabbit for possible adoption as the core of
> our storage within Sakai (An open source collaboration and learning
> management system).  If we were to jump in here with Jackrabbit, we would
> probably write/improve/contribute an open source high performance
> implementation to scale to 100000 users at a single site (yikes).   We are
> already well underway to building a fast, scalable, hierarchical
> authorizaion system using Hibernate.
> 
> I have 2-3 developers to dedicate to this effort for the next six months if
> we adopt Jackrabbit because our alternative is to write this ourselves from
> scratch with our own API.
> </blah-blah>

Charles, pleased to meet you. Happy to see the Sakai project thinking 
about adopting the JCR API.

I wear three hats here, one is my apache hat, one is my JSR-170 expert 
group member hat (very similar hats at the end of the day) and one is my 
MIT hat.

At MIT I'm part of the Digital Library Research Group and I've been 
pushing JSR170 with the OKI people (David also gave them a talk last 
year, when we held one of the JSR170 face2face meetings here in 
Cambridge) and I also push it for DSpace 2, even if with moderate 
traction so far, I must say. Having Sakai chosing JackRabbit will give 
me a very strong push for JCR into DSpace2, which might also be a useful 
thing in the global scheme of educational technologies in general.

So, count on me for help as I'm really interested in having Sakai use 
JackRabbit.

As for your problem, have you checked your classpath for weird 
xml-api.jar files? what JVM/OS are you running?

-- 
Stefano.

Re: Never Mind - Solved it myself Re: Introduction followed by Dumb, Obvious Developer Question

[Charles Severance <cs...@umich.edu>]

On 2/7/05 1:20 PM, "David Nuescheler" <da...@gmail.com> wrote:
> it really sounds like this could be the perfect jcr & jackrabbit
> showcase, let me assure at least from my end that you have
> my full support and let me know if there is anything that we can
> do to help your progress...
> 
> regards,
> david

David,

Thanks - it is quite likely that we will need help.  Once we decide and
launch the development effort, Sakai will be on a track where "failure is
not an option and we have no plan B". :).

/Chuck

Re: Never Mind - Solved it myself Re: Introduction followed by Dumb, Obvious Developer Question

[David Nuescheler <da...@gmail.com>]

hi chuck,

thanks for sharing all the interesting information.

> We will have plenty of writes.  Likely between 10-20% writes overall.  We
> may end up storing things like chat messages in JCR.  We will have
> applications that store material in properties, and place observers on
> changes in content to provide real-time interactions for things like chat.
cool, sounds like a perfect match for a fully jcr compliant repository.

> If this is something that is not so good for JR, we should know that.  Make
> sure to separate the API issues from the implementation issues - we can add
> our own implementation to address performance issues.  
> If we picked JCR/JR, we would not put it into production until June.
of course the JCR as API itself has been designed to handle exactly
applications like the one you describe. with respect to jackrabbit
as the implementation believe that with a decent persistence 
manager you should be in the clear. as you may have picked up 
in this list a number of people have already conducted load tests
(mostly writing). i personally find the performance more than satisfactory: 
for example the "1-million-nodes-test" was not a very big hurdle for 
jackrabbit itself and was completed in less than an hour. 
yesterday we tested an mp3 library import of 1gb which 
was completed less than 6 minutes on a single cpu intel box to
test the blob behaviour.

to sum it up from my perspective:

1) jcr is specifically designed to handle exactly what you describe.
i think sakai could be a perfect jcr showcase.

2) jackrabbit's design should also supports the metrics you describe.
you may have to tweak the persistence managers (maybe not even), 
and maybe configure the search index properly. of course the
"content-design" (nodetypes) may have an impact too.

> One early medium application of our software (current version at University
> of Michigan) is 20,000 users, 100GB, 700 simultaneous users.  This is in
> daily and continuous production.  Our next large deployment is Indiana
> University which will likely be 500GB+, 100,000 users, 1500+ simultaneous
> users.  We also expect 20-100 smaller installations with 100-5000 users
> within a year.
> We do performance testing as a matter of course, and will only put stress on
> the implementation when we are confident.
i would be very interested in conducting those tests, maybe a generic
opensource "jcr-content-repository-load-test" could be the result of this 
excercise?

> > with respect to the hierarchical authorization, i would certainly be
> > interested to find out if the "soon-to-come" built-in native jackrabbit
> > authorization is going to suit your needs, both functionality and
> > performance-wise.
> If we get started, our implementations will be done in public and
> open-source from their first lines of code.  I look forward to some good
> native JR implementations because they can bootstrap our process quite a
> bit.  It is much better to start with something working and speed it up.
> Our entire project is Jakarta-style license so we can only include other
> Jakarta-style licensed material in our distribution.

it really sounds like this could be the perfect jcr & jackrabbit 
showcase, let me assure at least from my end that you have 
my full support and let me know if there is anything that we can 
do to help your progress...

regards,
david

Re: Never Mind - Solved it myself Re: Introduction followed by Dumb, Obvious Developer Question

[Charles Severance <cs...@umich.edu>]

On 2/6/05 4:31 PM, "David Nuescheler" <da...@gmail.com> wrote:

> sounds great.  i would assume that your users would mostly read from
> the repository? jackrabbit should be well suited for that.  do you have
> any expectations with respect to how much content you are going to
> store in the repository? or how frequently content is written to
> the repository?

We will have plenty of writes.  Likely between 10-20% writes overall.  We
may end up storing things like chat messages in JCR.  We will have
applications that store material in properties, and place observers on
changes in content to provide real-time interactions for things like chat.

If this is something that is not so good for JR, we should know that.  Make
sure to separate the API issues from the implementation issues - we can add
our own implementation to address performance issues.  If we picked JCR/JR,
we would not put it into production until June.

One early medium application of our software (current version at University
of Michigan) is 20,000 users, 100GB, 700 simultaneous users.  This is in
daily and continuous production.  Our next large deployment is Indiana
University which will likely be 500GB+, 100,000 users, 1500+ simultaneous
users.  We also expect 20-100 smaller installations with 100-5000 users
within a year.

We do performance testing as a matter of course, and will only put stress on
the implementation when we are confident.

> with respect to the hierarchical authorization, i would certainly be
> interested to find out if the "soon-to-come" built-in native jackrabbit
> authorization is going to suit your needs, both functionality and
> performance-wise.

If we get started, our implementations will be done in public and
open-source from their first lines of code.  I look forward to some good
native JR implementations because they can bootstrap our process quite a
bit.  It is much better to start with something working and speed it up.

Our entire project is Jakarta-style license so we can only include other
Jakarta-style licensed material in our distribution.

/Chuck

Re: Never Mind - Solved it myself Re: Introduction followed by Dumb, Obvious Developer Question

[David Nuescheler <da...@gmail.com>]

hi chuck,

> I upgraded to the latest maven 1.0.2, whacked my maven repo, re-checked out
> JCR and voila - happy little unit tests run successfully.
> I will save my first dumb obvious question for later.
cool, looking forward to further discussions or comments from your end.

> > <blah-blah>
> > Hello to the JackRabbit community.  My name is Charles Severance, the Chief
> > Architect of the Sakai project - www.sakaiproject.org -
> > collab.sakaiproject.org has a sample system up and running.
> >
> > I am kicking the tires on Jackrabbit for possible adoption as the core of our
> > storage within Sakai (An open source collaboration and learning management
> > system).  If we were to jump in here with Jackrabbit, we would probably
> > write/improve/contribute an open source high performance implementation to
> > scale to 100000 users at a single site (yikes).   We are already well underway
> > to building a fast, scalable, hierarchical authorizaion system using
> > Hibernate.
> >
> > I have 2-3 developers to dedicate to this effort for the next six months if we
> > adopt Jackrabbit because our alternative is to write this ourselves from
> > scratch with our own API.
> > </blah-blah>
sounds great.  i would assume that your users would mostly read from 
the repository? jackrabbit should be well suited for that.  do you have
any expectations with respect to how much content you are going to 
store in the repository? or how frequently content is written to 
the repository?

with respect to the hierarchical authorization, i would certainly be 
interested to find out if the "soon-to-come" built-in native jackrabbit
authorization is going to suit your needs, both functionality and 
performance-wise.

regards,
david

Never Mind - Solved it myself Re: Introduction followed by Dumb, Obvious Developer Question

[Charles Severance <cs...@umich.edu>]

Sorry for the interruption.

I upgraded to the latest maven 1.0.2, whacked my maven repo, re-checked out
JCR and voila - happy little unit tests run successfully.

I will save my first dumb obvious question for later.

/Chuck


On 2/6/05 3:05 PM, "Charles Severance" <cs...@umich.edu> wrote:

> <blah-blah>
> Hello to the JackRabbit community.  My name is Charles Severance, the Chief
> Architect of the Sakai project - www.sakaiproject.org -
> collab.sakaiproject.org has a sample system up and running.
> 
> I am kicking the tires on Jackrabbit for possible adoption as the core of our
> storage within Sakai (An open source collaboration and learning management
> system).  If we were to jump in here with Jackrabbit, we would probably
> write/improve/contribute an open source high performance implementation to
> scale to 100000 users at a single site (yikes).   We are already well underway
> to building a fast, scalable, hierarchical authorizaion system using
> Hibernate.
> 
> I have 2-3 developers to dedicate to this effort for the next six months if we
> adopt Jackrabbit because our alternative is to write this ourselves from
> scratch with our own API.
> </blah-blah>
> 
> I am trying to do a quick test integration over the weekend and almost have it
> running - but all my unit tests are failing.  I checked out the code, upgraded
> my maven plugin, and ran "maven jar" just like the README says.  Everything
> compiles fine except the Unit tests all fail with:
> 
> [junit] Running org.apache.jackrabbit.test.observation.TestAll
> [junit] Exception in thread "main" java.lang.NoClassDefFoundError:
> org/w3c/dom/ranges/DocumentRange
> [junit]     at java.lang.ClassLoader.defineClass0(Native Method)
> [junit]     at java.lang.ClassLoader.defineClass(ClassLoader.java:539)
> 
> [snip]
> 
> [junit]     at 
> org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRun
> ner.java:536)
> [junit] [ERROR] TEST org.apache.jackrabbit.test.search.TestAll FAILED
> 
> I am betting that this has a really simple solution :)
> 
> Thanks in advance.
> 
> Charles Severance
> www.dr-chuck.com
> csev@umich.edu