IBM Roller development update

[Elias Torres <el...@torrez.us>]

Hi Everyone,

I know I have been away for over a month now but I still wanted to let
you know what we have been up to at IBM in regards to the Roller
project. First, we have some great news around the use of Roller at
IBM. Lotus announced Connections, a social networking package for the
enterprise that includes Roller as its blogging component.

Now that the product code is stabilizing, we want to share with you
the things that we have done and added and plan how we can give these
back to the project.

- New backend implementation using iBatis
  - Reduction of # of sql queries to increase performance
- Better cluster support:
  - MBeans implementation to support better administration in a
clustered environment
  - Completely reworked search to work in clustered environment
    and avoid re-indexing on improper shutdown
  - Tweaked caching strategy to avoid blowin up the heap with output content
  - Refactored code to have proper HTTP support for Reverse Proxies
  - Cleaned up unnecessary creation of sessions
- Support for authenticated comments
- A lot of tweaks to support container managed security and SSO environments
  - For example, MetaWeblog API relied on passwords stored in the db
regardless of the setup

I'm sure other things escape me, but I'll send more details as I
remember them. Anyways, we hadn't shared all of this publicly because
we are waiting for legal approval to share these code features back
with Roller as long as you are interested in approving the changes.

I guess the main discussion point I would like to start from this
email is which data access strategy we should use in Roller.  We have
an iBatis (Apache License) implementation that is working well and
continually being optimized. So far the results are very promising and
would like to donate that back. I just want to know if using iBatis
can still be a choice in the decision and what would need to do to
make sure it happens.

Regards,

-Elias

Re: IBM Roller development update

[Matt Raible <mr...@gmail.com>]

On 1/30/07, Dave <sn...@gmail.com> wrote:
> On 1/29/07, Elias Torres <el...@torrez.us> wrote:
> > I know I have been away for over a month now but I still wanted to let
> > you know what we have been up to at IBM in regards to the Roller
> > project. First, we have some great news around the use of Roller at
> > IBM. Lotus announced Connections, a social networking package for the
> > enterprise that includes Roller as its blogging component.
> >
> > Now that the product code is stabilizing, we want to share with you
> > the things that we have done and added and plan how we can give these
> > back to the project.
> >
> > - New backend implementation using iBatis
> >   - Reduction of # of sql queries to increase performance
> > - Better cluster support:
> >   - MBeans implementation to support better administration in a
> > clustered environment
> >   - Completely reworked search to work in clustered environment
> >     and avoid re-indexing on improper shutdown
> >   - Tweaked caching strategy to avoid blowin up the heap with output content
> >   - Refactored code to have proper HTTP support for Reverse Proxies
> >   - Cleaned up unnecessary creation of sessions
> > - Support for authenticated comments
> > - A lot of tweaks to support container managed security and SSO environments
> >   - For example, MetaWeblog API relied on passwords stored in the db
> > regardless of the setup
>
> Wow. Lots of changes and lots of good things. Thank you and IBM very
> much for offering to contribute this to the Roller community. I hope
> we can figure out how to get some or all of this work into Roller.
>
> I don't want to come off as negative, but I do have a couple of concerns.
>
> Since this work was done in a separate repository by folks who are not
> all Roller committers (correct me if I'm wrong), I assume you'll need
> to do code grant paperwork to contribute it to Roller.
>
> And since work was done isolation without the benefit of open
> discussion on the Roller mailing lists, we probably need to work up
> proposals on each of those features/changes so we can have those
> discussions now.
>
> And I don't want to get all preachy, but I would greatly prefer to see
> all Roller discussions happen on the mailing list and development
> happen in SVN. If developers are toiling away in isolation, we'll
> never get to know them, they won't get nominated as committers and the
> community won't grow as much as it could. IBM will have fewer votes
> too, hmm... maybe I should not be telling you this ;-)
>
>
> > I guess the main discussion point I would like to start from this
> > email is which data access strategy we should use in Roller.  We have
> > an iBatis (Apache License) implementation that is working well and
> > continually being optimized. So far the results are very promising and
> > would like to donate that back. I just want to know if using iBatis
> > can still be a choice in the decision and what would need to do to
> > make sure it happens.
>
> Oh boy.
>
> I don't think it is too late to discuss iBatis and it's fair to ask
> that iBatis be considered as our new persistence framework. We could
> get the new iBatis implementation into the Roller 4.0 branch and do a
> bake-off of JPA vs. iBatis to see how folks like the programming model
> and how performance compares. We could then vote to decide what to do.
>
> JPA does have several advantages over iBatis from my point of view.
> First, it's a Java standard API with multiple implementations
> supported Oracle, BEA, Sun, etc. Second, there is a Hibernate
> implementation of JPA, so folks can conceivably continue to use
> Hibernate if they want to. And third, for my work I have to support a
> JPA back-end. So if the Roller team picks iBatis, I'll be forced to
> maintain a second back-end -- not something I want to do.
>
> As much as I hate the idea, if different members of the community
> really want/need different back-end implementations, then I think we
> have figure out how to enable that or not. We come back to Craig's
> question, how to we let them co-exist. I think there are at least
> these issues:
>
> 1) Multiple back-end with equal standing. Developers must commit
> changes and get tests passing in both back-ends before committing.
>
> 2) Multiple back-ends but only one is primary. One back-end is
> designated the primary back-end and developers must ensure only that
> test pass in that primary back-end before committing.
>
> 3) One back-end allowed in SVN. Only one back-end is allowed in trunk
> and anybody who
>  wants some other backend will have to maintain it separately.
>
> Other points of view?

I don't see the problem with multiple backends if there's developers
on the team that are willing to support them.  We have 3 backends in
AppFuse (hibernate, jpa and ibatis) and we have developers to support
them.  It seems to work pretty well and we even have a way to easily
allow the end user to switch to the one they want to use.

Is now a good time to bring up how Maven 2 solves many of these module
problems? ;-)

I've learned a *ton* about Maven 2 working with the Ant -> Maven 2
conversion on AppFuse. I'd be happy to do the conversion for Roller if
there's interest.  The biggest reason I can think of for changing is
to make web development easier.

Matt

>
> - Dave
>


-- 
http://raibledesigns.com

Re: IBM Roller development update

[Allen Gilliland <al...@sun.com>]

Elias Torres wrote:
> On 1/30/07, Dave <sn...@gmail.com> wrote:
>> On 1/29/07, Elias Torres <el...@torrez.us> wrote:
>> > I know I have been away for over a month now but I still wanted to let
>> > you know what we have been up to at IBM in regards to the Roller
>> > project. First, we have some great news around the use of Roller at
>> > IBM. Lotus announced Connections, a social networking package for the
>> > enterprise that includes Roller as its blogging component.
>> >
>> > Now that the product code is stabilizing, we want to share with you
>> > the things that we have done and added and plan how we can give these
>> > back to the project.
>> >
>> > - New backend implementation using iBatis
>> >   - Reduction of # of sql queries to increase performance
>> > - Better cluster support:
>> >   - MBeans implementation to support better administration in a
>> > clustered environment
>> >   - Completely reworked search to work in clustered environment
>> >     and avoid re-indexing on improper shutdown
>> >   - Tweaked caching strategy to avoid blowin up the heap with output 
>> content
>> >   - Refactored code to have proper HTTP support for Reverse Proxies
>> >   - Cleaned up unnecessary creation of sessions
>> > - Support for authenticated comments
>> > - A lot of tweaks to support container managed security and SSO 
>> environments
>> >   - For example, MetaWeblog API relied on passwords stored in the db
>> > regardless of the setup
>>
>> Wow. Lots of changes and lots of good things. Thank you and IBM very
>> much for offering to contribute this to the Roller community. I hope
>> we can figure out how to get some or all of this work into Roller.
> 
> Cool.
> 
>>
>> I don't want to come off as negative, but I do have a couple of concerns.
>>
>> Since this work was done in a separate repository by folks who are not
>> all Roller committers (correct me if I'm wrong), I assume you'll need
>> to do code grant paperwork to contribute it to Roller.
> 
> That's correct.
> 
>>
>> And since work was done isolation without the benefit of open
>> discussion on the Roller mailing lists, we probably need to work up
>> proposals on each of those features/changes so we can have those
>> discussions now.
> 
> Fair enough.
> 
>>
>> And I don't want to get all preachy, but I would greatly prefer to see
>> all Roller discussions happen on the mailing list and development
>> happen in SVN. If developers are toiling away in isolation, we'll
>> never get to know them, they won't get nominated as committers and the
>> community won't grow as much as it could. IBM will have fewer votes
>> too, hmm... maybe I should not be telling you this ;-)
> 
> Too late, now we know. :-) I'll take this discussion back to IBM and
> ask the developers to start having those discussions in the mailing
> list.

I also agree with Dave that having discussions on the list is the best 
place, not only so that the developers can be properly recognized but 
also so that we make sure that work isn't being done that will 
ultimately not be able to go back into the project.

I understand why you had discussions off list and in truth I don't think 
that all discussions need to be on the list, but after hearing the big 
list of work that you guys did my biggest fear would be that some of 
that work can't be contributed because it was implemented in a different 
way than the community wants.  Hopefully that is not the case here, but 
having those discussions on the list helps prevent situations like that.


> 
>>
>>
>> > I guess the main discussion point I would like to start from this
>> > email is which data access strategy we should use in Roller.  We have
>> > an iBatis (Apache License) implementation that is working well and
>> > continually being optimized. So far the results are very promising and
>> > would like to donate that back. I just want to know if using iBatis
>> > can still be a choice in the decision and what would need to do to
>> > make sure it happens.
>>
>> Oh boy.
>>
>> I don't think it is too late to discuss iBatis and it's fair to ask
>> that iBatis be considered as our new persistence framework. We could
>> get the new iBatis implementation into the Roller 4.0 branch and do a
>> bake-off of JPA vs. iBatis to see how folks like the programming model
>> and how performance compares. We could then vote to decide what to do.
> 
> Let's do it. :-)

I'll let you guys organize that for the most part, but what this does 
bring up is something that I am interested in having for Roller, which 
is some automated performance tests.

We do performance testing against the application as a whole, but I 
would really like to get a standardized tool for running some automated 
performance tests put into the repository so that from release to 
release there is a solid way of measuring the performance of any part of 
the system.  I think the best place to start with this would be to setup 
tests against the backend for checking query performance which could be 
used in this bake off.

Anyone have any tools that they have used and like for doing automated 
performance testing of code?


> 
>>
>> JPA does have several advantages over iBatis from my point of view.
>> First, it's a Java standard API with multiple implementations
>> supported Oracle, BEA, Sun, etc. Second, there is a Hibernate
>> implementation of JPA, so folks can conceivably continue to use
>> Hibernate if they want to. And third, for my work I have to support a
>> JPA back-end. So if the Roller team picks iBatis, I'll be forced to
>> maintain a second back-end -- not something I want to do.
> 
> Oh boy. So on your end, it must be JPA?
> 
>>
>> As much as I hate the idea, if different members of the community
>> really want/need different back-end implementations, then I think we
>> have figure out how to enable that or not. We come back to Craig's
>> question, how to we let them co-exist. I think there are at least
>> these issues:
>>
>> 1) Multiple back-end with equal standing. Developers must commit
>> changes and get tests passing in both back-ends before committing.
>>
>> 2) Multiple back-ends but only one is primary. One back-end is
>> designated the primary back-end and developers must ensure only that
>> test pass in that primary back-end before committing.
>>
>> 3) One back-end allowed in SVN. Only one back-end is allowed in trunk
>> and anybody who
>>  wants some other backend will have to maintain it separately.
> 
> I'm certain that our intentions are not to make the Roller development
> by maintaining multiple back-end implementations, but we want to know
> what really makes JPA the best choice. So far the
> multiple-implementations story doesn't convince me. All we need is one
> that works with the different dbs and it's easy to use. The simpler
> the better.

I won't say that I have any strong opinions about either JPA or iBatis 
having not worked with either of them, but I do believe that Dave's 
point about JPA being an actual approved Java api is a noticeable 
benefit in my mind.

However, regardless of what backend is preferred, the one thing that I 
am still very much against is having multiple backends.  There is no way 
that I want to have to do double work and against 2 different apis in 
order to change anything to the Roller backend.  And code maintenance 
aside, the additional complexity that would present to users is just 
silly, there is no need for it and personally I am not even willing to 
consider it.  So, no matter what apis people prefer for the backend, at 
the end of the day I would only be willing to officially support one.

So I am supporting option #3 above.

-- Allen


> 
> -Elias

Re: IBM Roller development update

[Dave <sn...@gmail.com>]

On 1/30/07, Elias Torres <el...@torrez.us> wrote:
> On 1/30/07, Dave <sn...@gmail.com> wrote:
> > On 1/29/07, Elias Torres <el...@torrez.us> wrote:
> > JPA does have several advantages over iBatis from my point of view.
> > First, it's a Java standard API with multiple implementations
> > supported Oracle, BEA, Sun, etc. Second, there is a Hibernate
> > implementation of JPA, so folks can conceivably continue to use
> > Hibernate if they want to. And third, for my work I have to support a
> > JPA back-end. So if the Roller team picks iBatis, I'll be forced to
> > maintain a second back-end -- not something I want to do.
>
> Oh boy. So on your end, it must be JPA?

Currently that is the case. Of course, if the community decides on
Batis, that may become an untenable position.


> > As much as I hate the idea, if different members of the community
> > really want/need different back-end implementations, then I think we
> > have figure out how to enable that or not. We come back to Craig's
> > question, how to we let them co-exist. I think there are at least
> > these issues:
> >
> > 1) Multiple back-end with equal standing. Developers must commit
> > changes and get tests passing in both back-ends before committing.
> >
> > 2) Multiple back-ends but only one is primary. One back-end is
> > designated the primary back-end and developers must ensure only that
> > test pass in that primary back-end before committing.
> >
> > 3) One back-end allowed in SVN. Only one back-end is allowed in trunk
> > and anybody who
> >  wants some other backend will have to maintain it separately.
>
> I'm certain that our intentions are not to make the Roller development
> by maintaining multiple back-end implementations, but we want to know
> what really makes JPA the best choice. So far the
> multiple-implementations story doesn't convince me. All we need is one
> that works with the different dbs and it's easy to use. The simpler
> the better.

I agree. Let's let others respond, but I think the best course of
action is to start the roller_4.0 branch (I plan to do that tomorrow)
and get JPA and iBatis working there as quickly as possible so we can
compare.

- Dave

Re: IBM Roller development update

[Richard Jones <ri...@pixyblog.com>]

> Maybe you could just start by describing some of the things you have had
> to tweak to get Roller to be better tailored as a photoblogging app?

Sure - I will write-up a description next week and put it in a new post.

Thanks for your comments... plus Matt's post on Raible Designs - it
accounted for 50% of our traffic today :-)

- Richard

Re: IBM Roller development update

[Allen Gilliland <al...@sun.com>]

Richard Jones wrote:
> All the IBM development sounds sweet to me... I would love to get as
> much of it integrated into Roller as possible.
> 
> I'm afraid I'm one of those developers 'toiling' away at Roller and
> haven't really introduced myself to the Roller community and thought
> that this was a good opportunity.  For the last 6 months I've been
> turning Roller into photoblogging platform - you can see the most
> up-to-date version @ www.pixyblog.com.  Unlike the IBM developments,
> the majority of the current changes I've made to Roller would not be
> beneficial for inclusion into the Roller webapp, but I expect this to
> change as my project develops.

very cool.  I can see that what you are working on is definitely a 
specialized use of Roller, but I wouldn't write off the work you are 
doing as not beneficial for inclusion in the project so quickly.  I 
think one of the things that we want for Roller long term is to think of 
it more as a platform than as just a tool, so having extendable 
functionality like you have done here would be great.

Of course we would want to develop it as a plugin/addon type of 
situation, but I don't see any reason why the work you are doing 
wouldn't benefit other folks and help drive that effort.  Photoblogging 
is pretty popular so I could see a lot of folks having use for what you 
have developed.

Maybe you could just start by describing some of the things you have had 
to tweak to get Roller to be better tailored as a photoblogging app?


> 
> As for my 2 cents on the JPA vs iBate discussion - as long as it works
> and is easy to work with I have no preference, but I agree with Elias
> that I don't see the benefits for Roller to support multiple backends.
> 
> Great project guys!  I love working with the Roller source ;-)

glad to hear it :)

-- Allen


> 
> Cheers,
> 
> Richard Jones
> 
> 
> 
> On 30/01/07, James M Snell <ja...@gmail.com> wrote:
>> simplest + most efficient + easiest to optimize.
>>
>> - James
>>
>> Elias Torres wrote:
>> >[snip]
>> > multiple-implementations story doesn't convince me. All we need is one
>> > that works with the different dbs and it's easy to use. The simpler
>> > the better.
>> >
>> > -Elias
>> >
>>
>>

Re: IBM Roller development update

[Richard Jones <ri...@pixyblog.com>]

All the IBM development sounds sweet to me... I would love to get as
much of it integrated into Roller as possible.

I'm afraid I'm one of those developers 'toiling' away at Roller and
haven't really introduced myself to the Roller community and thought
that this was a good opportunity.  For the last 6 months I've been
turning Roller into photoblogging platform - you can see the most
up-to-date version @ www.pixyblog.com.  Unlike the IBM developments,
the majority of the current changes I've made to Roller would not be
beneficial for inclusion into the Roller webapp, but I expect this to
change as my project develops.

As for my 2 cents on the JPA vs iBate discussion - as long as it works
and is easy to work with I have no preference, but I agree with Elias
that I don't see the benefits for Roller to support multiple backends.

Great project guys!  I love working with the Roller source ;-)

Cheers,

Richard Jones



On 30/01/07, James M Snell <ja...@gmail.com> wrote:
> simplest + most efficient + easiest to optimize.
>
> - James
>
> Elias Torres wrote:
> >[snip]
> > multiple-implementations story doesn't convince me. All we need is one
> > that works with the different dbs and it's easy to use. The simpler
> > the better.
> >
> > -Elias
> >
>
>

Re: IBM Roller development update

[Matt Raible <mr...@gmail.com>]

I've worked with Hibernate, JPA and iBATIS.  IMO, iBATIS is the
easiest to learn and understand. ;-)

Matt

On 1/30/07, Allen Gilliland <al...@sun.com> wrote:
> I agree with all of those, but that also seems like a fairly developer
> centeric view.  Optimization is important to me as well, but I think we
> have to remember that a great deal of users who will deploy Roller will
> never see enough traffic for optimization to be a real concern, so we
> should make sure that users are considered in this decision ...
>
> most portable + best support community + best licensing options +
> easiest to develop against + easiest configuration
>
> -- Allen
>
>
> James M Snell wrote:
> > simplest + most efficient + easiest to optimize.
> >
> > - James
> >
> > Elias Torres wrote:
> >> [snip]
> >> multiple-implementations story doesn't convince me. All we need is one
> >> that works with the different dbs and it's easy to use. The simpler
> >> the better.
> >>
> >> -Elias
> >>
>


-- 
http://raibledesigns.com

Re: IBM Roller development update

[Allen Gilliland <al...@sun.com>]

I agree with all of those, but that also seems like a fairly developer 
centeric view.  Optimization is important to me as well, but I think we 
have to remember that a great deal of users who will deploy Roller will 
never see enough traffic for optimization to be a real concern, so we 
should make sure that users are considered in this decision ...

most portable + best support community + best licensing options + 
easiest to develop against + easiest configuration

-- Allen


James M Snell wrote:
> simplest + most efficient + easiest to optimize.
> 
> - James
> 
> Elias Torres wrote:
>> [snip]
>> multiple-implementations story doesn't convince me. All we need is one
>> that works with the different dbs and it's easy to use. The simpler
>> the better.
>>
>> -Elias
>>

Re: IBM Roller development update

[James M Snell <ja...@gmail.com>]

simplest + most efficient + easiest to optimize.

- James

Elias Torres wrote:
>[snip]
> multiple-implementations story doesn't convince me. All we need is one
> that works with the different dbs and it's easy to use. The simpler
> the better.
> 
> -Elias
> 

Re: IBM Roller development update

[Elias Torres <el...@torrez.us>]

On 1/30/07, Dave <sn...@gmail.com> wrote:
> On 1/29/07, Elias Torres <el...@torrez.us> wrote:
> > I know I have been away for over a month now but I still wanted to let
> > you know what we have been up to at IBM in regards to the Roller
> > project. First, we have some great news around the use of Roller at
> > IBM. Lotus announced Connections, a social networking package for the
> > enterprise that includes Roller as its blogging component.
> >
> > Now that the product code is stabilizing, we want to share with you
> > the things that we have done and added and plan how we can give these
> > back to the project.
> >
> > - New backend implementation using iBatis
> >   - Reduction of # of sql queries to increase performance
> > - Better cluster support:
> >   - MBeans implementation to support better administration in a
> > clustered environment
> >   - Completely reworked search to work in clustered environment
> >     and avoid re-indexing on improper shutdown
> >   - Tweaked caching strategy to avoid blowin up the heap with output content
> >   - Refactored code to have proper HTTP support for Reverse Proxies
> >   - Cleaned up unnecessary creation of sessions
> > - Support for authenticated comments
> > - A lot of tweaks to support container managed security and SSO environments
> >   - For example, MetaWeblog API relied on passwords stored in the db
> > regardless of the setup
>
> Wow. Lots of changes and lots of good things. Thank you and IBM very
> much for offering to contribute this to the Roller community. I hope
> we can figure out how to get some or all of this work into Roller.

Cool.

>
> I don't want to come off as negative, but I do have a couple of concerns.
>
> Since this work was done in a separate repository by folks who are not
> all Roller committers (correct me if I'm wrong), I assume you'll need
> to do code grant paperwork to contribute it to Roller.

That's correct.

>
> And since work was done isolation without the benefit of open
> discussion on the Roller mailing lists, we probably need to work up
> proposals on each of those features/changes so we can have those
> discussions now.

Fair enough.

>
> And I don't want to get all preachy, but I would greatly prefer to see
> all Roller discussions happen on the mailing list and development
> happen in SVN. If developers are toiling away in isolation, we'll
> never get to know them, they won't get nominated as committers and the
> community won't grow as much as it could. IBM will have fewer votes
> too, hmm... maybe I should not be telling you this ;-)

Too late, now we know. :-) I'll take this discussion back to IBM and
ask the developers to start having those discussions in the mailing
list.

>
>
> > I guess the main discussion point I would like to start from this
> > email is which data access strategy we should use in Roller.  We have
> > an iBatis (Apache License) implementation that is working well and
> > continually being optimized. So far the results are very promising and
> > would like to donate that back. I just want to know if using iBatis
> > can still be a choice in the decision and what would need to do to
> > make sure it happens.
>
> Oh boy.
>
> I don't think it is too late to discuss iBatis and it's fair to ask
> that iBatis be considered as our new persistence framework. We could
> get the new iBatis implementation into the Roller 4.0 branch and do a
> bake-off of JPA vs. iBatis to see how folks like the programming model
> and how performance compares. We could then vote to decide what to do.

Let's do it. :-)

>
> JPA does have several advantages over iBatis from my point of view.
> First, it's a Java standard API with multiple implementations
> supported Oracle, BEA, Sun, etc. Second, there is a Hibernate
> implementation of JPA, so folks can conceivably continue to use
> Hibernate if they want to. And third, for my work I have to support a
> JPA back-end. So if the Roller team picks iBatis, I'll be forced to
> maintain a second back-end -- not something I want to do.

Oh boy. So on your end, it must be JPA?

>
> As much as I hate the idea, if different members of the community
> really want/need different back-end implementations, then I think we
> have figure out how to enable that or not. We come back to Craig's
> question, how to we let them co-exist. I think there are at least
> these issues:
>
> 1) Multiple back-end with equal standing. Developers must commit
> changes and get tests passing in both back-ends before committing.
>
> 2) Multiple back-ends but only one is primary. One back-end is
> designated the primary back-end and developers must ensure only that
> test pass in that primary back-end before committing.
>
> 3) One back-end allowed in SVN. Only one back-end is allowed in trunk
> and anybody who
>  wants some other backend will have to maintain it separately.

I'm certain that our intentions are not to make the Roller development
by maintaining multiple back-end implementations, but we want to know
what really makes JPA the best choice. So far the
multiple-implementations story doesn't convince me. All we need is one
that works with the different dbs and it's easy to use. The simpler
the better.

-Elias

Re: IBM Roller development update

[Dave <sn...@gmail.com>]

On 1/29/07, Elias Torres <el...@torrez.us> wrote:
> I know I have been away for over a month now but I still wanted to let
> you know what we have been up to at IBM in regards to the Roller
> project. First, we have some great news around the use of Roller at
> IBM. Lotus announced Connections, a social networking package for the
> enterprise that includes Roller as its blogging component.
>
> Now that the product code is stabilizing, we want to share with you
> the things that we have done and added and plan how we can give these
> back to the project.
>
> - New backend implementation using iBatis
>   - Reduction of # of sql queries to increase performance
> - Better cluster support:
>   - MBeans implementation to support better administration in a
> clustered environment
>   - Completely reworked search to work in clustered environment
>     and avoid re-indexing on improper shutdown
>   - Tweaked caching strategy to avoid blowin up the heap with output content
>   - Refactored code to have proper HTTP support for Reverse Proxies
>   - Cleaned up unnecessary creation of sessions
> - Support for authenticated comments
> - A lot of tweaks to support container managed security and SSO environments
>   - For example, MetaWeblog API relied on passwords stored in the db
> regardless of the setup

Wow. Lots of changes and lots of good things. Thank you and IBM very
much for offering to contribute this to the Roller community. I hope
we can figure out how to get some or all of this work into Roller.

I don't want to come off as negative, but I do have a couple of concerns.

Since this work was done in a separate repository by folks who are not
all Roller committers (correct me if I'm wrong), I assume you'll need
to do code grant paperwork to contribute it to Roller.

And since work was done isolation without the benefit of open
discussion on the Roller mailing lists, we probably need to work up
proposals on each of those features/changes so we can have those
discussions now.

And I don't want to get all preachy, but I would greatly prefer to see
all Roller discussions happen on the mailing list and development
happen in SVN. If developers are toiling away in isolation, we'll
never get to know them, they won't get nominated as committers and the
community won't grow as much as it could. IBM will have fewer votes
too, hmm... maybe I should not be telling you this ;-)


> I guess the main discussion point I would like to start from this
> email is which data access strategy we should use in Roller.  We have
> an iBatis (Apache License) implementation that is working well and
> continually being optimized. So far the results are very promising and
> would like to donate that back. I just want to know if using iBatis
> can still be a choice in the decision and what would need to do to
> make sure it happens.

Oh boy.

I don't think it is too late to discuss iBatis and it's fair to ask
that iBatis be considered as our new persistence framework. We could
get the new iBatis implementation into the Roller 4.0 branch and do a
bake-off of JPA vs. iBatis to see how folks like the programming model
and how performance compares. We could then vote to decide what to do.

JPA does have several advantages over iBatis from my point of view.
First, it's a Java standard API with multiple implementations
supported Oracle, BEA, Sun, etc. Second, there is a Hibernate
implementation of JPA, so folks can conceivably continue to use
Hibernate if they want to. And third, for my work I have to support a
JPA back-end. So if the Roller team picks iBatis, I'll be forced to
maintain a second back-end -- not something I want to do.

As much as I hate the idea, if different members of the community
really want/need different back-end implementations, then I think we
have figure out how to enable that or not. We come back to Craig's
question, how to we let them co-exist. I think there are at least
these issues:

1) Multiple back-end with equal standing. Developers must commit
changes and get tests passing in both back-ends before committing.

2) Multiple back-ends but only one is primary. One back-end is
designated the primary back-end and developers must ensure only that
test pass in that primary back-end before committing.

3) One back-end allowed in SVN. Only one back-end is allowed in trunk
and anybody who
 wants some other backend will have to maintain it separately.

Other points of view?

- Dave

Re: IBM Roller development update

[James M Snell <ja...@gmail.com>]

Just another quick point: in addition to the features that have been
implemented to support the Lotus Connections product, the IBM internal
blogging environment team is developing a number of new features and
general improvements that we will also be looking to add back to the
roller base project.  One of these is a new entry rating system and a
"featured posts" query that will select featured posts based on
cumulative rating, comment count, view count, etc.  We're also looking
to revamp the hit count tracking and add the ability to filter the
aggregated list of entries by language/locale.

As Elias mentioned, we're in the process of securing all the legal
approvals we need to donate this stuff back to the Roller project.

- James

Elias Torres wrote:
> Hi Everyone,
> 
> I know I have been away for over a month now but I still wanted to let
> you know what we have been up to at IBM in regards to the Roller
> project. First, we have some great news around the use of Roller at
> IBM. Lotus announced Connections, a social networking package for the
> enterprise that includes Roller as its blogging component.
> 
> Now that the product code is stabilizing, we want to share with you
> the things that we have done and added and plan how we can give these
> back to the project.
> 
> - New backend implementation using iBatis
>  - Reduction of # of sql queries to increase performance
> - Better cluster support:
>  - MBeans implementation to support better administration in a
> clustered environment
>  - Completely reworked search to work in clustered environment
>    and avoid re-indexing on improper shutdown
>  - Tweaked caching strategy to avoid blowin up the heap with output content
>  - Refactored code to have proper HTTP support for Reverse Proxies
>  - Cleaned up unnecessary creation of sessions
> - Support for authenticated comments
> - A lot of tweaks to support container managed security and SSO
> environments
>  - For example, MetaWeblog API relied on passwords stored in the db
> regardless of the setup
> 
> I'm sure other things escape me, but I'll send more details as I
> remember them. Anyways, we hadn't shared all of this publicly because
> we are waiting for legal approval to share these code features back
> with Roller as long as you are interested in approving the changes.
> 
> I guess the main discussion point I would like to start from this
> email is which data access strategy we should use in Roller.  We have
> an iBatis (Apache License) implementation that is working well and
> continually being optimized. So far the results are very promising and
> would like to donate that back. I just want to know if using iBatis
> can still be a choice in the decision and what would need to do to
> make sure it happens.
> 
> Regards,
> 
> -Elias
> 

Re: IBM Roller development update

[Allen Gilliland <al...@sun.com>]

rob wrote:
> I work alongside Elias here at IBM and did a fair amount of the IBatis 
> work on Roller
> 
> Craig L Russell wrote:
>> Hi Elias,
>>
>> Showing my ignorance of iBatis here, but I thought that iBatis was a 
>> very thin wrapper around SQL.
> It is
>>
>> So if you support multiple databases, don't you need different SQL for 
>> each one?
> Not really, we now have support for DB2, Oracle and MySql and while we 
> did have to make some IBatis tweaks to get things working on Oracle we 
> have yet to find a case where we need to fire different SQL at different 
> back ends.

Hmmm, this part sounds questionable to me.  As I said before, I don't 
have any experience with iBatis so I don't know the full story, but if 
iBatis can't guarantee database independence like many of the other ORM 
solutions then that seems like a mess we wouldn't want to get into.

You say you have tested against DB2, Oracle, and MySql, but how many 
versions?  What about all the other dbs out there that have been used by 
Roller thus far ... Derby, Hsql, Postgres, MSSql, others?

I think we *really*, *really* don't want to get into a situation where 
users may have problems running Roller against the db of their choice 
due to disparities in sql syntax.  So that should be considered an 
important criteria when choosing which solution to choose for the backend.

-- Allen


>>
>> Is this an extended iBatis you are using or has iBatis changed in ways 
>> that I didn't know about?
> Regular Ibatis, no changes.
> 
> Rob
> 
> p.s. we will make a point of discussing features on this list going 
> forward, but it wasn't clear early in the project's life whether IBM 
> would permit us to submit code back to the community.  There's still a 
> few final legal hurdles to clear internally, but it looks like we are 
> now in the final stages.  Once complete there are probably 3 of us that 
> will need to do the code grant paperwork.

Re: IBM Roller development update

[rob <ro...@gmail.com>]

I work alongside Elias here at IBM and did a fair amount of the IBatis 
work on Roller

Craig L Russell wrote:
> Hi Elias,
>
> Showing my ignorance of iBatis here, but I thought that iBatis was a 
> very thin wrapper around SQL.
It is
>
> So if you support multiple databases, don't you need different SQL for 
> each one?
Not really, we now have support for DB2, Oracle and MySql and while we 
did have to make some IBatis tweaks to get things working on Oracle we 
have yet to find a case where we need to fire different SQL at different 
back ends.
>
> Is this an extended iBatis you are using or has iBatis changed in ways 
> that I didn't know about?
Regular Ibatis, no changes.

Rob

p.s. we will make a point of discussing features on this list going 
forward, but it wasn't clear early in the project's life whether IBM 
would permit us to submit code back to the community.  There's still a 
few final legal hurdles to clear internally, but it looks like we are 
now in the final stages.  Once complete there are probably 3 of us that 
will need to do the code grant paperwork.

Re: IBM Roller development update

[Craig L Russell <Cr...@Sun.COM>]

Hi Elias,

Showing my ignorance of iBatis here, but I thought that iBatis was a  
very thin wrapper around SQL.

So if you support multiple databases, don't you need different SQL  
for each one?

Is this an extended iBatis you are using or has iBatis changed in  
ways that I didn't know about?

Thanks,

Craig

On Jan 29, 2007, at 3:43 PM, Elias Torres wrote:

> Hi Everyone,
>
> I know I have been away for over a month now but I still wanted to let
> you know what we have been up to at IBM in regards to the Roller
> project. First, we have some great news around the use of Roller at
> IBM. Lotus announced Connections, a social networking package for the
> enterprise that includes Roller as its blogging component.
>
...
> I guess the main discussion point I would like to start from this
> email is which data access strategy we should use in Roller.  We have
> an iBatis (Apache License) implementation that is working well and
> continually being optimized. So far the results are very promising and
> would like to donate that back. I just want to know if using iBatis
> can still be a choice in the decision and what would need to do to
> make sure it happens.
>
> Regards,
>
> -Elias

Craig Russell
Architect, Sun Java Enterprise System http://java.sun.com/products/jdo
408 276-5638 mailto:Craig.Russell@sun.com
P.S. A good JDO? O, Gasp!