DBCP2 in Tomcat

[Rémy Maucherat <re...@apache.org>]

Hi,

In Tomcat, DBCP2 is missing the XA portion (all contained in a single
"managed" package). So at work I now got some people asking about that
removal, and that's always a bit annoying as they have to go to a separate
vanilla DBCP2 to get the functionality. Annoying sometimes.

So it would be possible to add the classes in Tomcat (including the
javax.transaction to build, as that's the Tomcat way to deal with that),
even though the user would need to add its own transaction manager to do
anything with it.

Should I now add it (only in 9/trunk) or instead leave things the way they
are ? Both work to be honest, it's just that I've been bitten by the "we
only ship 3/4 of DBCP and I didn't know it" bug.

Rémy

Re: DBCP2 in Tomcat

[Rémy Maucherat <re...@apache.org>]

On Thu, Mar 29, 2018 at 10:31 PM, Mark Thomas <ma...@apache.org> wrote:

> On 29/03/18 20:58, Mark Thomas wrote:
> > On 29/03/18 14:39, Christopher Schultz wrote:
> >> On 3/29/18 9:30 AM, Rémy Maucherat wrote:
> >>> On Thu, Mar 29, 2018 at 3:15 PM, Christopher Schultz <
> >>> chris@christopherschultz.net> wrote:
> >>>> On 3/29/18 8:42 AM, Rémy Maucherat wrote:
> >>>>> On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
> >>>>> chris@christopherschultz.net> wrote:
> >>>>>> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> In Tomcat, DBCP2 is missing the XA portion (all contained in a
> single
> >>>>>>> "managed" package). So at work I now got some people asking about
> that
> >>>>>>> removal, and that's always a bit annoying as they have to go to a
> >>>>>> separate
> >>>>>>> vanilla DBCP2 to get the functionality. Annoying sometimes.
> >>>>>>>
> >>>>>>> So it would be possible to add the classes in Tomcat (including the
> >>>>>>> javax.transaction to build, as that's the Tomcat way to deal with
> >>>> that),
> >>>>>>> even though the user would need to add its own transaction manager
> to
> >>>> do
> >>>>>>> anything with it.
> >>>>>>>
> >>>>>>> Should I now add it (only in 9/trunk) or instead leave things the
> way
> >>>>>> they
> >>>>>>> are ? Both work to be honest, it's just that I've been bitten by
> the
> >>>> "we
> >>>>>>> only ship 3/4 of DBCP and I didn't know it" bug.
>
> Sorry. I meant to reply to this bit as well but forgot.
>
> From memory I left it out on the basis few people needed it and that we
> could always add it in later if there was demand.
>
> There appears to be demand so no objections here to adding that code.
>
> Ok, so if nobody complains, I'll add the classes today [based on the
8a71764 revision that was integrated in 9.0.3], this will make my usecase
better. I had to make the casual cleanups to pass Tomcat's checkstyle,
should that be contributed to DBCP ?

Rémy

Re: DBCP2 in Tomcat

[Mark Thomas <ma...@apache.org>]

On 29/03/18 20:58, Mark Thomas wrote:
> On 29/03/18 14:39, Christopher Schultz wrote:
>> On 3/29/18 9:30 AM, Rémy Maucherat wrote:
>>> On Thu, Mar 29, 2018 at 3:15 PM, Christopher Schultz <
>>> chris@christopherschultz.net> wrote:
>>>> On 3/29/18 8:42 AM, Rémy Maucherat wrote:
>>>>> On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
>>>>> chris@christopherschultz.net> wrote:
>>>>>> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
>>>>>>> "managed" package). So at work I now got some people asking about that
>>>>>>> removal, and that's always a bit annoying as they have to go to a
>>>>>> separate
>>>>>>> vanilla DBCP2 to get the functionality. Annoying sometimes.
>>>>>>>
>>>>>>> So it would be possible to add the classes in Tomcat (including the
>>>>>>> javax.transaction to build, as that's the Tomcat way to deal with
>>>> that),
>>>>>>> even though the user would need to add its own transaction manager to
>>>> do
>>>>>>> anything with it.
>>>>>>>
>>>>>>> Should I now add it (only in 9/trunk) or instead leave things the way
>>>>>> they
>>>>>>> are ? Both work to be honest, it's just that I've been bitten by the
>>>> "we
>>>>>>> only ship 3/4 of DBCP and I didn't know it" bug.

Sorry. I meant to reply to this bit as well but forgot.

From memory I left it out on the basis few people needed it and that we
could always add it in later if there was demand.

There appears to be demand so no objections here to adding that code.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: DBCP2 in Tomcat

[Mark Thomas <ma...@apache.org>]

On 29/03/18 14:39, Christopher Schultz wrote:
> On 3/29/18 9:30 AM, Rémy Maucherat wrote:
>> On Thu, Mar 29, 2018 at 3:15 PM, Christopher Schultz <
>> chris@christopherschultz.net> wrote:
>>> On 3/29/18 8:42 AM, Rémy Maucherat wrote:
>>>> On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
>>>> chris@christopherschultz.net> wrote:
>>>>> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
>>>>>> Hi,
>>>>>>
>>>>>> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
>>>>>> "managed" package). So at work I now got some people asking about that
>>>>>> removal, and that's always a bit annoying as they have to go to a
>>>>> separate
>>>>>> vanilla DBCP2 to get the functionality. Annoying sometimes.
>>>>>>
>>>>>> So it would be possible to add the classes in Tomcat (including the
>>>>>> javax.transaction to build, as that's the Tomcat way to deal with
>>> that),
>>>>>> even though the user would need to add its own transaction manager to
>>> do
>>>>>> anything with it.
>>>>>>
>>>>>> Should I now add it (only in 9/trunk) or instead leave things the way
>>>>> they
>>>>>> are ? Both work to be honest, it's just that I've been bitten by the
>>> "we
>>>>>> only ship 3/4 of DBCP and I didn't know it" bug.


>>>>> I've always wondered why we bother to package-rename DBCP2 and check-in
>>>>> the source into our svn repo (soon to be Git). Why not pull DBCP2 from
>>>>> source during the build and re-name it on the fly?

So we can release Tomcat with the latest DBCP2 and POOL2 sources without
having to wait on Commons to produce a release.

I find the Commons release process a pain to navigate. The svn copy (now
a merge from git) is less hassle.

Commons is open to all ASF committers so if anyone here wants more
frequent DBCP2 and POOL2 releases I am sure the Commons community would
welcome them with open arms.

>> The rationale was probably: you can fix bugs without having to wait for a
>> DBCP release. I'm not certain this actually happened though.

It has.

>> I actually have a question. There's also that *second* connection pool,
>> Tomcat JDBC, although it is more "external" as it is located inside the
>> modules. So reading the list of "pros" from its docs, it seems DBCP
>> addressed most (all ?) of them, except that it's still significantly
>> bigger. So what's the current benefits of Tomcat JDBC over Tomcat DBCP2 ?
>> Did anyone do benchmarks or anything ?

Performance wise DBCP1, DBCP2 and Tomcat-JDBC are all pretty much equal
until you start to hammer the connection pool with concurrent borrows
and returns. At that point DBCP1 performance falls off a cliff. DBCP2
and Tomcat-JDBC are so close you probably won't notice but Tomcat-JDBC
does have a slight edge.

Functionality wise, DBCP2 handles more edge cases by default than
Tomcat-JDBC. Most (I'm not sure if all) can be handled by Tomcat-JDBC
with additional configuration (usually extra interceptors).

Tomcat-JDBC has better JMX support, particularly performance related
metrics.

> I believe Filip built that in response to the issues with DBCP at the
> time. I assume that benchmarks were done at the time proving the
> usefulness of the tomcat-pool over DBCP 1.x. I know of no new
> benchmarks. Sounds like fodder for an ApacheCon presentation. :)
> 
> Tomcat-pool does have some features that DBCP2 still lacks (such as
> interceptors), which probably means that it serves a niche. I'm not sure
> how large that niche is.

I'm aware of a couple of clients at $work that had performance issues
with DBCP1 that Tomcat-JDBC solved.

I'm not aware of anyone switching from DBCP2 to Tomcat-JDBC for
performance reasons. I do occasionally see folks switching in both
directions to either avoid an edge case bug or to get a particular feature.

> At some point, it probably makes sense to extract tomcat-pool from the
> Tomcat project and make it a separate entity. Probably not a TLP, but at
> least something that can be grabbed from source separately from Tomcat
> and used independently. Of course, you can already just grab
> tomcat-jdbc.jar and use it separately (right?) but we don't make it
> obvious how to do that (The build-tomcat-jdbc doesn't have a
> "description" attribute and so doesn't show up in ant -projecthelp).

We tried that before. Getting the votes to do a release was hard work. A
number of releases failed. That is why we moved to releasing it as part
of Tomcat.

> Perhaps the git-migration would be an opportunity to do that.

Or folks can just grab it from Maven Central which is what most users
would do.

FTR it has a dependency on JULI.

Mark

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tomcat.apache.org
For additional commands, e-mail: dev-help@tomcat.apache.org

Re: DBCP2 in Tomcat

[Christopher Schultz <ch...@christopherschultz.net>]

Rémy,

On 3/29/18 9:30 AM, Rémy Maucherat wrote:
> On Thu, Mar 29, 2018 at 3:15 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
> 
>> Rémy,
>>
>> On 3/29/18 8:42 AM, Rémy Maucherat wrote:
>>> On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
>>> chris@christopherschultz.net> wrote:
>>>
>>>> Rémy,
>>>>
>>>> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
>>>>> Hi,
>>>>>
>>>>> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
>>>>> "managed" package). So at work I now got some people asking about that
>>>>> removal, and that's always a bit annoying as they have to go to a
>>>> separate
>>>>> vanilla DBCP2 to get the functionality. Annoying sometimes.
>>>>>
>>>>> So it would be possible to add the classes in Tomcat (including the
>>>>> javax.transaction to build, as that's the Tomcat way to deal with
>> that),
>>>>> even though the user would need to add its own transaction manager to
>> do
>>>>> anything with it.
>>>>>
>>>>> Should I now add it (only in 9/trunk) or instead leave things the way
>>>> they
>>>>> are ? Both work to be honest, it's just that I've been bitten by the
>> "we
>>>>> only ship 3/4 of DBCP and I didn't know it" bug.
>>>>
>>>> I've always wondered why we bother to package-rename DBCP2 and check-in
>>>> the source into our svn repo (soon to be Git). Why not pull DBCP2 from
>>>> source during the build and re-name it on the fly?
>>>>
>>>> Because that's what was done before Tomcat 8.0 and it's not done like
>> that
>>> now :)
>>> Mark wrote this:
>>>         Switch to including Apache Commons DBCP via a package renamed svn
>>> copy
>>>         rather than building from a source release for consistency with
>>> other
>>>         Commons packages and to allow faster releases to fix DBCP related
>>>         issues. (markt)
>>>
>>> And you didn't complain then it seems.
>>
>> So we do it this way because I failed to speak-up? Unlikely.
>>
> 
> It was done in 8.0.6 with the justification I quoted above and I have to
> assume that nobody complained. I was busy doing NIO2 stuff at that time
> personally. That's all I can remember.
> 
>>
>> Anyhow, this seems like a "DBCP related issue" so we ought to be able to
>> do a "faster release" by duplicating more code, then, eh?
>>
> 
> The rationale was probably: you can fix bugs without having to wait for a
> DBCP release. I'm not certain this actually happened though.
> 
> I actually have a question. There's also that *second* connection pool,
> Tomcat JDBC, although it is more "external" as it is located inside the
> modules. So reading the list of "pros" from its docs, it seems DBCP
> addressed most (all ?) of them, except that it's still significantly
> bigger. So what's the current benefits of Tomcat JDBC over Tomcat DBCP2 ?
> Did anyone do benchmarks or anything ?

I believe Filip built that in response to the issues with DBCP at the
time. I assume that benchmarks were done at the time proving the
usefulness of the tomcat-pool over DBCP 1.x. I know of no new
benchmarks. Sounds like fodder for an ApacheCon presentation. :)

Tomcat-pool does have some features that DBCP2 still lacks (such as
interceptors), which probably means that it serves a niche. I'm not sure
how large that niche is.

At some point, it probably makes sense to extract tomcat-pool from the
Tomcat project and make it a separate entity. Probably not a TLP, but at
least something that can be grabbed from source separately from Tomcat
and used independently. Of course, you can already just grab
tomcat-jdbc.jar and use it separately (right?) but we don't make it
obvious how to do that (The build-tomcat-jdbc doesn't have a
"description" attribute and so doesn't show up in ant -projecthelp).

Perhaps the git-migration would be an opportunity to do that.

-chris

Re: DBCP2 in Tomcat

[Rémy Maucherat <re...@apache.org>]

On Thu, Mar 29, 2018 at 3:15 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:

> Rémy,
>
> On 3/29/18 8:42 AM, Rémy Maucherat wrote:
> > On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
> > chris@christopherschultz.net> wrote:
> >
> >> Rémy,
> >>
> >> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
> >>> Hi,
> >>>
> >>> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
> >>> "managed" package). So at work I now got some people asking about that
> >>> removal, and that's always a bit annoying as they have to go to a
> >> separate
> >>> vanilla DBCP2 to get the functionality. Annoying sometimes.
> >>>
> >>> So it would be possible to add the classes in Tomcat (including the
> >>> javax.transaction to build, as that's the Tomcat way to deal with
> that),
> >>> even though the user would need to add its own transaction manager to
> do
> >>> anything with it.
> >>>
> >>> Should I now add it (only in 9/trunk) or instead leave things the way
> >> they
> >>> are ? Both work to be honest, it's just that I've been bitten by the
> "we
> >>> only ship 3/4 of DBCP and I didn't know it" bug.
> >>
> >> I've always wondered why we bother to package-rename DBCP2 and check-in
> >> the source into our svn repo (soon to be Git). Why not pull DBCP2 from
> >> source during the build and re-name it on the fly?
> >>
> >> Because that's what was done before Tomcat 8.0 and it's not done like
> that
> > now :)
> > Mark wrote this:
> >         Switch to including Apache Commons DBCP via a package renamed svn
> > copy
> >         rather than building from a source release for consistency with
> > other
> >         Commons packages and to allow faster releases to fix DBCP related
> >         issues. (markt)
> >
> > And you didn't complain then it seems.
>
> So we do it this way because I failed to speak-up? Unlikely.
>

It was done in 8.0.6 with the justification I quoted above and I have to
assume that nobody complained. I was busy doing NIO2 stuff at that time
personally. That's all I can remember.

>
> Anyhow, this seems like a "DBCP related issue" so we ought to be able to
> do a "faster release" by duplicating more code, then, eh?
>

The rationale was probably: you can fix bugs without having to wait for a
DBCP release. I'm not certain this actually happened though.

I actually have a question. There's also that *second* connection pool,
Tomcat JDBC, although it is more "external" as it is located inside the
modules. So reading the list of "pros" from its docs, it seems DBCP
addressed most (all ?) of them, except that it's still significantly
bigger. So what's the current benefits of Tomcat JDBC over Tomcat DBCP2 ?
Did anyone do benchmarks or anything ?

Rémy


>
> I'm +0 on this, FTR.
>
> -chris
>
>

Re: DBCP2 in Tomcat

[Christopher Schultz <ch...@christopherschultz.net>]

Rémy,

On 3/29/18 8:42 AM, Rémy Maucherat wrote:
> On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
> chris@christopherschultz.net> wrote:
> 
>> Rémy,
>>
>> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
>>> Hi,
>>>
>>> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
>>> "managed" package). So at work I now got some people asking about that
>>> removal, and that's always a bit annoying as they have to go to a
>> separate
>>> vanilla DBCP2 to get the functionality. Annoying sometimes.
>>>
>>> So it would be possible to add the classes in Tomcat (including the
>>> javax.transaction to build, as that's the Tomcat way to deal with that),
>>> even though the user would need to add its own transaction manager to do
>>> anything with it.
>>>
>>> Should I now add it (only in 9/trunk) or instead leave things the way
>> they
>>> are ? Both work to be honest, it's just that I've been bitten by the "we
>>> only ship 3/4 of DBCP and I didn't know it" bug.
>>
>> I've always wondered why we bother to package-rename DBCP2 and check-in
>> the source into our svn repo (soon to be Git). Why not pull DBCP2 from
>> source during the build and re-name it on the fly?
>>
>> Because that's what was done before Tomcat 8.0 and it's not done like that
> now :)
> Mark wrote this:
>         Switch to including Apache Commons DBCP via a package renamed svn
> copy
>         rather than building from a source release for consistency with
> other
>         Commons packages and to allow faster releases to fix DBCP related
>         issues. (markt)
> 
> And you didn't complain then it seems.

So we do it this way because I failed to speak-up? Unlikely.

Anyhow, this seems like a "DBCP related issue" so we ought to be able to
do a "faster release" by duplicating more code, then, eh?

I'm +0 on this, FTR.

-chris

Re: DBCP2 in Tomcat

[Rémy Maucherat <re...@apache.org>]

On Thu, Mar 29, 2018 at 2:33 PM, Christopher Schultz <
chris@christopherschultz.net> wrote:

> Rémy,
>
> On 3/28/18 2:25 PM, Rémy Maucherat wrote:
> > Hi,
> >
> > In Tomcat, DBCP2 is missing the XA portion (all contained in a single
> > "managed" package). So at work I now got some people asking about that
> > removal, and that's always a bit annoying as they have to go to a
> separate
> > vanilla DBCP2 to get the functionality. Annoying sometimes.
> >
> > So it would be possible to add the classes in Tomcat (including the
> > javax.transaction to build, as that's the Tomcat way to deal with that),
> > even though the user would need to add its own transaction manager to do
> > anything with it.
> >
> > Should I now add it (only in 9/trunk) or instead leave things the way
> they
> > are ? Both work to be honest, it's just that I've been bitten by the "we
> > only ship 3/4 of DBCP and I didn't know it" bug.
>
> I've always wondered why we bother to package-rename DBCP2 and check-in
> the source into our svn repo (soon to be Git). Why not pull DBCP2 from
> source during the build and re-name it on the fly?
>
> Because that's what was done before Tomcat 8.0 and it's not done like that
now :)
Mark wrote this:
        Switch to including Apache Commons DBCP via a package renamed svn
copy
        rather than building from a source release for consistency with
other
        Commons packages and to allow faster releases to fix DBCP related
        issues. (markt)

And you didn't complain then it seems.

Rémy

Re: DBCP2 in Tomcat

[Christopher Schultz <ch...@christopherschultz.net>]

Rémy,

On 3/28/18 2:25 PM, Rémy Maucherat wrote:
> Hi,
> 
> In Tomcat, DBCP2 is missing the XA portion (all contained in a single
> "managed" package). So at work I now got some people asking about that
> removal, and that's always a bit annoying as they have to go to a separate
> vanilla DBCP2 to get the functionality. Annoying sometimes.
> 
> So it would be possible to add the classes in Tomcat (including the
> javax.transaction to build, as that's the Tomcat way to deal with that),
> even though the user would need to add its own transaction manager to do
> anything with it.
> 
> Should I now add it (only in 9/trunk) or instead leave things the way they
> are ? Both work to be honest, it's just that I've been bitten by the "we
> only ship 3/4 of DBCP and I didn't know it" bug.

I've always wondered why we bother to package-rename DBCP2 and check-in
the source into our svn repo (soon to be Git). Why not pull DBCP2 from
source during the build and re-name it on the fly?

-chris