Upstream diffs

[Karl Wright <kw...@metacarta.com>]

Hi Grant,

I think we need to figure out how we're going to tackle LCF upstream submissions for commons-httpclient and xerces-j.  The diffs 
in question are in trunk/upstream-diffs.  In both cases, the changes represent significant feature additions, so there may well 
be some politics involved in getting them accepted in their current form.

There's also a small matter of formatting, in that any added files may not yet adhere to Apache standards.  If that turns out to 
be the only problem, I'd be more than happy to reform and rediff accordingly.

Any advice as to how best to proceed here?

Thanks,
Karl

Re: Upstream diffs

[Karl Wright <kw...@metacarta.com>]

Grant Ingersoll wrote:
> On Feb 23, 2010, at 2:16 PM, Karl Wright wrote:
> 
>> Grant Ingersoll wrote:
>>> On Feb 23, 2010, at 12:58 PM, Karl Wright wrote:
>>>> Hi Grant,
>>>>
>>>> I think we need to figure out how we're going to tackle LCF upstream submissions for commons-httpclient and xerces-j.  The diffs in question are in trunk/upstream-diffs.  In both cases, the changes represent significant feature additions, so there may well be some politics involved in getting them accepted in their current form.
>>>>
>>> I guess I'd suggest that we submit patches and then also maintain our own versions as necessary by checking in the appropriate library as something like commons-httpclient-lcf (or something else that indicates we've changed it), either that or abstract them above the library as part of LCF code so that no modification of the actual library is required.
>> Well, the changes involved can't be structured as "wrapping", I'm afraid, or that's exactly how we'd have done it.
>> I'll rename the two libraries that are checked in.
>>
>> The xerces feature was already rejected by one of the maintainers.  It basically allows you to enable resilience in the face of encoding errors, which we found quite a number of times in various RSS feeds.  For some reason they didn't see the point.
> 
> That's fine.  As long as the patch lives somewhere out in the open so that we can say:  We applied patch XYZ (located at ...) to Xerces version A.B.
> 

Currently that's what's in upstream-diffs.

>> The biggest problem with patches will be in commons-httpclient, because there are several different features involved.  I'll try to tease them apart so they can be attached to independent JIRA tickets.  The big one there is complete NTLMv1, v2, and NTLM2 Session support.  I sent that one along at one point but got no response whatsoever.  There was also a feature involving the ability to supply your own protocol factory and have that be used everywhere, which I didn't even try to submit after the experience with the first one.
> 
> Same deal as above.
> 
>> I'll try it all again, see if anything different happens this time if I mention this is required by LCF.
> 
> If you've already submitted them, then I wouldn't bother.  However, it is often the case that Open Source libs find it hard to digest large patches, so if they can be split out to smaller ones, that might be better.

I broke the httpclient submission into three separate issues, and entered tickets and patches for them against the 3.1 
codestream.  The response was invariably that I should be using 4.x.  That would be fine, except that they've actually *removed* 
NTLM support from 4.x entirely - you now need jcifs (LGPL, remember) and a wrapper class to do your NTLM work for you.  It's 
unsupported as well - and that worries me because a lot of the problems we had with HttpClient had to do with state transitions 
involving NTLM.  This stuff is probably no longer even perfunctorily tested.  Nevertheless, since I submitted a fully tested, 
ASF granted implementation of NTLM, I have hopes that they may add NTLM support back into the 4.x stream.  Crossing my fingers 
here... I suppose we'll hear back sometime today as to whether they take this seriously or not.  If they don't, due to Apache 
licensing restrictions, all connectors that depend on httpclient would technically become optional also.

Plus, even the "testing" distribution of debian does not include 4.x - it still ships 3.1-9.  Given that the whole API has 
changed, it's not terribly surprising that this is the case.

So, for the moment, I think we'll need to stick with 3.1 and patches, until I either have time to port my implementation of 
these changes to 4.x, test them, and get them accepted, or until the world changes in some other way.

I intend to tackle the xerces-j issue today, to find out whether we're truly on our own here as well.

Karl

Re: Upstream diffs

[Grant Ingersoll <gs...@apache.org>]

On Feb 23, 2010, at 2:16 PM, Karl Wright wrote:

> Grant Ingersoll wrote:
>> On Feb 23, 2010, at 12:58 PM, Karl Wright wrote:
>>> Hi Grant,
>>> 
>>> I think we need to figure out how we're going to tackle LCF upstream submissions for commons-httpclient and xerces-j.  The diffs in question are in trunk/upstream-diffs.  In both cases, the changes represent significant feature additions, so there may well be some politics involved in getting them accepted in their current form.
>>> 
>> I guess I'd suggest that we submit patches and then also maintain our own versions as necessary by checking in the appropriate library as something like commons-httpclient-lcf (or something else that indicates we've changed it), either that or abstract them above the library as part of LCF code so that no modification of the actual library is required.
> 
> Well, the changes involved can't be structured as "wrapping", I'm afraid, or that's exactly how we'd have done it.
> I'll rename the two libraries that are checked in.
> 
> The xerces feature was already rejected by one of the maintainers.  It basically allows you to enable resilience in the face of encoding errors, which we found quite a number of times in various RSS feeds.  For some reason they didn't see the point.

That's fine.  As long as the patch lives somewhere out in the open so that we can say:  We applied patch XYZ (located at ...) to Xerces version A.B.

> 
> The biggest problem with patches will be in commons-httpclient, because there are several different features involved.  I'll try to tease them apart so they can be attached to independent JIRA tickets.  The big one there is complete NTLMv1, v2, and NTLM2 Session support.  I sent that one along at one point but got no response whatsoever.  There was also a feature involving the ability to supply your own protocol factory and have that be used everywhere, which I didn't even try to submit after the experience with the first one.

Same deal as above.

> 
> I'll try it all again, see if anything different happens this time if I mention this is required by LCF.

If you've already submitted them, then I wouldn't bother.  However, it is often the case that Open Source libs find it hard to digest large patches, so if they can be split out to smaller ones, that might be better.

Re: Upstream diffs

[Karl Wright <kw...@metacarta.com>]

Grant Ingersoll wrote:
> On Feb 23, 2010, at 12:58 PM, Karl Wright wrote:
> 
>> Hi Grant,
>>
>> I think we need to figure out how we're going to tackle LCF upstream submissions for commons-httpclient and xerces-j.  The diffs in question are in trunk/upstream-diffs.  In both cases, the changes represent significant feature additions, so there may well be some politics involved in getting them accepted in their current form.
>>
> 
> I guess I'd suggest that we submit patches and then also maintain our own versions as necessary by checking in the appropriate library as something like commons-httpclient-lcf (or something else that indicates we've changed it), either that or abstract them above the library as part of LCF code so that no modification of the actual library is required.
> 

Well, the changes involved can't be structured as "wrapping", I'm afraid, or that's exactly how we'd have done it.
I'll rename the two libraries that are checked in.

The xerces feature was already rejected by one of the maintainers.  It basically allows you to enable resilience in the face of 
encoding errors, which we found quite a number of times in various RSS feeds.  For some reason they didn't see the point.

The biggest problem with patches will be in commons-httpclient, because there are several different features involved.  I'll try 
to tease them apart so they can be attached to independent JIRA tickets.  The big one there is complete NTLMv1, v2, and NTLM2 
Session support.  I sent that one along at one point but got no response whatsoever.  There was also a feature involving the 
ability to supply your own protocol factory and have that be used everywhere, which I didn't even try to submit after the 
experience with the first one.

I'll try it all again, see if anything different happens this time if I mention this is required by LCF.

Karl

> 
>> There's also a small matter of formatting, in that any added files may not yet adhere to Apache standards.  If that turns out to be the only problem, I'd be more than happy to reform and rediff accordingly.
>>
>> Any advice as to how best to proceed here?
>>
>> Thanks,
>> Karl
>>
> 
> 


-- 
Karl Wright
Software Engineer

MetaCarta, Inc.
350 Massachusetts Avenue, 4th Floor, Cambridge, MA 02139 USA

(617)-301-5511

www.metacarta.com <http://www.metacarta.com>
Where to find it.

This message may contain privileged, proprietary, and otherwise private
information. If you are not the intended recipient, please notify the
sender immediately.

Re: Upstream diffs

[Grant Ingersoll <gs...@apache.org>]

On Feb 23, 2010, at 12:58 PM, Karl Wright wrote:

> 
> Hi Grant,
> 
> I think we need to figure out how we're going to tackle LCF upstream submissions for commons-httpclient and xerces-j.  The diffs in question are in trunk/upstream-diffs.  In both cases, the changes represent significant feature additions, so there may well be some politics involved in getting them accepted in their current form.
> 

I guess I'd suggest that we submit patches and then also maintain our own versions as necessary by checking in the appropriate library as something like commons-httpclient-lcf (or something else that indicates we've changed it), either that or abstract them above the library as part of LCF code so that no modification of the actual library is required.


> There's also a small matter of formatting, in that any added files may not yet adhere to Apache standards.  If that turns out to be the only problem, I'd be more than happy to reform and rediff accordingly.
> 
> Any advice as to how best to proceed here?
> 
> Thanks,
> Karl
>