You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@nutch.apache.org by Pascal Beis <pa...@gmail.com> on 2006/08/07 10:17:33 UTC
Patch: deflate encoding
Hi all,
I'v added support for deflate encoding (next to gzip) to nutch. Is there
interest to
include this into the main source repository?
Patch attached.
Cheers
Pascal
Re: Patch: deflate encoding
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
Just to complete this thread (for the archives :), Deflater in the JDK
has a... "feature" -- flush() is basically not implemented and thus
nonfunctional on compressed streams. This is a known limitation (bug's
parade mentions it as an 'enhancement request', although for most people
who faced the problem it'll be a plain bug.
Anyway, the workaround is to use a custom deflater (such as zlib) and
perform Z_SYNC_FLUSH which pads the compressed stream to a complete
block and allows flushing the content. This way you can flush a partial
compressed stream to the browser (for these DHTML lovers who like to
play with JavaScript, for instance).
We implemented a fixed GZIP/Deflate compression based on JZlib and
PJL-comp-filter (which in turn we changed slightly to compile under
JDK1.4). If you're interested, sources are in Carrot2 SVN.
https://svn.sourceforge.net/svnroot/carrot2/trunk/carrot2/components/carrot2-util-gzip/
Dawid
Dawid Weiss wrote:
>
> I believe both deflate and gzip (as well as zip) are included as servlet
> filters in:
>
> http://sourceforge.net/projects/pjl-comp-filter/
>
> Dawid
>
> Pascal Beis wrote:
>> Hi all,
>>
>> I'v added support for deflate encoding (next to gzip) to nutch. Is there
>> interest to
>> include this into the main source repository?
>>
>> Patch attached.
>>
>> Cheers
>>
>> Pascal
>>
Re: Patch: deflate encoding
Posted by Dawid Weiss <da...@cs.put.poznan.pl>.
I believe both deflate and gzip (as well as zip) are included as servlet
filters in:
http://sourceforge.net/projects/pjl-comp-filter/
Dawid
Pascal Beis wrote:
> Hi all,
>
> I'v added support for deflate encoding (next to gzip) to nutch. Is there
> interest to
> include this into the main source repository?
>
> Patch attached.
>
> Cheers
>
> Pascal
>
Re: Patch: deflate encoding
Posted by Pascal Beis <pa...@gmail.com>.
On 8/8/06, ogjunk-nutch@yahoo.com <og...@yahoo.com> wrote:
>
> Forgot to say - attachments get stripped. Please put them in JIRA.
Done that, see https://issues.apache.org/jira/browse/NUTCH-345
Cheers
Jan-Pascal
Re: Patch: deflate encoding
Posted by og...@yahoo.com.
Ja, ja!
Otis
----- Original Message ----
From: Pascal Beis
To: nutch-dev@lucene.apache.org
Sent: Monday, August 7, 2006 4:17:33 AM
Subject: Patch: deflate encoding
Hi all,
I'v added support for deflate encoding (next to gzip) to nutch. Is there interest to
include this into the main source repository?
Patch attached.
Cheers
Pascal
Re: Patch: deflate encoding
Posted by og...@yahoo.com.
Pascal,
Forgot to say - attachments get stripped. Please put them in JIRA.
Thanks,
Otis
----- Original Message ----
From: Pascal Beis <pa...@gmail.com>
To: nutch-dev@lucene.apache.org
Sent: Monday, August 7, 2006 4:17:33 AM
Subject: Patch: deflate encoding
Hi all,
I'v added support for deflate encoding (next to gzip) to nutch. Is there interest to
include this into the main source repository?
Patch attached.
Cheers
Pascal