You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hc.apache.org by "arturobernalg (via GitHub)" <gi...@apache.org> on 2024/09/19 20:08:53 UTC
[PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
arturobernalg opened a new pull request, #580:
URL: https://github.com/apache/httpcomponents-client/pull/580
This PR addresses [HTTPCLIENT-1843](https://issues.apache.org/jira/browse/HTTPCLIENT-1843), allowing Apache HttpClient to delegate compression handling to Apache Commons Compress. The goal is to provide broader support for various compression algorithms while reducing custom implementation and maintenance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768651788
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DeflateInputStream.java:
##########
@@ -38,7 +38,9 @@
/**
* Deflates an input stream. This class includes logic needed for various RFCs in order
* to reasonably implement the "deflate" compression algorithm.
+ * @deprecated
Review Comment:
The tag comment is missing: `Use {@link ...}.`
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DeflateInputStreamFactory.java:
##########
@@ -37,7 +37,9 @@
* {@link InputStreamFactory} for handling Deflate Content Coded responses.
*
* @since 5.0
+ * @deprecated
Review Comment:
The tag comment is missing: `Use {@link ...}.`
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DeflateDecompressingEntity.java:
##########
@@ -43,7 +43,9 @@
* @see GzipDecompressingEntity
*
* @since 4.1
+ * @deprecated
Review Comment:
The tag comment is missing: `Use {@link ...}.`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2482178234
> @arturobernalg
>
> Thank you for the review on [apache/commons-compress#605](https://github.com/apache/commons-compress/pull/605), I merged that PR.
All right. Let me know if we need to do another change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2847700666
@ok2c would you mind to validate this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2362136272
> 1.27.1
changed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768647926
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
+ }
+ return content;
+ }
+ return getDecompressingStream();
+ }
+
+ /**
+ * Writes the decompressed content to the specified {@link OutputStream}.
+ *
+ * @param outStream the {@link OutputStream} to which the decompressed content is written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during writing or decompression.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
Review Comment:
You can reuse Commons IO's `IOUtils.copy(InputStream, OutputStream) which Commons Compress uses already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768652643
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/LazyDecompressInputStream.java:
##########
@@ -0,0 +1,209 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.FilterInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.UncheckedIOException;
+
+import org.apache.hc.core5.io.Closer;
+
+
+/**
+ * A {@link FilterInputStream} that lazily initializes and applies decompression on the underlying input stream.
+ * This class supports multiple compression types and uses {@link CompressorFactory} to obtain the appropriate
+ * decompression stream when the first read operation occurs.
+ *
+ * <p>This implementation delays the creation of the decompression stream until it is required, optimizing
+ * the performance when the stream may not be read immediately or at all.</p>
+ *
+ * @since 5.4
+ */
+public class LazyDecompressInputStream extends FilterInputStream {
+
+ /**
+ * The lazily initialized decompression stream.
+ */
+ private InputStream wrapperStream;
+
+ /**
+ * The compression type used to determine which decompression algorithm to apply (e.g., "gzip", "deflate").
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
Review Comment:
Camel case name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2365037894
> Thank you for your update.
>
> * I have comments embedded throughout.
>
> * There is some lazy-initialization code here and there, this should be called out in the class Javadoc making the class thread-unsafe, we have an annotation for that: `org.apache.hc.core5.annotation.ThreadingBehavior`.
HI @garydgregory
I believe I’ve covered all the changes you requested.
TY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1842446270
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
Review Comment:
HI @garydgregory
Can you do another pass??
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1817920989
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
Review Comment:
Thanks, @garydgregory . Agreed—the Javadoc and implementation don’t match. We’ll:
- Throw an exception on unsupported formats instead of returning the original stream, aligning with Javadoc expectations.
- Rename to getDecompressorInputStream to reflect decompression.
- Standardize to “format type” instead of “compression format” to clarify.
-
These changes should make both the API and docs clearer. Thanks for flagging this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2479887725
> Hi @arturobernalg
>
> * TY for the work 👍
>
> * Minor: For the Javadoc `since` tags, I think the next feature release will be `5.5` since the current release is `5.4.1` and the git master POM already has 5.5 in it.
>
> * Side note: In Commons Compress, `CompressorException` usually wraps an `IOException`, so it's too bad that we wrap it in another IOException here ;-) Since compression and decompression is IO, I'll see about making CompressorException extend IOException.
Hi @garydgregory ,
Thanks for the feedback!. I've updated all the @since tags to reflect version 5.5 as suggested. Regarding the `CompressorException` wrapping `IOException`, I’ll wait for updates on whether it will extend IOException in Commons Compress. This will determine the best course of action here.
Let me know if there’s anything else to address.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768616169
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,294 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ private static final Map<String, String> COMPRESSION_ALIASES;
+ static {
+ final Map<String, String> aliases = new HashMap<>();
Review Comment:
@arturobernalg
Should I add an API to return `org.apache.commons.compress.compressors.CompressorStreamFactory.ALL_NAMES`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768580758
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,135 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.4
Review Comment:
Update `since` tags to `5.5` since we just released `5.4`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768640485
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
Review Comment:
If we do not lock, we should document this class as not thread-safe.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768616169
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,294 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ private static final Map<String, String> COMPRESSION_ALIASES;
+ static {
+ final Map<String, String> aliases = new HashMap<>();
Review Comment:
@arturobernalg
Why just those stream types?
How about using:
- `org.apache.commons.compress.compressors.CompressorStreamProvider.getInputStreamCompressorNames()`
- `org.apache.commons.compress.compressors.CompressorStreamProvider.getOutputStreamCompressorNames()`
- Should I add an API to return `org.apache.commons.compress.compressors.CompressorStreamFactory.ALL_NAMES`?
-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1769261744
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/extension/sync/StandardTestClientBuilder.java:
##########
@@ -72,6 +74,12 @@ public TestClientBuilder setTimeout(final Timeout timeout) {
return this;
}
+ @Override
+ public TestClientBuilder setnowrap(final boolean nowrap) {
Review Comment:
Camel case "noWrap".
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/sync/AbstractIntegrationTestBase.java:
##########
@@ -78,4 +78,8 @@ public TestClient client() throws Exception {
return testResources.client();
}
+ public TestClient client(final boolean nowrap) throws Exception {
Review Comment:
Camel case "noWrap".
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/extension/sync/StandardTestClientBuilder.java:
##########
@@ -57,6 +57,8 @@ final class StandardTestClientBuilder implements TestClientBuilder {
private HttpClientConnectionManager connectionManager;
+ private boolean nowrap;
Review Comment:
Camel case "noWrap".
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
Review Comment:
Use constants instead of magic strings.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.java:
##########
@@ -696,13 +698,62 @@ public final HttpClientBuilder setDefaultCookieSpecRegistry(
* to be used for automatic content decompression.
*
* @return this instance.
+ * @deprecated Use {@link HttpClientBuilder#setEncodings(List)}
*/
+ @Deprecated
public final HttpClientBuilder setContentDecoderRegistry(
final LinkedHashMap<String, InputStreamFactory> contentDecoderMap) {
this.contentDecoderMap = contentDecoderMap;
+ setEncodings(new ArrayList<>(contentDecoderMap.keySet()));
+ return this;
+ }
+
+
+ /**
+ * Sets the list of content encodings to be supported for automatic content compression and decompression.
+ * <p>
+ * This method allows the user to provide a list of content encodings that the client will support during HTTP
+ * requests and responses. It enables automatic handling of compressed data streams for specified encodings.
+ * Supported encodings could include algorithms like "gzip", "deflate", "br" (Brotli), or custom encodings
+ * depending on the available compression implementations.
+ * </p>
+ *
+ * <pre>
+ * Example:
+ * {@code
+ * List<String> supportedEncodings = Arrays.asList("gzip", "deflate", "br");
+ * HttpClientBuilder builder = HttpClientBuilder.create()
+ * .setEncodings(supportedEncodings);
+ * }
+ * </pre>
+ *
+ * @param encodings a list of encoding names to support for automatic content compression and decompression
+ * @return this {@code HttpClientBuilder} instance for method chaining
+ * @since 5.0
+ */
+ public final HttpClientBuilder setEncodings(final List<String> encodings) {
+ this.encodings = encodings;
+ return this;
+ }
+
+ /**
+ * Sets the "nowrap" option for the HTTP client.
+ * <p>
+ * When enabled, this option disables the zlib header and trailer in deflate compression streams.
+ * This is useful when working with servers that require or expect raw deflate streams without
+ * the standard zlib header and trailer.
+ * </p>
+ *
+ * @param nowrap if {@code true}, disables the zlib header and trailer in deflate streams.
+ * @return the updated {@link HttpClientBuilder} instance.
+ * @since 5.4
+ */
+ public final HttpClientBuilder setNowrap(final boolean nowrap) {
Review Comment:
Camel case "noWrap".
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+
+ }
+
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
Review Comment:
Inline single use "inputNames".
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/extension/sync/TestClientBuilder.java:
##########
@@ -98,6 +98,10 @@ default TestClientBuilder addExecInterceptorLast(String name, ExecChainHandler
throw new UnsupportedOperationException("Operation not supported by " + getProtocolLevel());
}
+ default TestClientBuilder setnowrap(boolean nowrap){
Review Comment:
- Extra space after "default"
- Camel case "noWrap".
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,132 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
Review Comment:
Remove extra blank line.
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/extension/sync/TestClientResources.java:
##########
@@ -118,4 +118,9 @@ public TestClient client() throws Exception {
return client;
}
+ public TestClient client(final boolean nowrap) throws Exception {
Review Comment:
Camel case "noWrap", "setNoWrap()".
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,132 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
+ public boolean isChunked() {
+ return super.isChunked();
+ }
+
+
+ /**
+ * This method is unsupported because the content is meant to be compressed during the
Review Comment:
First sentence should be "Throws UnsupportedOperationException."
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,132 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
+ public boolean isChunked() {
+ return super.isChunked();
+ }
+
+
+ /**
+ * This method is unsupported because the content is meant to be compressed during the
+ * {@link #writeTo(OutputStream)} operation.
+ *
+ * @throws UnsupportedOperationException always, as this method is not supported.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ throw new UnsupportedOperationException("Reading content is not supported for CompressingEntity");
+ }
+
+ /**
+ * Writes the compressed content to the provided {@link OutputStream}. Compression is performed
+ * using the content encoding provided during entity construction.
+ *
+ * @param outStream the {@link OutputStream} to which the compressed content will be written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during compression or writing.
+ * @throws UnsupportedOperationException if the specified compression type is not supported.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
+
Review Comment:
Remove extra blank line.
##########
httpclient5-testing/src/test/java/org/apache/hc/client5/testing/sync/compress/CompressedResponseHandlingExample.java:
##########
@@ -0,0 +1,113 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.testing.sync.compress;
+
+import java.util.Arrays;
+import java.util.List;
+
+import org.apache.hc.client5.http.entity.CompressorFactory;
+import org.apache.hc.client5.http.impl.classic.CloseableHttpClient;
+import org.apache.hc.client5.http.impl.classic.HttpClients;
+import org.apache.hc.core5.http.ClassicHttpRequest;
+import org.apache.hc.core5.http.ContentType;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.HttpHeaders;
+import org.apache.hc.core5.http.HttpHost;
+import org.apache.hc.core5.http.io.entity.EntityUtils;
+import org.apache.hc.core5.http.io.entity.StringEntity;
+import org.apache.hc.core5.http.io.support.ClassicRequestBuilder;
+import org.apache.hc.core5.io.CloseMode;
+import org.apache.hc.core5.testing.classic.ClassicTestServer;
+
+
Review Comment:
Remove extra blank line.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,132 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
+ public boolean isChunked() {
+ return super.isChunked();
+ }
+
+
+ /**
+ * This method is unsupported because the content is meant to be compressed during the
+ * {@link #writeTo(OutputStream)} operation.
+ *
+ * @throws UnsupportedOperationException always, as this method is not supported.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ throw new UnsupportedOperationException("Reading content is not supported for CompressingEntity");
+ }
+
+ /**
+ * Writes the compressed content to the provided {@link OutputStream}. Compression is performed
+ * using the content encoding provided during entity construction.
+ *
+ * @param outStream the {@link OutputStream} to which the compressed content will be written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during compression or writing.
+ * @throws UnsupportedOperationException if the specified compression type is not supported.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
+
+ // Get the compressor based on the specified content encoding
+ final OutputStream compressorStream;
+ try {
+ compressorStream = CompressorFactory.INSTANCE.getCompressorOutputStream(contentEncoding, outStream);
+ } catch (final CompressorException e) {
+ throw new IOException("Error initializing decompression stream", e);
+ }
+
Review Comment:
Remove extra blank line.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,132 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
Review Comment:
Remove method, all it does is call super(); anything to document can go in the class comment.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
Review Comment:
Remove extra blank line.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+
+ }
+
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
+ return inputNames.stream()
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ }
+
+ /**
+ * Fetches the available output stream compression providers from Commons Compress.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableOutputProviders() {
+ final Set<String> outputNames = compressorStreamFactory.getOutputStreamCompressorNames();
+ return outputNames.stream()
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ }
+
+ /**
+ * Creates a compressor input stream for the given compression format and input stream.
+ * <p>
+ * This method handles the special case for deflate compression where the zlib header can be optionally included.
+ * The noWrap parameter directly controls the behavior of the zlib header:
+ * - If noWrap is {@code true}, the deflate stream is processed without zlib headers (raw Deflate).
+ * - If noWrap is {@code false}, the deflate stream includes the zlib header.
+ * </p>
+ *
+ * @param name the compression format (e.g., "gzip", "deflate").
+ * @param inputStream the input stream to decompress; must not be {@code null}.
+ * @param noWrap if {@code true}, disables the zlib header and trailer for deflate streams (raw Deflate).
+ * @return a decompressed input stream, or {@code null} if an error occurs during stream creation.
+ * @throws CompressorException if an error occurs while creating the compressor input stream or if the compression format is unsupported.
+ */
+ private InputStream createCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ if ("deflate".equalsIgnoreCase(name)) {
Review Comment:
Use constant instead of magic string.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+
Review Comment:
Remove extra blank line.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.java:
##########
@@ -234,6 +232,10 @@ private ExecInterceptorEntry(
private List<Closeable> closeables;
+ private List<String> encodings;
+
+ private boolean nowrap;
Review Comment:
Camel case "noWrap".
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/LazyDecompressInputStream.java:
##########
@@ -0,0 +1,214 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.FilterInputStream;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.UncheckedIOException;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.io.Closer;
+
+
+/**
+ * A {@link FilterInputStream} that lazily initializes and applies decompression on the underlying input stream.
+ * This class supports multiple compression types and uses {@link CompressorFactory} to obtain the appropriate
+ * decompression stream when the first read operation occurs.
+ *
+ * <p>This implementation delays the creation of the decompression stream until it is required, optimizing
+ * the performance when the stream may not be read immediately or at all.</p>
+ *
+ * @since 5.5
+ */
+public class LazyDecompressInputStream extends FilterInputStream {
+
+ /**
+ * The lazily initialized decompression stream.
+ */
+ private InputStream wrapperStream;
+
+ /**
+ * The compression type used to determine which decompression algorithm to apply (e.g., "gzip", "deflate").
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (noWrap).
+ */
+ private final boolean noWrap;
+
+ /**
+ * Constructs a new {@link LazyDecompressInputStream} that applies the specified compression type and noWrap setting.
+ *
+ * @param wrappedStream the non-null {@link InputStream} to be wrapped and decompressed.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param noWrap whether to decompress without headers for certain compression formats.
+ */
+ public LazyDecompressInputStream(final InputStream wrappedStream, final String compressionType, final boolean noWrap) {
+ super(wrappedStream);
+ this.compressionType = compressionType;
+ this.noWrap = noWrap;
+ }
+
+ /**
+ * Constructs a new {@link LazyDecompressInputStream} that applies the specified compression type,
+ * defaulting to no noWrap handling.
+ *
+ * @param wrappedStream the non-null {@link InputStream} to be wrapped and decompressed.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public LazyDecompressInputStream(final InputStream wrappedStream, final String compressionType) {
+ this(wrappedStream, compressionType, false);
+ }
+
+ /**
+ * Initializes the decompression wrapper stream lazily, based on the compression type and noWrap flag.
+ *
+ * @return the initialized decompression stream.
+ * @throws IOException if an error occurs during initialization.
+ */
+ private InputStream initWrapper() throws IOException {
+ if (wrapperStream == null) {
Review Comment:
It does not look like there is any locking here, so the class should be Javadoc'd as thread unsafe.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/impl/classic/HttpClientBuilder.java:
##########
@@ -696,13 +698,62 @@ public final HttpClientBuilder setDefaultCookieSpecRegistry(
* to be used for automatic content decompression.
*
* @return this instance.
+ * @deprecated Use {@link HttpClientBuilder#setEncodings(List)}
*/
+ @Deprecated
public final HttpClientBuilder setContentDecoderRegistry(
final LinkedHashMap<String, InputStreamFactory> contentDecoderMap) {
this.contentDecoderMap = contentDecoderMap;
+ setEncodings(new ArrayList<>(contentDecoderMap.keySet()));
+ return this;
+ }
+
Review Comment:
Remove extra blank line.
##########
httpclient5/src/test/java/org/apache/hc/client5/http/entity/TestGZip.java:
##########
@@ -47,46 +47,56 @@
class TestGZip {
@Test
- void testBasic() throws Exception {
+ void testBasic() {
final String s = "some kind of text";
- final StringEntity e = new StringEntity(s, ContentType.TEXT_PLAIN, false);
- try (final GzipCompressingEntity gzipe = new GzipCompressingEntity(e)) {
- Assertions.assertTrue(gzipe.isChunked());
- Assertions.assertEquals(-1, gzipe.getContentLength());
- Assertions.assertNotNull(gzipe.getContentEncoding());
- Assertions.assertEquals("gzip", gzipe.getContentEncoding());
- }
+ final HttpEntity entity = CompressorFactory.INSTANCE.decompressEntity(new StringEntity(s, ContentType.TEXT_PLAIN, false), "gz");
+ Assertions.assertEquals(17, entity.getContentLength());
+ Assertions.assertNotNull(entity.getContentEncoding());
+ Assertions.assertEquals("gz", entity.getContentEncoding());
}
@Test
void testCompressionDecompression() throws Exception {
final StringEntity in = new StringEntity("some kind of text", ContentType.TEXT_PLAIN);
- try (final GzipCompressingEntity gzipe = new GzipCompressingEntity(in)) {
- final ByteArrayOutputStream buf = new ByteArrayOutputStream();
- gzipe.writeTo(buf);
- final ByteArrayEntity out = new ByteArrayEntity(buf.toByteArray(), ContentType.APPLICATION_OCTET_STREAM);
- final GzipDecompressingEntity gunzipe = new GzipDecompressingEntity(out);
- Assertions.assertEquals("some kind of text", EntityUtils.toString(gunzipe, StandardCharsets.US_ASCII));
- }
+
Review Comment:
Remove extra blank lines, the comments suffice.
##########
httpclient5/src/test/java/org/apache/hc/client5/http/entity/TestEntityBuilder.java:
##########
@@ -116,12 +120,32 @@ void testBuildChunked() {
@Test
void testBuildGZipped() {
- final HttpEntity entity = EntityBuilder.create().setText("stuff").gzipCompressed().build();
+ final HttpEntity entity = EntityBuilder.create().setText("stuff").setCompressed(true).setContentEncoding("gzip").build();
Assertions.assertNotNull(entity);
Assertions.assertNotNull(entity.getContentType());
Assertions.assertEquals("text/plain; charset=UTF-8", entity.getContentType());
Assertions.assertNotNull(entity.getContentEncoding());
- Assertions.assertEquals("gzip", entity.getContentEncoding());
+ Assertions.assertEquals("gz", entity.getContentEncoding());
+ }
+
+ @Test
+ public void testCompressionDecompression() throws Exception {
+
Review Comment:
Remove all blank like, if something needs to be called out, `// say it`.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,282 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+
+ }
+
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
+ return inputNames.stream()
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ }
+
+ /**
+ * Fetches the available output stream compression providers from Commons Compress.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableOutputProviders() {
+ final Set<String> outputNames = compressorStreamFactory.getOutputStreamCompressorNames();
Review Comment:
Inline single use "outputNames".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2480552696
> Hi @arturobernalg,
>
> What do you think about [apache/commons-compress#605](https://github.com/apache/commons-compress/pull/605)
> Hi @arturobernalg,
>
> What do you think about [apache/commons-compress#605](https://github.com/apache/commons-compress/pull/605)
Hi @garydgregory
The changes make sense. Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1810552783
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
Review Comment:
This API should not do any logging, especially at such a high level. This is just a string-to-string conversion API, it has no business rasing warnings IMO.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressingEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
+ return inputNames.stream()
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ }
+
+ /**
+ * Fetches the available output stream compression providers from Commons Compress.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableOutputProviders() {
Review Comment:
The "fetch" method name prefix is unusual and surprising to me, I would follow the "get" naming convention.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
Review Comment:
This class should not do any logging IMO, especially warnings, it should throw an illegal argument exception or let the call site deal with a null result.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
Review Comment:
I don't understand what this Javadoc means. What's a "formatted" name? Maybe the API is misnamed? Is the job of the method to map from a content type value to a Commons Compress key? Is it something else? I think the comment should at least make it clear, which should guide us to a better API name.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingEntity.java:
##########
@@ -0,0 +1,127 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressingFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
+ public boolean isChunked() {
+ return super.isChunked();
+ }
+
+ /**
+ * This method is unsupported because the content is meant to be compressed during the
+ * {@link #writeTo(OutputStream)} operation.
+ *
+ * @throws UnsupportedOperationException always, as this method is not supported.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ throw new UnsupportedOperationException("Reading content is not supported for CompressingEntity");
+ }
+
+ /**
+ * Writes the compressed content to the provided {@link OutputStream}. Compression is performed
+ * using the content encoding provided during entity construction.
+ *
+ * @param outStream the {@link OutputStream} to which the compressed content will be written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during compression or writing.
+ * @throws UnsupportedOperationException if the specified compression type is not supported.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
+ // Get the compressor based on the specified content encoding
+ final OutputStream compressorStream;
+ try {
+ compressorStream = CompressingFactory.INSTANCE.getCompressorOutputStream(contentEncoding, outStream);
+ } catch (final CompressorException e) {
+ throw new IOException("Error initializing decompression stream", e);
Review Comment:
Don't you mean "Error initializing compression stream"?
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
Review Comment:
The Javadoc and implementation don't match:
- Javadoc states decompression is performed
- The code doesn't decompress if the format isn't supported.
Either the Javadoc needs updating, or, more likely, we should throw an exception indicating the operation is not supported for that format type name.
I must say I find the API and Javadoc confusing, we are creating a "Compressor" input stream in order to "decompress" with a "compression" format! What?
- Maybe calling the API `getDecompressorInputStream` would help because it "decompresses". I expect a "Compressor" to compress, not the other way around.
- Maybe using the term "format type name" instead "compression format" would remove the need for the reader to constantly parse "compress" vs. "decompress" in the text.
- Maybe only using one term "decompress" or "compress" for one method (Javadoc and code) would alleviate the issue.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressingEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
+ return inputNames.stream()
+ .map(String::toLowerCase)
Review Comment:
This might have some odd results depending on the locale context (https://garygregory.wordpress.com/2015/11/03/java-lowercase-conversion-turkey/) if you do not use a Locale like the PR does above using ROOT.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressingEntity.java:
##########
@@ -38,7 +38,9 @@
* Common base class for decompressing {@link HttpEntity} implementations.
*
* @since 4.4
+ * @deprecated
Review Comment:
The Javadoc tag is missing its comment to redirect reader.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/DecompressingEntity.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.hc.core5.annotation.Contract;
+import org.apache.hc.core5.annotation.ThreadingBehavior;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., noWrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressingInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * <p>
+ * Note: This class uses lazy initialization for certain fields, making it <b>not thread-safe</b>.
+ * If multiple threads access an instance of this class concurrently, they must synchronize on the instance
+ * to ensure correct behavior.
+ * </p>
+ *
+ * @since 5.5
+ */
+@Contract(threading = ThreadingBehavior.UNSAFE)
+public class DecompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (noWrap).
+ */
+ private final boolean noWrap;
+
+ /**
+ * Constructs a new {@link DecompressingEntity} with the specified compression type and noWrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param noWrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressingEntity(final HttpEntity wrapped, final String compressionType, final boolean noWrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.noWrap = noWrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressingEntity} with the specified compression type, defaulting to no noWrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressingEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressingInputStream(super.getContent(), compressionType, noWrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
Review Comment:
As I commented elsewhere in this PR, the thread safety or lack thereof needs to be documented if not addressed.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingFactory.java:
##########
@@ -0,0 +1,277 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ *
+ * @since 5.5
+ */
+public class CompressingFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressingFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressingFactory INSTANCE = new CompressingFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return inputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableInputProviders());
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return outputProvidersCache.updateAndGet(existing -> existing != null ? existing : fetchAvailableOutputProviders());
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> {
+ if ("gzip".equals(key) || "x-gzip".equals(key)) {
+ return "gz";
+ } else if ("compress".equals(key)) {
+ return "z";
+ }
+ return key;
+ });
+ }
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "noWrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean noWrap) throws CompressorException {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, noWrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) throws CompressorException {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param noWrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean noWrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressingEntity(entity, contentEncoding, noWrap);
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Fetches the available input stream compression providers from Commons Compress.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableInputProviders() {
+ final Set<String> inputNames = compressorStreamFactory.getInputStreamCompressorNames();
+ return inputNames.stream()
+ .map(String::toLowerCase)
+ .collect(Collectors.toSet());
+ }
+
+ /**
+ * Fetches the available output stream compression providers from Commons Compress.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ private Set<String> fetchAvailableOutputProviders() {
+ final Set<String> outputNames = compressorStreamFactory.getOutputStreamCompressorNames();
+ return outputNames.stream()
+ .map(String::toLowerCase)
Review Comment:
See above.
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/compress/CompressingEntity.java:
##########
@@ -0,0 +1,127 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity.compress;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.compress.compressors.CompressorException;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressingFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.5
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+ /**
+ * Returns whether the entity is chunked. This is determined by the wrapped entity.
+ *
+ * @return {@code true} if the entity is chunked, {@code false} otherwise.
+ */
+ @Override
+ public boolean isChunked() {
+ return super.isChunked();
+ }
+
+ /**
+ * This method is unsupported because the content is meant to be compressed during the
+ * {@link #writeTo(OutputStream)} operation.
+ *
+ * @throws UnsupportedOperationException always, as this method is not supported.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ throw new UnsupportedOperationException("Reading content is not supported for CompressingEntity");
+ }
+
+ /**
+ * Writes the compressed content to the provided {@link OutputStream}. Compression is performed
+ * using the content encoding provided during entity construction.
+ *
+ * @param outStream the {@link OutputStream} to which the compressed content will be written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during compression or writing.
+ * @throws UnsupportedOperationException if the specified compression type is not supported.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
+ // Get the compressor based on the specified content encoding
+ final OutputStream compressorStream;
+ try {
+ compressorStream = CompressingFactory.INSTANCE.getCompressorOutputStream(contentEncoding, outStream);
+ } catch (final CompressorException e) {
+ throw new IOException("Error initializing decompression stream", e);
+ }
+ if (compressorStream != null) {
+ // Write compressed data
+ super.writeTo(compressorStream);
+ // Close the compressor stream after writing
+ compressorStream.close();
Review Comment:
Could this method be rewritten to use a try-with-resources block?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768616169
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,294 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ private static final Map<String, String> COMPRESSION_ALIASES;
+ static {
+ final Map<String, String> aliases = new HashMap<>();
Review Comment:
@arturobernalg
How about using:
- org.apache.commons.compress.compressors.CompressorStreamProvider.getInputStreamCompressorNames()
- org.apache.commons.compress.compressors.CompressorStreamProvider.getOutputStreamCompressorNames()
- Should I add an API to return `org.apache.commons.compress.compressors.CompressorStreamFactory.ALL_NAMES`?
-
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768634734
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
Review Comment:
I think the buffer size should be 8K, like in NIO Files.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2730403554
Hi @garydgregory , do I need to do anything else for PR #[580](https://github.com/apache/httpcomponents-client/pull/580)? Any timing to validate it? Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2480579933
@arturobernalg
Thank you for the review on https://github.com/apache/commons-compress/pull/605, I merged that PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768632771
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressorFactory.java:
##########
@@ -0,0 +1,294 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+
+package org.apache.hc.client5.http.entity;
+
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.stream.Collectors;
+
+import org.apache.commons.compress.compressors.CompressorStreamFactory;
+import org.apache.commons.compress.compressors.deflate.DeflateCompressorInputStream;
+import org.apache.commons.compress.compressors.deflate.DeflateParameters;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.util.Args;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A factory class for managing compression and decompression of HTTP entities using different compression formats.
+ * <p>
+ * This factory uses a cache to optimize access to available input and output stream providers for compression formats.
+ * It also allows the use of aliases (e.g., "gzip" and "x-gzip") and automatically formats the compression names
+ * to ensure consistency.
+ * </p>
+ *
+ * <p>
+ * Supported compression formats include gzip, deflate, and other available formats provided by the
+ * {@link CompressorStreamFactory}.
+ * </p>
+ *
+ * <p>
+ * This class is thread-safe and uses {@link AtomicReference} to cache the available input and output stream providers.
+ * </p>
+ */
+public class CompressorFactory {
+
+ private static final Logger LOG = LoggerFactory.getLogger(CompressorFactory.class);
+ /**
+ * Singleton instance of the factory.
+ */
+ public static final CompressorFactory INSTANCE = new CompressorFactory();
+
+ private final CompressorStreamFactory compressorStreamFactory = new CompressorStreamFactory();
+ private final AtomicReference<Set<String>> inputProvidersCache = new AtomicReference<>();
+ private final AtomicReference<Set<String>> outputProvidersCache = new AtomicReference<>();
+ private final Map<String, String> formattedNameCache = new ConcurrentHashMap<>();
+
+ private static final Map<String, String> COMPRESSION_ALIASES;
+ static {
+ final Map<String, String> aliases = new HashMap<>();
+ aliases.put("gzip", "gz");
+ aliases.put("x-gzip", "gz");
+ aliases.put("compress", "z");
+ COMPRESSION_ALIASES = Collections.unmodifiableMap(aliases);
+ }
+
+ /**
+ * Returns a set of available input stream compression providers.
+ *
+ * @return a set of available input stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableInputProviders() {
+ return getAvailableProviders(inputProvidersCache, false);
+ }
+
+ /**
+ * Returns a set of available output stream compression providers.
+ *
+ * @return a set of available output stream compression providers in lowercase.
+ */
+ public Set<String> getAvailableOutputProviders() {
+ return getAvailableProviders(outputProvidersCache, true);
+ }
+
+ /**
+ * Returns the formatted name of the provided compression format.
+ * <p>
+ * If the provided name matches an alias (e.g., "gzip" or "x-gzip"), the method will return the standard name.
+ * </p>
+ *
+ * @param name the compression format name.
+ * @return the formatted name, or the original name if no alias is found.
+ * @throws IllegalArgumentException if the name is null or empty.
+ */
+ public String getFormattedName(final String name) {
+ if (name == null || name.isEmpty()) {
+ LOG.warn("Compression name is null or empty");
+ return null;
+ }
+ final String lowerCaseName = name.toLowerCase(Locale.ROOT);
+ return formattedNameCache.computeIfAbsent(lowerCaseName, key -> COMPRESSION_ALIASES.getOrDefault(key, key));
+ }
+
+
+ /**
+ * Creates an input stream for the specified compression format and decompresses the provided input stream.
+ * <p>
+ * This method uses the specified compression name to decompress the input stream and supports the "nowrap" option
+ * for deflate streams.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param nowrap if true, disables the zlib header and trailer for deflate streams.
+ * @return the decompressed input stream, or the original input stream if the format is not supported.
+ */
+ public InputStream getCompressorInputStream(final String name, final InputStream inputStream, final boolean nowrap) {
+ Args.notNull(inputStream, "InputStream");
+ Args.notNull(name, "name");
+
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, false)
+ ? createCompressorInputStream(formattedName, inputStream, nowrap)
+ : inputStream;
+ }
+
+ /**
+ * Creates an output stream for the specified compression format and compresses the provided output stream.
+ *
+ * @param name the compression format.
+ * @param outputStream the output stream to compress.
+ * @return the compressed output stream, or the original output stream if the format is not supported.
+ */
+ public OutputStream getCompressorOutputStream(final String name, final OutputStream outputStream) {
+ final String formattedName = getFormattedName(name);
+ return isSupported(formattedName, true)
+ ? createCompressorOutputStream(formattedName, outputStream)
+ : outputStream;
+ }
+
+ /**
+ * Compresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to compress.
+ * @param contentEncoding the compression format.
+ * @return a compressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity compressEntity(final HttpEntity entity, final String contentEncoding) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, true)) {
+ LOG.warn("Unsupported compression type: {}", contentEncoding);
+ return null;
+ }
+ return new CompressingEntity(entity, contentEncoding);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding) {
+ return decompressEntity(entity, contentEncoding, false);
+ }
+
+ /**
+ * Decompresses the provided HTTP entity using the specified compression format with the option for deflate streams.
+ *
+ * @param entity the HTTP entity to decompress.
+ * @param contentEncoding the compression format.
+ * @param nowrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed {@link HttpEntity}, or {@code null} if the compression format is unsupported.
+ */
+ public HttpEntity decompressEntity(final HttpEntity entity, final String contentEncoding, final boolean nowrap) {
+ Args.notNull(entity, "Entity");
+ Args.notNull(contentEncoding, "Content Encoding");
+ if (!isSupported(contentEncoding, false)) {
+ LOG.warn("Unsupported decompression type: {}", contentEncoding);
+ return null;
+ }
+ return new DecompressEntity(entity, contentEncoding, nowrap);
+ }
+
+ /**
+ * Creates a compressor input stream for the given compression format and input stream.
+ * <p>
+ * This method handles the special case for deflate compression where the zlib header can be skipped.
+ * </p>
+ *
+ * @param name the compression format.
+ * @param inputStream the input stream to decompress.
+ * @param nowrap if true, disables the zlib header and trailer for deflate streams.
+ * @return a decompressed input stream, or null if an error occurs.
+ */
+ private InputStream createCompressorInputStream(final String name, final InputStream inputStream, final boolean nowrap) {
+ try {
+ if ("deflate".equalsIgnoreCase(name)) {
+ final DeflateParameters parameters = new DeflateParameters();
+ parameters.setWithZlibHeader(nowrap);
+ return new DeflateCompressorInputStream(inputStream, parameters);
+ }
+ return compressorStreamFactory.createCompressorInputStream(name, inputStream, true);
+ } catch (final Exception ex) {
+ LOG.warn("Could not create compressor {} input stream", name, ex);
Review Comment:
Why not let the exception percolate up?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1769073337
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
+ }
+ return content;
+ }
+ return getDecompressingStream();
+ }
+
+ /**
+ * Writes the decompressed content to the specified {@link OutputStream}.
+ *
+ * @param outStream the {@link OutputStream} to which the decompressed content is written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during writing or decompression.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
Review Comment:
Hello @arturobernalg
It is not deprecated. You're confusing Commons IO IOUtils and the Commons Compress IOUtils, which delegates to Commons IO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r2072637431
##########
httpclient5/pom.xml:
##########
@@ -97,6 +97,10 @@
<artifactId>mockito-core</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-compress</artifactId>
Review Comment:
@arturobernalg This is wrong. `commos-compress` must be optional
##########
pom.xml:
##########
@@ -192,6 +193,12 @@
<artifactId>junit-jupiter</artifactId>
<version>${testcontainers.version}</version>
</dependency>
+ <dependency>
+ <groupId>org.apache.commons</groupId>
+ <artifactId>commons-compress</artifactId>
+ <version>${commpress.version}</version>
+ <optional>true</optional>
Review Comment:
@arturobernalg I do not think this has any effect. Anyway, `dependencyManagement` section is the wrong place to declare artifact's optionality. This needs to be declared in the module's `dependecies` section, not here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768637298
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
Review Comment:
Use camel-case: "noWrap".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768647926
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
+ }
+ return content;
+ }
+ return getDecompressingStream();
+ }
+
+ /**
+ * Writes the decompressed content to the specified {@link OutputStream}.
+ *
+ * @param outStream the {@link OutputStream} to which the decompressed content is written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during writing or decompression.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
Review Comment:
You can reuse Commons IO's `IOUtils.copy(InputStream, OutputStream)` which Commons Compress uses already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768582843
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/CompressingEntity.java:
##########
@@ -0,0 +1,135 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that applies compression to the content before writing it to
+ * an output stream. This class supports various compression algorithms based on the
+ * specified content encoding.
+ *
+ * <p>Compression is performed using {@link CompressorFactory}, which returns a corresponding
+ * {@link OutputStream} for the requested compression type. This class does not support
+ * reading the content directly through {@link #getContent()} as the content is always compressed
+ * during write operations.</p>
+ *
+ * @since 5.4
+ */
+public class CompressingEntity extends HttpEntityWrapper {
+
+ /**
+ * The content encoding type, e.g., "gzip", "deflate", etc.
+ */
+ private final String contentEncoding;
+
+ /**
+ * Creates a new {@link CompressingEntity} that compresses the wrapped entity's content
+ * using the specified content encoding.
+ *
+ * @param entity the {@link HttpEntity} to wrap and compress; must not be {@code null}.
+ * @param contentEncoding the content encoding to use for compression, e.g., "gzip".
+ */
+ public CompressingEntity(final HttpEntity entity, final String contentEncoding) {
+ super(entity);
+ this.contentEncoding = Args.notNull(contentEncoding, "Content encoding");
+ }
+
+ /**
+ * Returns the content encoding used for compression.
+ *
+ * @return the content encoding (e.g., "gzip", "deflate").
+ */
+ @Override
+ public String getContentEncoding() {
+ return contentEncoding;
+ }
+
+ /**
+ * Returns the length of the wrapped entity. As the content is compressed,
+ * this will return the length of the wrapped entity's compressed data.
+ *
+ * @return the length of the compressed content, or {@code -1} if unknown.
+ */
+ @Override
+ public long getContentLength() {
+ return super.getContentLength();
Review Comment:
These 2 calls to `super` are redundant, the comments can go in the class Javadoc IMO.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2364570970
HI @garydgregory
Please do another pass.
TY
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg closed pull request #580: [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress
URL: https://github.com/apache/httpcomponents-client/pull/580
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "ok2c (via GitHub)" <gi...@apache.org>.
ok2c commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2848569349
> @ok2c would you mind to validate this PR.
@arturobernalg Sure. Will do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1769040048
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
+ }
+ return content;
+ }
+ return getDecompressingStream();
+ }
+
+ /**
+ * Writes the decompressed content to the specified {@link OutputStream}.
+ *
+ * @param outStream the {@link OutputStream} to which the decompressed content is written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during writing or decompression.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
Review Comment:
HI @garydgregory
That method is deprecated
`/**
* Copies the content of a InputStream into an OutputStream. Uses a default buffer size of 8024 bytes.
*
* @param input the InputStream to copy
* @param output the target, may be null to simulate output to dev/null on Linux and NUL on Windows
* @return the number of bytes copied
* @throws IOException if an error occurs
* @deprecated Use {@link org.apache.commons.io.IOUtils#copy(InputStream, OutputStream)}.
*/
@Deprecated
public static long copy(final InputStream input, final OutputStream output) throws IOException {
return org.apache.commons.io.IOUtils.copy(input, output);
}`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1768648365
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,160 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., nowrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * @since 5.4
+ */
+public class DecompressEntity extends HttpEntityWrapper {
+
+ /**
+ * Default buffer size used during decompression.
+ */
+ private static final int BUFFER_SIZE = 1024 * 2;
+
+ /**
+ * The content input stream, initialized lazily during the first read.
+ */
+ private InputStream content;
+
+ /**
+ * The compression type used for decompression (e.g., gzip, deflate).
+ */
+ private final String compressionType;
+
+ /**
+ * The flag indicating if decompression should skip certain headers (nowrap).
+ */
+ private final boolean nowrap;
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type and nowrap setting.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ * @param nowrap whether to decompress without headers for certain compression formats.
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType, final boolean nowrap) {
+ super(wrapped);
+ this.compressionType = compressionType;
+ this.nowrap = nowrap;
+ }
+
+ /**
+ * Constructs a new {@link DecompressEntity} with the specified compression type, defaulting to no nowrap handling.
+ *
+ * @param wrapped the non-null {@link HttpEntity} to be wrapped.
+ * @param compressionType the compression type (e.g., "gzip", "deflate").
+ */
+ public DecompressEntity(final HttpEntity wrapped, final String compressionType) {
+ this(wrapped, compressionType, false);
+ }
+
+ /**
+ * Initializes and returns a stream for decompression.
+ * The decompression is applied lazily on the wrapped entity's content.
+ *
+ * @return a lazily initialized {@link InputStream} that decompresses the content.
+ * @throws IOException if an error occurs during decompression.
+ */
+ private InputStream getDecompressingStream() throws IOException {
+ return new LazyDecompressInputStream(super.getContent(), compressionType, nowrap);
+ }
+
+ /**
+ * Returns the decompressed content stream. If the entity is streaming,
+ * the same {@link InputStream} is returned on subsequent calls.
+ *
+ * @return the decompressed {@link InputStream}.
+ * @throws IOException if an error occurs during decompression.
+ */
+ @Override
+ public InputStream getContent() throws IOException {
+ if (super.isStreaming()) {
+ if (content == null) {
+ content = getDecompressingStream();
+ }
+ return content;
+ }
+ return getDecompressingStream();
+ }
+
+ /**
+ * Writes the decompressed content to the specified {@link OutputStream}.
+ *
+ * @param outStream the {@link OutputStream} to which the decompressed content is written; must not be {@code null}.
+ * @throws IOException if an I/O error occurs during writing or decompression.
+ */
+ @Override
+ public void writeTo(final OutputStream outStream) throws IOException {
+ Args.notNull(outStream, "Output stream");
+ try (InputStream inStream = getContent()) {
+ final byte[] buffer = new byte[BUFFER_SIZE];
+ int l;
+ while ((l = inStream.read(buffer)) != -1) {
+ outStream.write(buffer, 0, l);
+ }
+ }
+ }
+
+ /**
+ * Returns the compression type (e.g., "gzip", "deflate").
+ *
+ * @return the content encoding (compression type).
+ */
+ @Override
+ public String getContentEncoding() {
+ return compressionType;
+ }
+
+ /**
+ * Returns the length of the decompressed content. Since the content is being decompressed,
+ * the exact length is unknown and this method returns the length of the wrapped entity.
+ *
+ * @return the content length, or {@code -1} if unknown.
+ */
+ @Override
+ public long getContentLength() {
+ return super.getContentLength();
Review Comment:
See above.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2480317343
Hi @arturobernalg,
What do you think about https://github.com/apache/commons-compress/pull/605
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "arturobernalg (via GitHub)" <gi...@apache.org>.
arturobernalg commented on PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#issuecomment-2427517243
> Hi @arturobernalg
>
> I git-hub resolved most of the comment I wrote. I think we should consistently name ALL our new classes either with or without the "ing" word part, either `[Compress|Decompress]Entity` or `[Compressing|Decompressing]Entity`. I think "ing" is better and in keeping with our existing other `HttpEntity` implementations like `IncomingHttpEntity` and `DecompressingEntity` (conflicting). I also think that all this new compression/decompression code should live in its own package, maybe `org.apache.hc.client5.http.entity.compress`.
Please @garydgregory take another look.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org
Re: [PR] [HTTPCLIENT-1843] - Delegate compression handling to Apache Commons Compress [httpcomponents-client]
Posted by "garydgregory (via GitHub)" <gi...@apache.org>.
garydgregory commented on code in PR #580:
URL: https://github.com/apache/httpcomponents-client/pull/580#discussion_r1808718309
##########
httpclient5/src/main/java/org/apache/hc/client5/http/entity/DecompressEntity.java:
##########
@@ -0,0 +1,149 @@
+/*
+ * ====================================================================
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ * ====================================================================
+ *
+ * This software consists of voluntary contributions made by many
+ * individuals on behalf of the Apache Software Foundation. For more
+ * information on the Apache Software Foundation, please see
+ * <http://www.apache.org/>.
+ *
+ */
+package org.apache.hc.client5.http.entity;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.hc.core5.annotation.Contract;
+import org.apache.hc.core5.annotation.ThreadingBehavior;
+import org.apache.hc.core5.http.HttpEntity;
+import org.apache.hc.core5.http.io.entity.HttpEntityWrapper;
+import org.apache.hc.core5.util.Args;
+
+/**
+ * An {@link HttpEntity} wrapper that decompresses the content of the wrapped entity.
+ * This class supports different compression types and can handle both standard
+ * compression (e.g., gzip, deflate) and variations that require a custom handling (e.g., noWrap).
+ *
+ * <p>Decompression is performed using a {@link LazyDecompressInputStream} that
+ * applies decompression lazily when content is requested.</p>
+ *
+ * <p>
+ * Note: This class uses lazy initialization for certain fields, making it <b>not thread-safe</b>.
+ * If multiple threads access an instance of this class concurrently, they must synchronize on the instance
+ * to ensure correct behavior.
+ * </p>
+ *
+ * @since 5.5
+ */
+@Contract(threading = ThreadingBehavior.UNSAFE)
+public class DecompressEntity extends HttpEntityWrapper {
Review Comment:
The class name is mismatched with the "CompressingEntity" name, either they should both be "...ing" names, or not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@hc.apache.org
For additional commands, e-mail: dev-help@hc.apache.org