You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@commons.apache.org by sebb <se...@gmail.com> on 2013/05/13 19:14:33 UTC
Re: svn commit: r1481509 - in /commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz:
BoundedRandomAccessFileInputStream.java Coders.java SevenZFile.java
On 12 May 2013 11:12, <da...@apache.org> wrote:
> Author: damjan
> Date: Sun May 12 10:12:16 2013
> New Revision: 1481509
>
> URL: http://svn.apache.org/r1481509
> Log:
> Add support for BZIP2 decompression and AES-256 + SHA-256 decryption
> to the 7z archive format.
>
-0
Please ensure code will compile with the target Java version - see
invalid IOException ctors below.
I have fixed the bugs; please fix your local build - thanks.
>
> Added:
> commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java (with props)
> commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java (with props)
> Modified:
> commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java
>
> Added: commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java
> URL: http://svn.apache.org/viewvc/commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java?rev=1481509&view=auto
> ==============================================================================
> --- commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java (added)
> +++ commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java Sun May 12 10:12:16 2013
> @@ -0,0 +1,63 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements. See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License. You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + *
> + */
> +package org.apache.commons.compress.archivers.sevenz;
> +
> +import java.io.IOException;
> +import java.io.InputStream;
> +import java.io.RandomAccessFile;
> +
> +class BoundedRandomAccessFileInputStream extends InputStream {
> + private final RandomAccessFile file;
> + private long bytesRemaining;
> +
> + public BoundedRandomAccessFileInputStream(final RandomAccessFile file,
> + final long size) {
> + this.file = file;
> + this.bytesRemaining = size;
> + }
> +
> + @Override
> + public int read() throws IOException {
> + if (bytesRemaining > 0) {
> + --bytesRemaining;
> + return file.read();
> + } else {
> + return -1;
> + }
> + }
> +
> + @Override
> + public int read(byte[] b, int off, int len) throws IOException {
> + if (bytesRemaining == 0) {
> + return -1;
> + }
> + int bytesToRead = len;
> + if (bytesToRead > bytesRemaining) {
> + bytesToRead = (int) bytesRemaining;
> + }
> + final int bytesRead = file.read(b, off, bytesToRead);
> + if (bytesRead >= 0) {
> + bytesRemaining -= bytesRead;
> + }
> + return bytesRead;
> + }
> +
> + @Override
> + public void close() {
> + }
> +}
> \ No newline at end of file
>
> Propchange: commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/BoundedRandomAccessFileInputStream.java
> ------------------------------------------------------------------------------
> svn:eol-style = native
>
> Added: commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java
> URL: http://svn.apache.org/viewvc/commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java?rev=1481509&view=auto
> ==============================================================================
> --- commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java (added)
> +++ commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java Sun May 12 10:12:16 2013
> @@ -0,0 +1,182 @@
> +/*
> + * Licensed to the Apache Software Foundation (ASF) under one or more
> + * contributor license agreements. See the NOTICE file distributed with
> + * this work for additional information regarding copyright ownership.
> + * The ASF licenses this file to You under the Apache License, Version 2.0
> + * (the "License"); you may not use this file except in compliance with
> + * the License. You may obtain a copy of the License at
> + *
> + * http://www.apache.org/licenses/LICENSE-2.0
> + *
> + * Unless required by applicable law or agreed to in writing, software
> + * distributed under the License is distributed on an "AS IS" BASIS,
> + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> + * See the License for the specific language governing permissions and
> + * limitations under the License.
> + *
> + */
> +package org.apache.commons.compress.archivers.sevenz;
> +
> +import java.io.IOException;
> +import java.io.InputStream;
> +import java.security.GeneralSecurityException;
> +import java.security.MessageDigest;
> +import java.security.NoSuchAlgorithmException;
> +import java.util.Arrays;
> +
> +import javax.crypto.Cipher;
> +import javax.crypto.CipherInputStream;
> +import javax.crypto.SecretKey;
> +import javax.crypto.spec.IvParameterSpec;
> +import javax.crypto.spec.SecretKeySpec;
> +
> +import org.apache.commons.compress.compressors.bzip2.BZip2CompressorInputStream;
> +import org.tukaani.xz.LZMA2InputStream;
> +
> +class Coders {
> + static InputStream addDecoder(final InputStream is,
> + final Coder coder, final String password) throws IOException {
> + for (final CoderId coderId : coderTable) {
> + if (Arrays.equals(coderId.id, coder.decompressionMethodId)) {
> + return coderId.coder.decode(is, coder, password);
> + }
> + }
> + throw new IOException("Unsupported compression method " +
> + Arrays.toString(coder.decompressionMethodId));
> + }
> +
> + static CoderId[] coderTable = new CoderId[] {
> + new CoderId(new byte[] { (byte)0x00 }, new CopyDecoder()),
> + new CoderId(new byte[] { (byte)0x21 }, new LZMA2Decoder()),
> + // FIXME: gives corrupt output
> + //new CoderId(new byte[] { (byte)0x04, (byte)0x01, (byte)0x08 }, new DeflateDecoder()),
> + new CoderId(new byte[] { (byte)0x04, (byte)0x02, (byte)0x02 }, new BZIP2Decoder()),
> + new CoderId(new byte[] { (byte)0x06, (byte)0xf1, (byte)0x07, (byte)0x01 }, new AES256SHA256Decoder())
> + };
> +
> + static class CoderId {
> + CoderId(final byte[] id, final CoderBase coder) {
> + this.id = id;
> + this.coder = coder;
> + }
> +
> + final byte[] id;
> + final CoderBase coder;
> + }
> +
> + static abstract class CoderBase {
> + abstract InputStream decode(final InputStream in, final Coder coder,
> + String password) throws IOException;
> + }
> +
> + static class CopyDecoder extends CoderBase {
> + @Override
> + InputStream decode(final InputStream in, final Coder coder,
> + String password) throws IOException {
> + return in;
> + }
> + }
> +
> + static class LZMA2Decoder extends CoderBase {
> + @Override
> + InputStream decode(final InputStream in, final Coder coder,
> + String password) throws IOException {
> + final int dictionarySizeBits = 0xff & coder.properties[0];
> + if ((dictionarySizeBits & (~0x3f)) != 0) {
> + throw new IOException("Unsupported LZMA2 property bits");
> + }
> + if (dictionarySizeBits > 40) {
> + throw new IOException("Dictionary larger than 4GiB maximum size");
> + }
> + final int dictionarySize;
> + if (dictionarySizeBits == 40) {
> + dictionarySize = 0xFFFFffff;
> + } else {
> + dictionarySize = (2 | (dictionarySizeBits & 0x1)) << (dictionarySizeBits / 2 + 11);
> + }
> + return new LZMA2InputStream(in, dictionarySize);
> + }
> + }
> +
> +// static class DeflateDecoder extends CoderBase {
> +// @Override
> +// InputStream decode(final InputStream in, final Coder coder, final String password)
> +// throws IOException {
> +// System.out.println("deflate prop count = " + (coder.properties == null ? -1 : coder.properties.length));
> +// return new DeflaterInputStream(in, new Deflater(Deflater.DEFAULT_COMPRESSION, true));
> +// //return new GZIPInputStream(in);
> +// }
> +// }
> +
> + static class BZIP2Decoder extends CoderBase {
> + @Override
> + InputStream decode(final InputStream in, final Coder coder, final String password)
> + throws IOException {
> + return new BZip2CompressorInputStream(in);
> + }
> + }
> +
> + static class AES256SHA256Decoder extends CoderBase {
> + @Override
> + InputStream decode(final InputStream in, final Coder coder,
> + String password) throws IOException {
> + final int byte0 = 0xff & coder.properties[0];
> + final int numCyclesPower = byte0 & 0x3f;
> + final int byte1 = 0xff & coder.properties[1];
> + final int ivSize = ((byte0 >> 6) & 1) + (byte1 & 0x0f);
> + final int saltSize = ((byte0 >> 7) & 1) + (byte1 >> 4);
> + //debug("numCyclesPower=" + numCyclesPower + ", saltSize=" + saltSize + ", ivSize=" + ivSize);
> + if (2 + saltSize + ivSize > coder.properties.length) {
> + throw new IOException("Salt size + IV size too long");
> + }
> + final byte[] salt = new byte[saltSize];
> + System.arraycopy(coder.properties, 2, salt, 0, saltSize);
> + final byte[] iv = new byte[16];
> + System.arraycopy(coder.properties, 2 + saltSize, iv, 0, ivSize);
> +
> + if (password == null) {
> + throw new IOException("Cannot read encrypted files without a password");
> + }
> + final byte[] passwordBytes = password.getBytes("UTF-16LE");
> + final byte[] aesKeyBytes;
> + if (numCyclesPower == 0x3f) {
> + aesKeyBytes = new byte[32];
> + System.arraycopy(salt, 0, aesKeyBytes, 0, saltSize);
> + System.arraycopy(passwordBytes, 0, aesKeyBytes, saltSize,
> + Math.min(passwordBytes.length, aesKeyBytes.length - saltSize));
> + } else {
> + final MessageDigest digest;
> + try {
> + digest = MessageDigest.getInstance("SHA-256");
> + } catch (NoSuchAlgorithmException noSuchAlgorithmException) {
> + throw new IOException("SHA-256 is unsupported by your Java implementation",
> + noSuchAlgorithmException);
Requires Java 1.6+
> + }
> + final byte[] extra = new byte[8];
> + for (long j = 0; j < (1L << numCyclesPower); j++) {
> + digest.update(salt);
> + digest.update(passwordBytes);
> + digest.update(extra);
> + for (int k = 0; k < extra.length; k++) {
> + ++extra[k];
> + if (extra[k] != 0) {
> + break;
> + }
> + }
> + }
> + aesKeyBytes = digest.digest();
> + }
> +
> + final SecretKey aesKey = new SecretKeySpec(aesKeyBytes, "AES");
> + try {
> + Cipher cipher = Cipher.getInstance("AES/CBC/NoPadding");
> + cipher.init(Cipher.DECRYPT_MODE, aesKey, new IvParameterSpec(iv));
> + return new CipherInputStream(in, cipher);
> + } catch (GeneralSecurityException generalSecurityException) {
> + throw new IOException("Decryption error " +
> + "(do you have the JCE Unlimited Strength Jurisdiction Policy Files installed?)",
> + generalSecurityException);
Requires Java 1.6+
> + }
> + }
> + }
> +}
>
> Propchange: commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/Coders.java
> ------------------------------------------------------------------------------
> svn:eol-style = native
>
> Modified: commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java
> URL: http://svn.apache.org/viewvc/commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java?rev=1481509&r1=1481508&r2=1481509&view=diff
> ==============================================================================
> --- commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java (original)
> +++ commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz/SevenZFile.java Sun May 12 10:12:16 2013
> @@ -29,22 +29,23 @@ import java.util.BitSet;
> import java.util.zip.CRC32;
>
> import org.apache.commons.compress.utils.CRC32VerifyingInputStream;
> -import org.tukaani.xz.LZMA2InputStream;
>
> /**
> * Reads a 7z file, using RandomAccessFile under
> * the covers.
> * <p>
> * The 7z file format is a flexible container
> - * that can contain many compression types, but
> - * at the moment only Copy and LZMA2 are
> - * supported, and archive header compression
> + * that can contain many compression and
> + * encryption types, but at the moment only
> + * only Copy, LZMA2, BZIP2, and AES-256 + SHA-256
> + * are supported, and archive header compression
> * (which always uses the unsupported LZMA
> * compression) isn't. So the only archives
> * that can be read are the following:
> * <pre>
> - * 7z -mhc=off -mx=0 archive.7z files
> - * 7z -mhc=off -m0=LZMA2 archive.7z files
> + * 7z -mhc=off -mx=0 [-ppassword] archive.7z files
> + * 7z -mhc=off -m0=LZMA2 [-ppassword] archive.7z files
> + * 7z -mhc=off -m0=BZIP2 [-ppassword] archive.7z files
> * </pre>
> * <p>
> * The format is very Windows/Intel specific,
> @@ -74,6 +75,7 @@ public class SevenZFile {
> private int currentFolderIndex = -1;
> private InputStream currentFolderInputStream = null;
> private InputStream currentEntryInputStream = null;
> + private String password;
>
> private static final byte[] sevenZSignature = {
> (byte)'7', (byte)'z', (byte)0xBC, (byte)0xAF, (byte)0x27, (byte)0x1C
> @@ -92,6 +94,11 @@ public class SevenZFile {
> }
> }
>
> + public SevenZFile(final File filename, final String password) throws IOException {
> + this(filename);
> + this.password = password;
> + }
> +
> public void close() {
> if (file != null) {
> try {
> @@ -178,7 +185,7 @@ public class SevenZFile {
> DataInputStream dataInputStream = null;
> try {
> dataInputStream = new DataInputStream(new CRC32VerifyingInputStream(
> - new BoundedRandomAccessFileInputStream(20), 20, startHeaderCrc));
> + new BoundedRandomAccessFileInputStream(file, 20), 20, startHeaderCrc));
> startHeader.nextHeaderOffset = Long.reverseBytes(dataInputStream.readLong());
> startHeader.nextHeaderSize = Long.reverseBytes(dataInputStream.readLong());
> startHeader.nextHeaderCrc = Integer.reverseBytes(dataInputStream.readInt());
> @@ -836,65 +843,14 @@ public class SevenZFile {
>
> private InputStream buildDecoderStack(final Folder folder, final long folderOffset,
> final int firstPackStreamIndex) throws IOException {
> - InputStream inputStreamStack = null;
> - for (int i = 0; i < folder.coders.length; i++) {
> - if (i > 0) {
> - throw new IOException("Unsupported multi-codec stream");
> - }
> - file.seek(folderOffset);
> - if (folder.coders[i].decompressionMethodId.length == 1 &&
> - folder.coders[i].decompressionMethodId[0] == 0) {
> - // 00 - Copy
> - inputStreamStack = new BoundedRandomAccessFileInputStream(
> - archive.packSizes[firstPackStreamIndex]);
> - // FIXME: LZMA is the default coder yet ironically we don't have it.
> -// } else if (folder.coders[i].decompressionMethodId.length == 3 &&
> -// folder.coders[i].decompressionMethodId[0] == 3 &&
> -// folder.coders[i].decompressionMethodId[1] == 1 &&
> -// folder.coders[i].decompressionMethodId[2] == 1) {
> -// // 03.. - 7z
> -// // 01 - LZMA
> -// // 01 - Version
> - } else if (folder.coders[i].decompressionMethodId.length == 1 &&
> - folder.coders[i].decompressionMethodId[0] == 0x21) {
> - // 21 - LZMA2
> - final int dictionarySizeBits = 0xff & folder.coders[i].properties[0];
> - if ((dictionarySizeBits & (~0x3f)) != 0) {
> - throw new IOException("Unsupported LZMA2 property bits");
> - }
> - if (dictionarySizeBits > 40) {
> - throw new IOException("Dictionary larger than 4GiB maximum size");
> - }
> - final int dictionarySize;
> - if (dictionarySizeBits == 40) {
> - dictionarySize = 0xFFFFffff;
> - } else {
> - dictionarySize = (2 | (dictionarySizeBits & 0x1)) << (dictionarySizeBits / 2 + 11);
> - }
> - inputStreamStack = new LZMA2InputStream(
> - new BoundedRandomAccessFileInputStream(
> - archive.packSizes[firstPackStreamIndex]),
> - dictionarySize);
> - // FIXME: gives corrupt output:
> -// } else if (folder.coders[i].decompressionMethodId.length == 3 &&
> -// folder.coders[i].decompressionMethodId[0] == 0x4 &&
> -// folder.coders[i].decompressionMethodId[1] == 0x1 &&
> -// folder.coders[i].decompressionMethodId[2] == 0x8) {
> -// // 04.. - Misc
> -// // 00 - Reserved
> -// // 01 - Zip
> -// // 00 - Copy (not used). Use {00} instead
> -// // 01 - Shrink
> -// // 06 - Implode
> -// // 08 - Deflate
> -// return new DeflaterInputStream(
> -// new BoundedRandomAccessFileInputStream(
> -// archive.packSizes[firstPackStreamIndex]),
> -// new Deflater(Deflater.DEFAULT_COMPRESSION, true));
> - } else {
> - throw new IOException("Unsupported compression method " +
> - Arrays.toString(folder.coders[i].decompressionMethodId));
> + file.seek(folderOffset);
> + InputStream inputStreamStack = new BoundedRandomAccessFileInputStream(file,
> + archive.packSizes[firstPackStreamIndex]);
> + for (final Coder coder : folder.coders) {
> + if (coder.numInStreams != 1 || coder.numOutStreams != 1) {
> + throw new IOException("Multi input/output stream coders are not yet supported");
> }
> + inputStreamStack = Coders.addDecoder(inputStreamStack, coder, password);
> }
> if (folder.hasCrc) {
> return new CRC32VerifyingInputStream(inputStreamStack,
> @@ -931,44 +887,6 @@ public class SevenZFile {
> return value;
> }
>
> - private class BoundedRandomAccessFileInputStream extends InputStream {
> - private long bytesRemaining;
> -
> - public BoundedRandomAccessFileInputStream(final long size) {
> - bytesRemaining = size;
> - }
> -
> - @Override
> - public int read() throws IOException {
> - if (bytesRemaining > 0) {
> - --bytesRemaining;
> - return file.read();
> - } else {
> - return -1;
> - }
> - }
> -
> - @Override
> - public int read(byte[] b, int off, int len) throws IOException {
> - if (bytesRemaining == 0) {
> - return -1;
> - }
> - int bytesToRead = len;
> - if (bytesToRead > bytesRemaining) {
> - bytesToRead = (int) bytesRemaining;
> - }
> - final int bytesRead = file.read(b, off, bytesToRead);
> - if (bytesRead >= 0) {
> - bytesRemaining -= bytesRead;
> - }
> - return bytesRead;
> - }
> -
> - @Override
> - public void close() {
> - }
> - }
> -
> private static class BoundedInputStream extends InputStream {
> private InputStream is;
> private long bytesRemaining;
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: svn commit: r1481509 - in /commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz:
BoundedRandomAccessFileInputStream.java Coders.java SevenZFile.java
Posted by sebb <se...@gmail.com>.
On 14 May 2013 06:03, Damjan Jovanovic <da...@apache.org> wrote:
> On Mon, May 13, 2013 at 7:14 PM, sebb <se...@gmail.com> wrote:
>> On 12 May 2013 11:12, <da...@apache.org> wrote:
>>> Author: damjan
>>> Date: Sun May 12 10:12:16 2013
>>> New Revision: 1481509
>>>
>>> URL: http://svn.apache.org/r1481509
>>> Log:
>>> Add support for BZIP2 decompression and AES-256 + SHA-256 decryption
>>> to the 7z archive format.
>>>
>>
>> -0
>>
>> Please ensure code will compile with the target Java version - see
>> invalid IOException ctors below.
>>
>> I have fixed the bugs; please fix your local build - thanks.
>>
>
> Sorry about that. I've just committed a patch that checks the API
> version using the animal-sniffer plugin.
I saw that, however it is not suitable as it stands.
I meant it's best if your local build / IDE is set to use Java 1.5.
> Damjan
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
> For additional commands, e-mail: dev-help@commons.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org
Re: svn commit: r1481509 - in /commons/proper/compress/trunk/src/main/java/org/apache/commons/compress/archivers/sevenz:
BoundedRandomAccessFileInputStream.java Coders.java SevenZFile.java
Posted by Damjan Jovanovic <da...@apache.org>.
On Mon, May 13, 2013 at 7:14 PM, sebb <se...@gmail.com> wrote:
> On 12 May 2013 11:12, <da...@apache.org> wrote:
>> Author: damjan
>> Date: Sun May 12 10:12:16 2013
>> New Revision: 1481509
>>
>> URL: http://svn.apache.org/r1481509
>> Log:
>> Add support for BZIP2 decompression and AES-256 + SHA-256 decryption
>> to the 7z archive format.
>>
>
> -0
>
> Please ensure code will compile with the target Java version - see
> invalid IOException ctors below.
>
> I have fixed the bugs; please fix your local build - thanks.
>
Sorry about that. I've just committed a patch that checks the API
version using the animal-sniffer plugin.
Damjan
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@commons.apache.org
For additional commands, e-mail: dev-help@commons.apache.org