You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@maven.apache.org by "Michael Osipov (Jira)" <ji...@apache.org> on 2021/11/18 19:59:00 UTC
[jira] [Commented] (MTOOLCHAINS-39) Can't unzip/unjar maven-toolchain-1.0.jar that is available at Maven Central
[ https://issues.apache.org/jira/browse/MTOOLCHAINS-39?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17446123#comment-17446123 ]
Michael Osipov commented on MTOOLCHAINS-39:
-------------------------------------------
Well, I can confirm this and thanks to you I learned a bit more about ZIP today. Lets dive into the analysis:
My setup:
{noformat}
osipovmi@deblndw011x:/tmp/maven-toolchains
$ uname -a
FreeBSD deblndw011x.ad001.siemens.net 12.2-STABLE FreeBSD 12.2-STABLE #0: Thu Sep 23 18:58:50 CEST 2021
osipovmi@deblndw011x:/tmp/maven-toolchains
$ which unzip
/usr/bin/unzip
osipovmi@deblndw011x:/tmp/maven-toolchains
$ pkg which /usr/local/bin/zipdetails
/usr/local/bin/zipdetails was installed by package perl5-5.30.3_1
osipovmi@deblndw011x:/tmp/maven-toolchains
$ java -version
openjdk version "1.8.0_302"
OpenJDK Runtime Environment (build 1.8.0_302-b08)
OpenJDK 64-Bit Server VM (build 25.302-b08, mixed mode)
{noformat}
Lets unzip first:
{noformat}
$ unzip maven-toolchain-1.0.jar
Archive: maven-toolchain-1.0.jar
extracting: META-INF
extracting: META-INF/MANIFEST.MF
extracting: org
extracting: org/apache
extracting: org/apache/maven
extracting: org/apache/maven/toolchain
extracting: org/apache/maven/toolchain/java
extracting: org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class
extracting: org/apache/maven/toolchain/java/JavaToolChain.class
extracting: org/apache/maven/toolchain/java/DefaultJavaToolChain.class
extracting: org/apache/maven/toolchain/ToolchainManagerPrivate.class
extracting: org/apache/maven/toolchain/DefaultToolchain.class
extracting: org/apache/maven/toolchain/ToolchainPrivate.class
extracting: org/apache/maven/toolchain/MisconfiguredToolchainException.class
extracting: org/apache/maven/toolchain/ToolchainManager.class
extracting: org/apache/maven/toolchain/ToolchainFactory$1.class
extracting: org/apache/maven/toolchain/model
extracting: org/apache/maven/toolchain/model/ToolchainModel.class
extracting: org/apache/maven/toolchain/model/PersistedToolchains.class
extracting: org/apache/maven/toolchain/model/io
extracting: org/apache/maven/toolchain/model/io/xpp3
extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class
extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class
extracting: org/apache/maven/toolchain/Toolchain.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$1.class
extracting: org/apache/maven/toolchain/ToolchainManagerPrivate$1.class
extracting: org/apache/maven/toolchain/DefaultToolchainManager.class
extracting: org/apache/maven/toolchain/ToolchainFactory.class
extracting: org/apache/maven/toolchain/ToolchainManager$1.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class
extracting: org/apache/maven/toolchain/RequirementMatcher.class
extracting: META-INF/plexus
extracting: META-INF/plexus/components.xml
extracting: META-INF/maven
extracting: META-INF/maven/org.apache.maven
extracting: META-INF/maven/org.apache.maven/maven-toolchain
extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.xml
extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.properties
osipovmi@deblndw011x:/tmp/maven-toolchains
$ unzip maven-toolchain-2.0.9.jar
Archive: maven-toolchain-2.0.9.jar
creating: META-INF/
extracting: META-INF/MANIFEST.MF
extracting: META-INF/LICENSE
extracting: META-INF/NOTICE
creating: META-INF/plexus/
extracting: META-INF/plexus/components.xml
creating: org/
creating: org/apache/
creating: org/apache/maven/
creating: org/apache/maven/toolchain/
extracting: org/apache/maven/toolchain/DefaultToolchain.class
extracting: org/apache/maven/toolchain/DefaultToolchainManager.class
creating: org/apache/maven/toolchain/java/
extracting: org/apache/maven/toolchain/java/DefaultJavaToolChain.class
extracting: org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class
extracting: org/apache/maven/toolchain/java/JavaToolChain.class
extracting: org/apache/maven/toolchain/MisconfiguredToolchainException.class
creating: org/apache/maven/toolchain/model/
creating: org/apache/maven/toolchain/model/io/
creating: org/apache/maven/toolchain/model/io/xpp3/
extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class
extracting: org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class
extracting: org/apache/maven/toolchain/model/PersistedToolchains.class
extracting: org/apache/maven/toolchain/model/ToolchainModel.class
extracting: org/apache/maven/toolchain/RequirementMatcher.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$1.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class
extracting: org/apache/maven/toolchain/RequirementMatcherFactory.class
extracting: org/apache/maven/toolchain/Toolchain.class
extracting: org/apache/maven/toolchain/ToolchainFactory$1.class
extracting: org/apache/maven/toolchain/ToolchainFactory.class
extracting: org/apache/maven/toolchain/ToolchainManager$1.class
extracting: org/apache/maven/toolchain/ToolchainManager.class
extracting: org/apache/maven/toolchain/ToolchainManagerPrivate$1.class
extracting: org/apache/maven/toolchain/ToolchainManagerPrivate.class
extracting: org/apache/maven/toolchain/ToolchainPrivate.class
creating: META-INF/maven/
creating: META-INF/maven/org.apache.maven/
creating: META-INF/maven/org.apache.maven/maven-toolchain/
extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.xml
extracting: META-INF/maven/org.apache.maven/maven-toolchain/pom.properties
{noformat}
You already see that the output is different in 1.0 it is extracting dirs and in 2.0.x is creating them. {{unzip}} handles this case for you.
Where is the difference now?
Spec says:
{quote}
4.4.15 external file attributes: (4 bytes)
The mapping of the external attributes is
host-system dependent (see 'version made by'). For
MS-DOS, the low order byte is the MS-DOS directory
attribute byte. If input came from standard input, this
field is set to zero.
{noformat}
Lets check the ZIP details on {{META-INF}} for 1.0:
{noformat}
....
0000 0004 50 4B 03 04 LOCAL HEADER #1 04034B50
0004 0001 14 Extract Zip Spec 14 '2.0'
0005 0001 00 Extract OS 00 'MS-DOS'
0006 0002 08 00 General Purpose Flag 0008
[Bits 1-2] 0 'Normal Compression'
[Bit 3] 1 'Streamed'
0008 0002 08 00 Compression Method 0008 'Deflated'
000A 0004 70 9F 5C 38 Last Mod Time 385C9F70 'Thu Feb 28 19:59:32 2008'
000E 0004 00 00 00 00 CRC 00000000
0012 0004 00 00 00 00 Compressed Length 00000000
0016 0004 00 00 00 00 Uncompressed Length 00000000
001A 0002 08 00 Filename Length 0008
001C 0002 04 00 Extra Length 0004
001E 0008 4D 45 54 41 Filename 'META-INF'
2D 49 4E 46
0026 0002 FE CA Extra ID #0001 CAFE 'Java Executable'
0028 0002 00 00 Length 0000
002A 0002 03 00 PAYLOAD ..
...
7246 0004 50 4B 01 02 CENTRAL HEADER #1 02014B50
724A 0001 14 Created Zip Spec 14 '2.0'
724B 0001 00 Created OS 00 'MS-DOS'
724C 0001 14 Extract Zip Spec 14 '2.0'
724D 0001 00 Extract OS 00 'MS-DOS'
724E 0002 08 00 General Purpose Flag 0008
[Bits 1-2] 0 'Normal Compression'
[Bit 3] 1 'Streamed'
7250 0002 08 00 Compression Method 0008 'Deflated'
7252 0004 70 9F 5C 38 Last Mod Time 385C9F70 'Thu Feb 28 19:59:32 2008'
7256 0004 00 00 00 00 CRC 00000000
725A 0004 02 00 00 00 Compressed Length 00000002
725E 0004 00 00 00 00 Uncompressed Length 00000000
7262 0002 08 00 Filename Length 0008
7264 0002 04 00 Extra Length 0004
7266 0002 00 00 Comment Length 0000
7268 0002 00 00 Disk Start 0000
726A 0002 00 00 Int File Attributes 0000
[Bit 0] 0 'Binary Data'
726C 0004 00 00 00 00 Ext File Attributes 00000000
7270 0004 00 00 00 00 Local Header Offset 00000000
7274 0008 4D 45 54 41 Filename 'META-INF'
2D 49 4E 46
727C 0002 FE CA Extra ID #0001 CAFE 'Java Executable'
727E 0002 00 00 Length 0000
...
{noformat}
Lets check the ZIP details on {{META-INF}} for 2.0.x:
{noformat}
....
0000 0004 50 4B 03 04 LOCAL HEADER #1 04034B50
0004 0001 14 Extract Zip Spec 14 '2.0'
0005 0001 00 Extract OS 00 'MS-DOS'
0006 0002 08 00 General Purpose Flag 0008
[Bits 1-2] 0 'Normal Compression'
[Bit 3] 1 'Streamed'
0008 0002 08 00 Compression Method 0008 'Deflated'
000A 0004 D4 62 87 38 Last Mod Time 388762D4 'Mon Apr 7 12:22:40 2008'
000E 0004 00 00 00 00 CRC 00000000
0012 0004 00 00 00 00 Compressed Length 00000000
0016 0004 00 00 00 00 Uncompressed Length 00000000
001A 0002 09 00 Filename Length 0009
001C 0002 04 00 Extra Length 0004
001E 0009 4D 45 54 41 Filename 'META-INF/'
2D 49 4E 46
2F
0027 0002 FE CA Extra ID #0001 CAFE 'Java Executable'
0029 0002 00 00 Length 0000
002B 0002 03 00 PAYLOAD ..
...
8583 0004 50 4B 01 02 CENTRAL HEADER #1 02014B50
8587 0001 14 Created Zip Spec 14 '2.0'
8588 0001 00 Created OS 00 'MS-DOS'
8589 0001 14 Extract Zip Spec 14 '2.0'
858A 0001 00 Extract OS 00 'MS-DOS'
858B 0002 08 00 General Purpose Flag 0008
[Bits 1-2] 0 'Normal Compression'
[Bit 3] 1 'Streamed'
858D 0002 08 00 Compression Method 0008 'Deflated'
858F 0004 D4 62 87 38 Last Mod Time 388762D4 'Mon Apr 7 12:22:40 2008'
8593 0004 00 00 00 00 CRC 00000000
8597 0004 02 00 00 00 Compressed Length 00000002
859B 0004 00 00 00 00 Uncompressed Length 00000000
859F 0002 09 00 Filename Length 0009
85A1 0002 04 00 Extra Length 0004
85A3 0002 00 00 Comment Length 0000
85A5 0002 00 00 Disk Start 0000
85A7 0002 00 00 Int File Attributes 0000
[Bit 0] 0 'Binary Data'
85A9 0004 00 00 00 00 Ext File Attributes 00000000
85AD 0004 00 00 00 00 Local Header Offset 00000000
85B1 0009 4D 45 54 41 Filename 'META-INF/'
2D 49 4E 46
2F
85BA 0002 FE CA Extra ID #0001 CAFE 'Java Executable'
85BC 0002 00 00 Length 0000
...
{noformat}
Well, although this is DOS vendor, the dir byte is not set, but the filename contains a trailing slash to denote a directory. Now why does Java fail?
{code:java}
public boolean isDirectory() {
return name.endsWith("/");
}
{code}
Why does my FreeBSD {{unzip}} does work?
Magic from {{libarchive}}: https://github.com/libarchive/libarchive/blob/master/libarchive/archive_read_support_format_zip.c
{code:c}
} else if (zip_entry->system == 0) {
// Interpret MSDOS directory bit
if (0x10 == (external_attributes & 0x10)) {
zip_entry->mode = AE_IFDIR | 0775;
} else {
zip_entry->mode = AE_IFREG | 0664;
}
if (0x01 == (external_attributes & 0x01)) {
// Read-only bit; strip write permissions
zip_entry->mode &= 0555;
}
{code}
and
{code:c}
/* Make sure that entries with a trailing '/' are marked as directories
* even if the External File Attributes contains bogus values. If this
* is not a directory and there is no type, assume a regular file. */
if ((zip_entry->mode & AE_IFMT) != AE_IFDIR) {
int has_slash;
wp = archive_entry_pathname_w(entry);
if (wp != NULL) {
len = wcslen(wp);
has_slash = len > 0 && wp[len - 1] == L'/';
} else {
cp = archive_entry_pathname(entry);
len = (cp != NULL)?strlen(cp):0;
has_slash = len > 0 && cp[len - 1] == '/';
}
/* Correct file type as needed. */
if (has_slash) {
zip_entry->mode &= ~AE_IFMT;
zip_entry->mode |= AE_IFDIR;
zip_entry->mode |= 0111;
} else if ((zip_entry->mode & AE_IFMT) == 0) {
zip_entry->mode |= AE_IFREG;
}
}
/* Make sure directories end in '/' */
if ((zip_entry->mode & AE_IFMT) == AE_IFDIR) {
wp = archive_entry_pathname_w(entry);
if (wp != NULL) {
len = wcslen(wp);
if (len > 0 && wp[len - 1] != L'/') {
struct archive_wstring s;
archive_string_init(&s);
archive_wstrcat(&s, wp);
archive_wstrappend_wchar(&s, L'/');
archive_entry_copy_pathname_w(entry, s.s);
archive_wstring_free(&s);
}
} else {
cp = archive_entry_pathname(entry);
len = (cp != NULL)?strlen(cp):0;
if (len > 0 && cp[len - 1] != '/') {
struct archive_string s;
archive_string_init(&s);
archive_strcat(&s, cp);
archive_strappend_char(&s, '/');
archive_entry_set_pathname(entry, s.s);
archive_string_free(&s);
}
}
}
{code}
Although the payload is not empty (0x03, 0x00), it is decompressed, although it does not make sense for dirs, zlib inflates this and since no data is there is is extracted with nothing writtten. Subsequent files in that dir likely notice that parent does not exist and create it.
So, actually both files are crap, but 1.0 jus stinks more. Lets have a look at a more recent file:
{noformat}
osipovmi@deblndw011x:~/apache-maven-4.0.0-alpha-1-SNAPSHOT/lib
$ zipdetails maven-core-4.0.0-alpha-1-SNAPSHOT.jar | less
...
000E3 LOCAL HEADER #2 04034B50
000E7 Extract Zip Spec 0A '1.0'
000E8 Extract OS 00 'MS-DOS'
000E9 General Purpose Flag 0800
[Bit 11] 1 'Language Encoding'
000EB Compression Method 0000 'Stored'
000ED Last Mod Time 52854189 'Mon Apr 5 08:12:18 2021'
000F1 CRC 00000000
000F5 Compressed Length 00000000
000F9 Uncompressed Length 00000000
000FD Filename Length 0009
000FF Extra Length 0000
00101 Filename 'META-INF/'
...
967E6 CENTRAL HEADER #2 02014B50
967EA Created Zip Spec 14 '2.0'
967EB Created OS 03 'Unix'
967EC Extract Zip Spec 0A '1.0'
967ED Extract OS 00 'MS-DOS'
967EE General Purpose Flag 0800
[Bit 11] 1 'Language Encoding'
967F0 Compression Method 0000 'Stored'
967F2 Last Mod Time 52854189 'Mon Apr 5 08:12:18 2021'
967F6 CRC 00000000
967FA Compressed Length 00000000
967FE Uncompressed Length 00000000
96802 Filename Length 0009
96804 Extra Length 0000
96806 Comment Length 0000
96808 Disk Start 0000
9680A Int File Attributes 0000
[Bit 0] 0 'Binary Data'
9680C Ext File Attributes 41ED0010
[Bit 4] Directory
96810 Local Header Offset 000000E3
96814 Filename 'META-INF/'
...
{noformat}
Looks just the way is should look like, stored, trailing slash and 0x10 for directory is set int he extra file attributes.
Do you rely on this file?
> Can't unzip/unjar maven-toolchain-1.0.jar that is available at Maven Central
> ----------------------------------------------------------------------------
>
> Key: MTOOLCHAINS-39
> URL: https://issues.apache.org/jira/browse/MTOOLCHAINS-39
> Project: Maven Toolchains Plugin
> Issue Type: Bug
> Affects Versions: 1.0
> Reporter: Thomas Cunningham
> Priority: Major
>
> jar tvf maven-toolchain-1.0.jar that is available from maven central reports reasonable results
> However....
> jar xvf doesn't work
> ❯ jar xvf maven-toolchain-1.0.jar
> inflated: META-INF
> java.io.IOException: META-INF : could not create directory
> at jdk.jartool/sun.tools.jar.Main.extractFile(Main.java:1449)
> at jdk.jartool/sun.tools.jar.Main.extract(Main.java:1364)
> at jdk.jartool/sun.tools.jar.Main.run(Main.java:409)
> at jdk.jartool/sun.tools.jar.Main.main(Main.java:1681)
> unzip doesn't work
> ❯ unzip maven-toolchain-1.0.jar
> Archive: maven-toolchain-1.0.jar
> replace META-INF? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
> inflating: META-INF
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/MANIFEST.MF.
> inflating: org
> checkdir error: org exists but is not directory
> unable to process org/apache.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/java.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/java/DefaultJavaToolchainFactory.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/java/JavaToolChain.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/java/DefaultJavaToolChain.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainManagerPrivate.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/DefaultToolchain.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainPrivate.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/MisconfiguredToolchainException.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainManager.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainFactory$1.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/ToolchainModel.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/PersistedToolchains.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/io.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/io/xpp3.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Reader.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/model/io/xpp3/MavenToolchainsXpp3Writer.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/RequirementMatcherFactory$ExactMatcher.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/Toolchain.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/RequirementMatcherFactory.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/RequirementMatcherFactory$1.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainManagerPrivate$1.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/DefaultToolchainManager.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainFactory.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/ToolchainManager$1.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/RequirementMatcherFactory$VersionMatcher.class.
> checkdir error: org exists but is not directory
> unable to process org/apache/maven/toolchain/RequirementMatcher.class.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/plexus.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/plexus/components.xml.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/maven.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/maven/org.apache.maven.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/maven/org.apache.maven/maven-toolchain.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/maven/org.apache.maven/maven-toolchain/pom.xml.
> checkdir error: META-INF exists but is not directory
> unable to process META-INF/maven/org.apache.maven/maven-toolchain/pom.properties.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)