You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@ant.apache.org by "Robin Fernandes (JIRA)" <ji...@apache.org> on 2009/04/09 12:37:12 UTC
[jira] Created: (IVY-1061) ChecksumHelper.check() fails on
non-ASCII platforms
ChecksumHelper.check() fails on non-ASCII platforms
---------------------------------------------------
Key: IVY-1061
URL: https://issues.apache.org/jira/browse/IVY-1061
Project: Ivy
Issue Type: Bug
Components: Core
Affects Versions: 2.0, 2.1.0, trunk
Environment: z/OS 1.9
java version "1.6.0"
Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
Reporter: Robin Fernandes
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}
A workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (IVY-1061) ChecksumHelper.check() fails on
non-ASCII platforms
Posted by "Robin Fernandes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Fernandes updated IVY-1061:
---------------------------------
Description:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
was:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}
A workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
> ChecksumHelper.check() fails on non-ASCII platforms
> ---------------------------------------------------
>
> Key: IVY-1061
> URL: https://issues.apache.org/jira/browse/IVY-1061
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0, 2.1.0, trunk
> Environment: z/OS 1.9
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
> Reporter: Robin Fernandes
>
> As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> //...
> {code}
> FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
> In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
> {code}
> Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (IVY-1061) ChecksumHelper.check() fails on
non-ASCII platforms
Posted by "Robin Fernandes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Fernandes updated IVY-1061:
---------------------------------
Description:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum file to a String:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the platform's default encoding, because no other encoding is explicitly specified.
The checksum can be a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the platform that generated it.
On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum String ends up as garbage and the checksum comparison fails.
In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum String solves the problem:
{code}
Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
===================================================================
--- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
+++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
@@ -234,7 +234,7 @@
File csFile = File.createTempFile("ivytemp", algorithm);
try {
FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
- .getBytes()), csFile, null);
+ .getBytes("ISO-8859-1")), csFile, null);
repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
} finally {
Index: src/java/org/apache/ivy/util/ChecksumHelper.java
===================================================================
--- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
+++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
@@ -23,6 +23,7 @@
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
+import java.io.InputStreamReader;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
@@ -53,7 +54,7 @@
*/
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
- new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
+ new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
String expected;
if (csFileContent.indexOf(' ') > -1
&& (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
{code}
I'm not sure whether this is a sufficiently generic solution - for example, it could cause backwards compatibility issues. An alternative might be to attempt to guess the encoding of the checksum file when it is read, based on the byte values.
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
was:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum file to a String:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the platform's default encoding, because no other encoding is explicitly specified.
The checksum can be a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the platform that generated it.
On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum String ends up as garbage and the checksum comparison fails.
In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum String solves the problem:
{code}
Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
===================================================================
--- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
+++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
@@ -234,7 +234,7 @@
File csFile = File.createTempFile("ivytemp", algorithm);
try {
FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
- .getBytes()), csFile, null);
+ .getBytes("ISO-8859-1")), csFile, null);
repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
} finally {
Index: src/java/org/apache/ivy/util/ChecksumHelper.java
===================================================================
--- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
+++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
@@ -23,6 +23,7 @@
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
+import java.io.InputStreamReader;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
@@ -53,7 +54,7 @@
*/
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
- new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
+ new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
String expected;
if (csFileContent.indexOf(' ') > -1
&& (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
{code}
I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read, based on the byte values.
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
> ChecksumHelper.check() fails on non-ASCII platforms
> ---------------------------------------------------
>
> Key: IVY-1061
> URL: https://issues.apache.org/jira/browse/IVY-1061
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0, 2.1.0, trunk
> Environment: z/OS 1.9
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
> Reporter: Robin Fernandes
>
> As part of the checksum verification algorithm, ChecksumHelper converts the checksum file to a String:
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> //...
> {code}
> FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the platform's default encoding, because no other encoding is explicitly specified.
> The checksum can be a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the platform that generated it.
> On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum String ends up as garbage and the checksum comparison fails.
> In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum String solves the problem:
> {code}
> Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
> ===================================================================
> --- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
> +++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
> @@ -234,7 +234,7 @@
> File csFile = File.createTempFile("ivytemp", algorithm);
> try {
> FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
> - .getBytes()), csFile, null);
> + .getBytes("ISO-8859-1")), csFile, null);
> repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
> artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
> } finally {
> Index: src/java/org/apache/ivy/util/ChecksumHelper.java
> ===================================================================
> --- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
> +++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
> @@ -23,6 +23,7 @@
> import java.io.FileReader;
> import java.io.IOException;
> import java.io.InputStream;
> +import java.io.InputStreamReader;
> import java.security.MessageDigest;
> import java.security.NoSuchAlgorithmException;
> import java.util.HashMap;
> @@ -53,7 +54,7 @@
> */
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> - new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> + new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
> String expected;
> if (csFileContent.indexOf(' ') > -1
> && (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
> {code}
> I'm not sure whether this is a sufficiently generic solution - for example, it could cause backwards compatibility issues. An alternative might be to attempt to guess the encoding of the checksum file when it is read, based on the byte values.
> Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (IVY-1061) ChecksumHelper.check() fails on
non-ASCII platforms
Posted by "Robin Fernandes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Fernandes updated IVY-1061:
---------------------------------
Description:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum file to a String:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the platform's default encoding, because no other encoding is explicitly specified.
The checksum can be a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the platform that generated it.
On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum String ends up as garbage and the checksum comparison fails.
In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum String solves the problem:
{code}
Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
===================================================================
--- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
+++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
@@ -234,7 +234,7 @@
File csFile = File.createTempFile("ivytemp", algorithm);
try {
FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
- .getBytes()), csFile, null);
+ .getBytes("ISO-8859-1")), csFile, null);
repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
} finally {
Index: src/java/org/apache/ivy/util/ChecksumHelper.java
===================================================================
--- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
+++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
@@ -23,6 +23,7 @@
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
+import java.io.InputStreamReader;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
@@ -53,7 +54,7 @@
*/
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
- new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
+ new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
String expected;
if (csFileContent.indexOf(' ') > -1
&& (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
{code}
I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read, based on the byte values.
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
was:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified).
The checksum is stored may be stored in a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the system that generated the checksum file.
On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum string solves the problem:
{code}
Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
===================================================================
--- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
+++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
@@ -234,7 +234,7 @@
File csFile = File.createTempFile("ivytemp", algorithm);
try {
FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
- .getBytes()), csFile, null);
+ .getBytes("ISO-8859-1")), csFile, null);
repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
} finally {
Index: src/java/org/apache/ivy/util/ChecksumHelper.java
===================================================================
--- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
+++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
@@ -23,6 +23,7 @@
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
+import java.io.InputStreamReader;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
@@ -53,7 +54,7 @@
*/
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
- new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
+ new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
String expected;
if (csFileContent.indexOf(' ') > -1
&& (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
{code}
I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read.
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
> ChecksumHelper.check() fails on non-ASCII platforms
> ---------------------------------------------------
>
> Key: IVY-1061
> URL: https://issues.apache.org/jira/browse/IVY-1061
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0, 2.1.0, trunk
> Environment: z/OS 1.9
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
> Reporter: Robin Fernandes
>
> As part of the checksum verification algorithm, ChecksumHelper converts the checksum file to a String:
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> //...
> {code}
> FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the platform's default encoding, because no other encoding is explicitly specified.
> The checksum can be a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the platform that generated it.
> On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum String ends up as garbage and the checksum comparison fails.
> In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum String solves the problem:
> {code}
> Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
> ===================================================================
> --- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
> +++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
> @@ -234,7 +234,7 @@
> File csFile = File.createTempFile("ivytemp", algorithm);
> try {
> FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
> - .getBytes()), csFile, null);
> + .getBytes("ISO-8859-1")), csFile, null);
> repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
> artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
> } finally {
> Index: src/java/org/apache/ivy/util/ChecksumHelper.java
> ===================================================================
> --- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
> +++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
> @@ -23,6 +23,7 @@
> import java.io.FileReader;
> import java.io.IOException;
> import java.io.InputStream;
> +import java.io.InputStreamReader;
> import java.security.MessageDigest;
> import java.security.NoSuchAlgorithmException;
> import java.util.HashMap;
> @@ -53,7 +54,7 @@
> */
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> - new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> + new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
> String expected;
> if (csFileContent.indexOf(' ') > -1
> && (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
> {code}
> I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read, based on the byte values.
> Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
[jira] Updated: (IVY-1061) ChecksumHelper.check() fails on
non-ASCII platforms
Posted by "Robin Fernandes (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/IVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Robin Fernandes updated IVY-1061:
---------------------------------
Description:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified).
The checksum is stored may be stored in a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the system that generated the checksum file.
On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum string solves the problem:
{code}
Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
===================================================================
--- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
+++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
@@ -234,7 +234,7 @@
File csFile = File.createTempFile("ivytemp", algorithm);
try {
FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
- .getBytes()), csFile, null);
+ .getBytes("ISO-8859-1")), csFile, null);
repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
} finally {
Index: src/java/org/apache/ivy/util/ChecksumHelper.java
===================================================================
--- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
+++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
@@ -23,6 +23,7 @@
import java.io.FileReader;
import java.io.IOException;
import java.io.InputStream;
+import java.io.InputStreamReader;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.HashMap;
@@ -53,7 +54,7 @@
*/
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
- new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
+ new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
String expected;
if (csFileContent.indexOf(' ') > -1
&& (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
{code}
I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read.
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
was:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}
FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
String csFileContent = FileUtil.readEntirely(
new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}
Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
> ChecksumHelper.check() fails on non-ASCII platforms
> ---------------------------------------------------
>
> Key: IVY-1061
> URL: https://issues.apache.org/jira/browse/IVY-1061
> Project: Ivy
> Issue Type: Bug
> Components: Core
> Affects Versions: 2.0, 2.1.0, trunk
> Environment: z/OS 1.9
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
> Reporter: Robin Fernandes
>
> As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> //...
> {code}
> FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified).
> The checksum is stored may be stored in a local file or retrieved over the network. Currently, the encoding of the checksum file depends on the default encoding of the system that generated the checksum file.
> On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, if the checksum file was created on an ASCII system, the checksum string ends up as garbage and the checksum comparison fails.
> In my environment, specifying ISO-8859-1 explicitly both when encoding and decoding the checksum string solves the problem:
> {code}
> Index: src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java
> ===================================================================
> --- src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (revision 763312)
> +++ src/java/org/apache/ivy/plugins/resolver/RepositoryResolver.java (working copy)
> @@ -234,7 +234,7 @@
> File csFile = File.createTempFile("ivytemp", algorithm);
> try {
> FileUtil.copy(new ByteArrayInputStream(ChecksumHelper.computeAsString(src, algorithm)
> - .getBytes()), csFile, null);
> + .getBytes("ISO-8859-1")), csFile, null);
> repository.put(DefaultArtifact.cloneWithAnotherTypeAndExt(artifact, algorithm,
> artifact.getExt() + "." + algorithm), csFile, dest + "." + algorithm, overwrite);
> } finally {
> Index: src/java/org/apache/ivy/util/ChecksumHelper.java
> ===================================================================
> --- src/java/org/apache/ivy/util/ChecksumHelper.java (revision 763312)
> +++ src/java/org/apache/ivy/util/ChecksumHelper.java (working copy)
> @@ -23,6 +23,7 @@
> import java.io.FileReader;
> import java.io.IOException;
> import java.io.InputStream;
> +import java.io.InputStreamReader;
> import java.security.MessageDigest;
> import java.security.NoSuchAlgorithmException;
> import java.util.HashMap;
> @@ -53,7 +54,7 @@
> */
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
> String csFileContent = FileUtil.readEntirely(
> - new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> + new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
> String expected;
> if (csFileContent.indexOf(' ') > -1
> && (csFileContent.startsWith("md") || csFileContent.startsWith("sha"))) {
> {code}
> I'm not sure whether this is a sufficiently generic solution. An alternative might be to attempt to guess the encoding of the checksum file when it is read.
> Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.