[jira] Updated: (IVY-1061) ChecksumHelper.check() fails on non-ASCII platforms

["Robin Fernandes (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/IVY-1061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Robin Fernandes updated IVY-1061:
---------------------------------

    Description: 
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:

{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
    String csFileContent = FileUtil.readEntirely(
        new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}

FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.

In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?

{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
    String csFileContent = FileUtil.readEntirely(
            new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}


Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.

  was:
As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:

{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
    String csFileContent = FileUtil.readEntirely(
        new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
//...
{code}

FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.

In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?

{code}
public static void check(File dest, File checksumFile, String algorithm) throws IOException {
    String csFileContent = FileUtil.readEntirely(
            new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
{code}


A workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.


> ChecksumHelper.check() fails on non-ASCII platforms
> ---------------------------------------------------
>
>                 Key: IVY-1061
>                 URL: https://issues.apache.org/jira/browse/IVY-1061
>             Project: Ivy
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 2.0, 2.1.0, trunk
>         Environment: z/OS 1.9
> java version "1.6.0"
> Java(TM) SE Runtime Environment (build pmz3160sr3-20081108_01(SR3))
>            Reporter: Robin Fernandes
>
> As part of the checksum verification algorithm, ChecksumHelper converts the checksum bytes to a String using the default encoding:
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
>     String csFileContent = FileUtil.readEntirely(
>         new BufferedReader(new FileReader(checksumFile))).trim().toLowerCase(Locale.US);
> //...
> {code}
> FileReader reads the file as a sequence of bytes, which FileUtil.readEntirely() then converts to a String using the default encoding (because no other encoding is explicitly specified). On z/OS, the default encoding is EBCDIC (IBM-1047). Therefore, the checksum string ends up as garbage and the checksum comparison fails.
> In my environment, I can work around the issue by specifying ISO-8859-1 explicitly as follows. I'm not sure whether this is a generic solution: can we assume that the algorithm will always work if the checksum bytes are interpreted as ASCII? If not, how do we determine the correct encoding to use?
> {code}
> public static void check(File dest, File checksumFile, String algorithm) throws IOException {
>     String csFileContent = FileUtil.readEntirely(
>             new BufferedReader(new InputStreamReader(new FileInputStream(checksumFile), "ISO-8859-1"))).trim().toLowerCase(Locale.US);
> {code}
> Another workaround could be to specify the system property -Dfile.encoding=ISO-8559-1 on the command line, but this is a bit of a big hammer. In particular, it is not suitable when Ivy is used within an application where we don't to assume all input is ISO-8559-1. This is related to issue IVY-1060.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.