[jira] Issue Comment Edited: (TAPESTRY-2525) Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding

["Andy Blower (JIRA)" <de...@tapestry.apache.org>]

    [ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615646#action_12615646 ] 

andyb edited comment on TAPESTRY-2525 at 7/22/08 8:53 AM:
----------------------------------------------------------------

Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.

I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.

    /**
     * Creates and returns a new map that contains properties read from the properties file.
     */
    private Map<String, String> readProperties(Resource resource)
    {
        if (!resource.exists()) return emptyMap;

        tracker.add(resource.toURL());

        Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();

        Properties p = new Properties();
        InputStream is = null;

        try
        {
            is = resource.openStream();

			try
			{	// Use new reader loader for > JDK1.6 via reflection.
				Method newLoader = Properties.class.getMethod("load", Reader.class);
				Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
				newLoader.invoke(p, propReader);
			}
			catch (NoSuchMethodException e)
			{	// Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
				p.load(is);
			}

            is.close();

            is = null;
        }
        catch (Exception ex)
        {
            throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
        }
        finally
        {
            InternalUtils.close(is);
        }

        for (Map.Entry e : p.entrySet())
        {
            String key = e.getKey().toString();

            String value = p.getProperty(key);

            result.put(key, value);
        }

        return result;
    }


      was (Author: andyb):
    Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.

I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.

	/**
	 * Creates and returns a new map that contains properties read from the properties file.
	 */
	private Map<String, String> readProperties(Resource resource)
	{
		if (!resource.exists())
			return emptyMap;

		tracker.add(resource.toURL());

		Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();

		Properties p = new Properties();
		InputStream is = null;

		try
		{
			is = resource.openStream();

			try
			{	// Use new reader loader for > JDK1.6 via reflection.
				Method newLoader = Properties.class.getMethod("load", Reader.class);
				Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
				newLoader.invoke(p, propReader);
			}
			catch (NoSuchMethodException e)
			{	// Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
				p.load(is);
			}

			is.close();

			is = null;
		}
		catch (Exception ex)
		{
			throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
		}
		finally
		{
			InternalUtils.close(is);
		}

		for (Map.Entry e : p.entrySet())
		{
			String key = e.getKey().toString();

			String value = p.getProperty(key);

			result.put(key, value);
		}

		return result;
	}

  
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
>                 Key: TAPESTRY-2525
>                 URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
>             Project: Tapestry
>          Issue Type: Improvement
>          Components: tapestry-core
>    Affects Versions: 5.0.13
>            Reporter: Andy Blower
>            Assignee: Howard M. Lewis Ship
>             Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org