You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@tapestry.apache.org by "Andy Blower (JIRA)" <de...@tapestry.apache.org> on 2008/07/17 10:21:31 UTC
[jira] Created: (TAPESTRY-2525) Allow different encoding for
properties files
Allow different encoding for properties files
---------------------------------------------
Key: TAPESTRY-2525
URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
Project: Tapestry
Issue Type: Improvement
Components: tapestry-core
Affects Versions: 5.0.13
Reporter: Andy Blower
Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Issue Comment Edited: (TAPESTRY-2525) Properties files in a
message catalog should be read using UTF-8 encoding, rather than default
encoding
Posted by "Andy Blower (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615646#action_12615646 ]
andyb edited comment on TAPESTRY-2525 at 7/22/08 8:54 AM:
----------------------------------------------------------------
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists()) return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
was (Author: andyb):
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists()) return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Updated: (TAPESTRY-2525) Properties files in a message
catalog should be read using UTF-8 encoding, rather than default encoding
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship updated TAPESTRY-2525:
-------------------------------------------
Assignee: Howard M. Lewis Ship
Summary: Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding (was: Allow different encoding for properties files)
Tapestry 4 had a more elaborate system of meta-data used to determine the encoding when reading an individual properties file. Lets see how well it works when we just assume a UTF-8 encoding (which seems to read normal ASCII files quite well).
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Issue Comment Edited: (TAPESTRY-2525) Properties files in a
message catalog should be read using UTF-8 encoding, rather than default
encoding
Posted by "Andy Blower (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615646#action_12615646 ]
andyb edited comment on TAPESTRY-2525 at 7/22/08 8:53 AM:
----------------------------------------------------------------
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists()) return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
was (Author: andyb):
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists())
return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Reopened: (TAPESTRY-2525) Properties files in a message
catalog should be read using UTF-8 encoding, rather than default encoding
Posted by "Andy Blower (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andy Blower reopened TAPESTRY-2525:
-----------------------------------
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists())
return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Issue Comment Edited: (TAPESTRY-2525) Properties files in a
message catalog should be read using UTF-8 encoding, rather than default
encoding
Posted by "Andy Blower (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615646#action_12615646 ]
andyb edited comment on TAPESTRY-2525 at 7/22/08 8:55 AM:
----------------------------------------------------------------
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant ("UTF-8") might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists()) return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
was (Author: andyb):
Unfortunately, the fix doesn't work as implemented Howard. Basically, the readStreamAsUTF8() method does nothing, because it reads the file as UTF8 and then encodes it again as UTF8. Even if the StringBuffer is encoded as ISO-8859-1, it cannot express the characters so this approach is not going to work as far as I can see.
I've implemented a fix which reads UTF8 encoded properties files if the new Properties.load(Reader) method is available (JDK1.6 and above) and it works perfectly. Here's the fixed version of MessagesSourceImpl.readProperties() - I think that the CHARSET constant might be a candidate for a symbol so it can be changed, but since we're using UTF8 I'm not bothered.
/**
* Creates and returns a new map that contains properties read from the properties file.
*/
private Map<String, String> readProperties(Resource resource)
{
if (!resource.exists()) return emptyMap;
tracker.add(resource.toURL());
Map<String, String> result = CollectionFactory.newCaseInsensitiveMap();
Properties p = new Properties();
InputStream is = null;
try
{
is = resource.openStream();
try
{ // Use new reader loader for > JDK1.6 via reflection.
Method newLoader = Properties.class.getMethod("load", Reader.class);
Reader propReader = new BufferedReader(new InputStreamReader(is, CHARSET));
newLoader.invoke(p, propReader);
}
catch (NoSuchMethodException e)
{ // Use old stream loader for < JDK1.6 (properties files must be ISO-8859-1 encoded)
p.load(is);
}
is.close();
is = null;
}
catch (Exception ex)
{
throw new RuntimeException(ServicesMessages.failureReadingMessages(resource, ex), ex);
}
finally
{
InternalUtils.close(is);
}
for (Map.Entry e : p.entrySet())
{
String key = e.getKey().toString();
String value = p.getProperty(key);
result.put(key, value);
}
return result;
}
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Closed: (TAPESTRY-2525) Properties files in a message
catalog should be read using UTF-8 encoding, rather than default encoding
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship closed TAPESTRY-2525.
------------------------------------------
Resolution: Fixed
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org
[jira] Closed: (TAPESTRY-2525) Properties files in a message
catalog should be read using UTF-8 encoding, rather than default encoding
Posted by "Howard M. Lewis Ship (JIRA)" <de...@tapestry.apache.org>.
[ https://issues.apache.org/jira/browse/TAPESTRY-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Howard M. Lewis Ship closed TAPESTRY-2525.
------------------------------------------
Resolution: Fixed
Fix Version/s: 5.0.14
> Properties files in a message catalog should be read using UTF-8 encoding, rather than default encoding
> -------------------------------------------------------------------------------------------------------
>
> Key: TAPESTRY-2525
> URL: https://issues.apache.org/jira/browse/TAPESTRY-2525
> Project: Tapestry
> Issue Type: Improvement
> Components: tapestry-core
> Affects Versions: 5.0.13
> Reporter: Andy Blower
> Assignee: Howard M. Lewis Ship
> Fix For: 5.0.14
>
>
> Allow different encodings to be used for properties files so that native2ascii is not necessary. Possibly utilise the new constructors in the Java library that take a Reader. (Added in 1.6)
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@tapestry.apache.org
For additional commands, e-mail: dev-help@tapestry.apache.org