You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@avro.apache.org by "Sam Pullara (JIRA)" <ji...@apache.org> on 2011/04/13 21:56:05 UTC
[jira] [Created] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Java generated Avro classes make using Avro painful and surprising
------------------------------------------------------------------
Key: AVRO-803
URL: https://issues.apache.org/jira/browse/AVRO-803
Project: Avro
Issue Type: Improvement
Components: java
Affects Versions: 1.5.0
Environment: Any
Reporter: Sam Pullara
Fix For: 1.5.1
Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Resolution: Fixed
Status: Resolved (was: Patch Available)
I committed this.
Scott, I hope you're okay with the default. If you and/or others feel strongly that we should break compatibility for this, we might consider changing that default, but I wanted to get this committed before it became stale, as it touches a lot of files.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Assignee: Doug Cutting
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Sam Pullara (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105816#comment-13105816 ]
Sam Pullara commented on AVRO-803:
----------------------------------
This is what I did in my generated classes. I think it is a pretty good solution to the problem and gives developers the option. Will it also work as easily with Maps?
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105697#comment-13105697 ]
Doug Cutting commented on AVRO-803:
-----------------------------------
What's a concrete proposal? Should we just switch generated code back to Utf8?
Or we might:
- Use Utf8 for field values, permitting efficient reuse in loops that read data;
- Provide getter methods that return String. A Utf8 memoizes its string conversion, so repeated calls to the getter would only allocate a single String. Applications that wanted to avoid that could use the field directly.
- Provide setter methods that accept CharSequence. This would check the runtime type and convert String to Utf8.
- For lists and maps, use wrappers to adapt w/o copying the entire list.
{code}
class Foo {
public Utf8 x;
public String getX() {
return x == null ? null : x.toString();
}
public void setX(CharSequence x) {
this.x = x instanceof Utf8 ? (Utf8)x : new Utf8(x.toString());
}
public List<Utf8> values;
public List<String> getValues() {
return new AbstractList<String> {
public String get(int i) {
Utf8 value = values.get(i);
return value == null ? null : value.toString();
}
public int size() { return values.size(); }
};
public void setValues(final List<? extends CharSequence> values) {
this.values = new AbstractList<Utf8> {
public Utf8 get(int i) {
CharSequence value = values.get(i);
if (value instanceof Utf8)
return (Utf8)value;
else if (value == null)
return null;
else
return new Utf8(value.toString());
}
public int size() { return values.size(); }
};
}
{code}
Thoughts?
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Attachment: AVRO-803.patch
Here's a patch that implements the above proposal.
Applications need only add <stringType>String</stringType> to the configuration of avro-maven-plugin in their pom.xml to switch to using String in their generated code.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13108077#comment-13108077 ]
Scott Carey commented on AVRO-803:
----------------------------------
I like the idea of forcing String for map keys by default. There is less value to using Utf8 there since Utf8 is mutable. Plus, Strings in the JVM are heavily optimized for use as keys in maps.
I believe that users will sometimes want to force their value types to any of the three choices: String, Utf8, and CharSequence. We should have reasonable defaults, but make it easier for users to choose what they want for their use case. For many use cases, direct conversion to Strings is better. For others, using Utf8 to be as lazy as possible in the expensive Utf8 <-> Utf16 conversion is.
Some extra laziness in Utf8 can help in a few places as well, by making the cost of creating a Utf8 from a String cheaper.
Right now, Utf8 when called with a String parameter in the constructor is not lazy, and generates the utf8 byte[]. It could leave this null, and only lazily create the byte[] if needed, just like it lazily creates the String only if needed.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Scott Carey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123893#comment-13123893 ]
Scott Carey commented on AVRO-803:
----------------------------------
We broke back-compatibility in Avro 1.5 -- formerly we used Utf8 and made maps with Utf8 keys. I think we can break back compatibility in 1.6.0 again. And we probably should for Maps since it is not appropriate to ever have a map with a CharSequence key; hashCode and equals are not compatible between implementations.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Assignee: Doug Cutting
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122281#comment-13122281 ]
Doug Cutting commented on AVRO-803:
-----------------------------------
> Perhaps "avro.java.stringImpl":"String"?
It should really be "avro.java.string.class.name":"java.lang.String", but that seems way too verbose. Maybe I went too far in my brevity, though. How about "avro.java.string":"String"?
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Scott Carey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13125569#comment-13125569 ]
Scott Carey commented on AVRO-803:
----------------------------------
I'm fine with this. Any discussion about changing the default can wait and would likely require more user input now that we have many more users. Keeping it the way it is now, with a work-around and some documentation / examples for best practices is a lot better than where we are in 1.5.x.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Assignee: Doug Cutting
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "George Fletcher (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105358#comment-13105358 ]
George Fletcher commented on AVRO-803:
--------------------------------------
We've run into this as well and for now, I wrote my own velocity template that makes the data elements private and generates getters/setters (similar to what's in 1.6.0). In these getters/setters I convert the CharSequence to a String and then do copies for arrays and maps to convert the Utf8 type to a String. This is an ugly work around and has a big performance impact. Are there any plans to fix this in 1.6.0?
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Attachment: AVRO-803.patch
New version of patch with some cleanups. Also updates mapred, protobuf & thrift code to take advantage of these changes.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Attachment: Foo.java
A couple more observations:
- I'd rather keep the raw fields using CharSequence, so that we can avoid any conversions on write. The low-level output routines can handle either Utf8 or String, so we should pass along whichever the application prefers to create.
- Map keys and values are not currently reused, so there's little point to using Utf8 there. So we might just always use String in maps, especially as keys to avoid comparison problems.
- It is possible to wrap a map and convert keys and/or values.
I've attached code that illustrates how this might look. This assumes that map keys are now always String but that values can be any CharSequence.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (Updated) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Assignee: Doug Cutting
Status: Patch Available (was: Open)
I'll commit this soon unless there are objections.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Assignee: Doug Cutting
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Scott Carey (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122222#comment-13122222 ]
Scott Carey commented on AVRO-803:
----------------------------------
Sounds good to me.
One thing though: "java":"String"
I think we may want to use the avro reserved namespace on this property to avoid collisions. Perhaps "avro.java.stringImpl":"String" ?
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13122154#comment-13122154 ]
Doug Cutting commented on AVRO-803:
-----------------------------------
Here's a new proposal:
- add a new Decoder method, 'String readString()' implemented to avoid allocating new intermediate byte arrays for each call as is currently done when Utf8's are not reused.
- change generated specific code to optionally use String everywhere instead of CharSequence. (We could also add an option to emit Utf8 everywhere.) When String is used we add a property to the string schemas in the generated code so they become {"type":"string", "java":"String"}.
- GenericData#readString() would call the new Decoder method when "java":"String" is present in the String's schema.
This is totally back-compatible.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
> Attachments: Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (AVRO-803) Java generated Avro classes make using
Avro painful and surprising
Posted by "Doug Cutting (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Doug Cutting updated AVRO-803:
------------------------------
Fix Version/s: (was: 1.5.2)
1.6.0
I don't think this would be a compatible change, so should go into 1.6.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.6.0
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13019525#comment-13019525 ]
Scott Carey commented on AVRO-803:
----------------------------------
A co-worker highlighted this and brought this to my attention very recently. CharSequence's contract explicity states that hashCode() and equals() are not consistent across implementations.
http://download.oracle.com/javase/6/docs/api/java/lang/CharSequence.html
And thus a CharSequence can't be used for a key in a map.
In Java, we'll have to use a specific implementation of CharSequence inside any map keys, and can't generalize on CharSequence in a few places. The low level Encoder / Decoder can happily use CharSequence in many places, but generated classes and interfaces on the higher layer APIs have to be more strict.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Fix For: 1.5.1
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by Scott Carey <sc...@apache.org>.
On 10/11/11 10:53 AM, "Doug Cutting" <cu...@apache.org> wrote:
>On 10/10/2011 02:51 PM, Scott Carey wrote:
>> Does it work with <stringType>Utf8</stringType> ? (Sorry, I've been
>> unable to test many patches recently).
>
>Yes.
>
>> I'm not a big fan of what our docs will say: "never use the default of
>> CharSequence, its broken". But I can live with that.
>
>It's only as broken as the use of Object in the Map API is broken, and
>as broken as the entire collections API was broken before generics were
>added to Java. Which is to say, there are bugs that could be caught at
>compile time that will instead only be manifested at runtime. This is
>not ideal, but neither is it fatal. It requires that applications
>depend a bit more on good unit tests than just on successful compilation.
Good point, we just need to be clear that map keys as CharSequence can be
dangerous. Its not an issue at all with every other use case.
>
>Long-term it would certainly be nice to change the default to something
>that better leverages compile time type checking. But for the 1.6
>release I think it would be a mistake to require application code
>changes. We just got a bunch of projects to upgrade to 1.5. It'd be
>really nice if 1.6 could be a drop-in replacement and that we not
>require changes and releases to these projects before any can upgrade.
I agree. Perhaps later (after 1.6.0) we can add something that makes
code generation use a concrete type for map keys, but still uses
CharSequence for fields or map values.
>
>Doug
Re: [jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by Doug Cutting <cu...@apache.org>.
On 10/10/2011 02:51 PM, Scott Carey wrote:
> Does it work with <stringType>Utf8</stringType> ? (Sorry, I've been
> unable to test many patches recently).
Yes.
> I'm not a big fan of what our docs will say: "never use the default of
> CharSequence, its broken". But I can live with that.
It's only as broken as the use of Object in the Map API is broken, and
as broken as the entire collections API was broken before generics were
added to Java. Which is to say, there are bugs that could be caught at
compile time that will instead only be manifested at runtime. This is
not ideal, but neither is it fatal. It requires that applications
depend a bit more on good unit tests than just on successful compilation.
Long-term it would certainly be nice to change the default to something
that better leverages compile time type checking. But for the 1.6
release I think it would be a mistake to require application code
changes. We just got a bunch of projects to upgrade to 1.5. It'd be
really nice if 1.6 could be a drop-in replacement and that we not
require changes and releases to these projects before any can upgrade.
Doug
Re: [jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by Scott Carey <sc...@richrelevance.com>.
Does it work with <stringType>Utf8</stringType> ? (Sorry, I've been
unable to test many patches recently).
I'm not a big fan of what our docs will say: "never use the default of
CharSequence, its broken". But I can live with that.
On 10/10/11 10:07 AM, "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>
wrote:
>
> [
>https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plu
>gin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124299#comme
>nt-13124299 ]
>
>Doug Cutting commented on AVRO-803:
>-----------------------------------
>
>I'd prefer not to break back-compatibility this time. It makes it
>impossible for folks to upgrade one project without making source code
>changes to other projects. If you specify
><stringType>String</stringType> in your pom.xml then all your Map keys
>become java.lang.String.
>
>> Java generated Avro classes make using Avro painful and surprising
>> ------------------------------------------------------------------
>>
>> Key: AVRO-803
>> URL: https://issues.apache.org/jira/browse/AVRO-803
>> Project: Avro
>> Issue Type: Improvement
>> Components: java
>> Affects Versions: 1.5.0
>> Environment: Any
>> Reporter: Sam Pullara
>> Assignee: Doug Cutting
>> Fix For: 1.6.0
>>
>> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>>
>>
>> Currently the Avro generated Java classes expose CharSequence in their
>>API. However, you cannot use any old CharSequence when interacting with
>>them. In fact, you have to use the Utf8 class if you want to get
>>consistent results. I think that Avro should work with any CharSequence
>>if that is the API. Here is an example where this happens:
>>
>>https://github.com/spullara/avro-generated-code/blob/master/src/test/java
>>/AnnoyingTest.java
>> That prints out 'false' three times unexpectedly. If you can't get it
>>to print 'true' three times then you should probably change it back to
>>Utf8.
>
>--
>This message is automatically generated by JIRA.
>If you think it was sent incorrectly, please contact your JIRA
>administrators:
>https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
>For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
[jira] [Commented] (AVRO-803) Java generated Avro classes make
using Avro painful and surprising
Posted by "Doug Cutting (Commented) (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/AVRO-803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13124299#comment-13124299 ]
Doug Cutting commented on AVRO-803:
-----------------------------------
I'd prefer not to break back-compatibility this time. It makes it impossible for folks to upgrade one project without making source code changes to other projects. If you specify <stringType>String</stringType> in your pom.xml then all your Map keys become java.lang.String.
> Java generated Avro classes make using Avro painful and surprising
> ------------------------------------------------------------------
>
> Key: AVRO-803
> URL: https://issues.apache.org/jira/browse/AVRO-803
> Project: Avro
> Issue Type: Improvement
> Components: java
> Affects Versions: 1.5.0
> Environment: Any
> Reporter: Sam Pullara
> Assignee: Doug Cutting
> Fix For: 1.6.0
>
> Attachments: AVRO-803.patch, AVRO-803.patch, Foo.java
>
>
> Currently the Avro generated Java classes expose CharSequence in their API. However, you cannot use any old CharSequence when interacting with them. In fact, you have to use the Utf8 class if you want to get consistent results. I think that Avro should work with any CharSequence if that is the API. Here is an example where this happens:
> https://github.com/spullara/avro-generated-code/blob/master/src/test/java/AnnoyingTest.java
> That prints out 'false' three times unexpectedly. If you can't get it to print 'true' three times then you should probably change it back to Utf8.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira