You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Luke Lu (JIRA)" <ji...@apache.org> on 2011/05/06 20:24:03 UTC
[jira] [Created] (HADOOP-7264) Bump avro version to at least 1.4.1
Bump avro version to at least 1.4.1
-----------------------------------
Key: HADOOP-7264
URL: https://issues.apache.org/jira/browse/HADOOP-7264
Project: Hadoop Common
Issue Type: Improvement
Components: io
Affects Versions: 0.21.0
Reporter: Luke Lu
Fix For: 0.23.0
Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087419#comment-13087419 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
I just noticed that v4 (since v2) keeps avro source (*.av{sc,pr}) in java source directory, which requires significant explicit configuration of avro-maven-plugin, unlike in MRv2, all avro source is in avro's source directory, which I (and Sidd) think, is more convenient and conforming to maven conventions, which means default configuration would work with the plugin. But I can see some benefit of keep foreign source in java source as well. What other people think about this?
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087556#comment-13087556 ]
Hadoop QA commented on HADOOP-7264:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12490927/HADOOP-7264v5.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 23 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/55//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/55//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/55//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/55//console
This message is automatically generated.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7264:
---------------------------------------
Attachment: HADOOP-7264v2.patch
Updated patch for common using Avro 1.5.2.
Remove unnecessary/unrelated changes to the POMs
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085996#comment-13085996 ]
Scott Carey commented on HADOOP-7264:
-------------------------------------
Avro-1.5.x also allows you to pick and choose specific dependency subsets as needed, while all were in one jar file before.
avro-ipc seems to be used. If so the dependency on avro is not needed (avro is an implicit transitive dependency of avro-ipc).
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087817#comment-13087817 ]
Hadoop QA commented on HADOOP-7264:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12490970/HADOOP-7264v6.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 21 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/58//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/58//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/58//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/58//console
This message is automatically generated.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087812#comment-13087812 ]
Tom White commented on HADOOP-7264:
-----------------------------------
+1
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030097#comment-13030097 ]
Todd Lipcon commented on HADOOP-7264:
-------------------------------------
k, thanks for the perspective. I bring this up because we've heard from a number of users that, when we change the version of dependencies like thrift, avro, or (especially) Jackson, it hurts users who use these same libraries in their jobs.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13084422#comment-13084422 ]
Tom White commented on HADOOP-7264:
-----------------------------------
Avro 1.5.2 has just been released today, so it would be good to use that version.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081349#comment-13081349 ]
Arun C Murthy commented on HADOOP-7264:
---------------------------------------
Luke, we do have the patch for 'classic' MR trunk in y-merge, can you pls port it and commit? Thanks.
This will ensure nothing breaks... with or without the MR-279 merge.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087420#comment-13087420 ]
Alejandro Abdelnur commented on HADOOP-7264:
--------------------------------------------
I'm OK either way.
Actually I prefer your approach (I didn't see your first patch until I've uploaded my first version).
My motivation for doing the way it is in my patches was that I'm trying to be as little invasive as possible for all Mavenization changes, seen many things that could be done better by rearranging things. I just didn't want to add more noise than the strictly necessary during the Mavenization.
But, if strong about doing this for this JIRA, I can update the patch.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087238#comment-13087238 ]
Tom White commented on HADOOP-7264:
-----------------------------------
Sorry, I missed those failures. It looks like the problem is that AvroTestUtil uses a buffered binary encoder which isn't properly flushed. It would be simpler to use a direct binary encoder in this case.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Tom White updated HADOOP-7264:
------------------------------
Resolution: Fixed
Hadoop Flags: [Incompatible change, Reviewed] (was: [Incompatible change])
Status: Resolved (was: Patch Available)
I've just committed this. Thanks Alejandro and Luke!
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081262#comment-13081262 ]
Arun C Murthy commented on HADOOP-7264:
---------------------------------------
+1 for bumping - MR moved to 1.4.* long ago.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087882#comment-13087882 ]
Alejandro Abdelnur commented on HADOOP-7264:
--------------------------------------------
@Luke, I have not seen any failure related to this
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087130#comment-13087130 ]
Alejandro Abdelnur commented on HADOOP-7264:
--------------------------------------------
@Scott, if avro classes are used directly in hadoop-common then an explicit dependency is the correct way of doing it.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030105#comment-13030105 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
We do want to improve the isolation of dependencies between framework and user code, which I believe is related but really orthogonal to this jira. HADOOP-7150 is trying to achieve that. I believe it's mostly achievable via careful refactor of classpath construction and a little shade/jar/jar in the near term and completely nailed with some like OSGi in the long term.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087139#comment-13087139 ]
Hadoop QA commented on HADOOP-7264:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12490821/HADOOP-7264v3.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 core tests. The patch failed these core unit tests:
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/48//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/48//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/48//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/48//console
This message is automatically generated.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087779#comment-13087779 ]
Tom White commented on HADOOP-7264:
-----------------------------------
I notice the latest patch moves MapReduce Avro protocol files into Common. Shouldn't they stay in the MapReduce module?
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Arun C Murthy (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081263#comment-13081263 ]
Arun C Murthy commented on HADOOP-7264:
---------------------------------------
Todd in MR2, the NM's deps don't leak to the containers (tasks).
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Tom White (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087167#comment-13087167 ]
Tom White commented on HADOOP-7264:
-----------------------------------
+1 This looks good to me. I'm assuming that the MapReduce portion will be handled by MAPREDUCE-2848.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu reassigned HADOOP-7264:
-------------------------------
Assignee: Luke Lu
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7264:
---------------------------------------
Attachment: HADOOP-7264v4.patch
'v4' fixing failing tests
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030077#comment-13030077 ]
Todd Lipcon commented on HADOOP-7264:
-------------------------------------
Is it worth considering maven shade or jarjar for avro? Or in MR2 do we no longer have the issue that the daemon formerly known as the TaskTracker leaks its jar dependencies onto the user classpath?
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087871#comment-13087871 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
Not sure about the current status of HDFS mavenization. But the trunk HDFS build might fail because paranamer-ant transitive dependency is gone. HDFS-2268 was part of my original v1 patch.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7264:
---------------------------------------
Attachment: HADOOP-7264v3.patch
'v3' version, rebased to current trunk (post MR279), leaving outside upgrade of of other components (slf4j)
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087179#comment-13087179 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
There're 3 avro related [test failures|https://builds.apache.org/job/PreCommit-HADOOP-Build/48//testReport/]
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Scott Carey (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087426#comment-13087426 ]
Scott Carey commented on HADOOP-7264:
-------------------------------------
Maven convention is to have each type of source in its own tree. avro-maven-plugin will work if you point it elsewhere, but some other tools might not.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated HADOOP-7264:
----------------------------
Status: Patch Available (was: Open)
TestLeafQueue doesn't pass now in MR-279, which seems to be unrelated. Arun is looking at it.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hudson (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087833#comment-13087833 ]
Hudson commented on HADOOP-7264:
--------------------------------
Integrated in Hadoop-Common-trunk-Commit #757 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/757/])
HADOOP-7264. Bump avro version to at least 1.4.1. Contributed by Alejandro Abdelnur
tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1159699
Files :
* /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/io/AvroTestUtil.java
* /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/ipc/TestAvroRpc.java
* /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/ipc/AvroTestProtocol.java
* /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/ipc/AvroRpcEngine.java
* /hadoop/common/trunk/hadoop-common/src/test/avro
* /hadoop/common/trunk/hadoop-common/src/test/avro/avroRecord.avsc
* /hadoop/common/trunk/hadoop-common/CHANGES.txt
* /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/ipc/AvroSpecificTestProtocol.avpr
* /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/io/serializer/avro/AvroSerialization.java
* /hadoop/common/trunk/hadoop-common/src/test/java/org/apache/hadoop/io/serializer/avro/avroRecord.avsc
* /hadoop/common/trunk/hadoop-common/src/main/java/org/apache/hadoop/ipc/AvroSpecificRpcEngine.java
* /hadoop/common/trunk/hadoop-common/pom.xml
* /hadoop/common/trunk/hadoop-common/src/test/avro/AvroSpecificTestProtocol.avpr
* /hadoop/common/trunk/hadoop-project/pom.xml
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Todd Lipcon (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081265#comment-13081265 ]
Todd Lipcon commented on HADOOP-7264:
-------------------------------------
Great news :)
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated HADOOP-7264:
----------------------------
Attachment: hadoop-7264-mr2-v1.patch
mr2 v1 patch is the for MR-279 branch only.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: hadoop-7264-mr2-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13085962#comment-13085962 ]
Hadoop QA commented on HADOOP-7264:
-----------------------------------
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12490572/HADOOP-7264v2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/41//console
This message is automatically generated.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7264:
---------------------------------------
Attachment: HADOOP-7264v6.patch
v5 was incorrectly moving an avro file from MR, v6 fixes that
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13081329#comment-13081329 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
BTW, HADOOP-6671 bumped versions of many packages (jetty, jackson, slf4j) with no ill-effect yet :). The avro bump from 1.3+ to 1.4+ will break the build of trunk mapreduce jobhistory, which will be replaced by MR-279, which is already on 1.4+.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Luke Lu updated HADOOP-7264:
----------------------------
Attachment: hadoop-7264-v1.patch
v1 patch took the bump avro to the current 1.5.1. Since it'd be an incompatible change to bump to 1.4.1 anyway, we might as well upgrade the current stable version with an official maven plugin.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HADOOP-7264) Bump avro version to at least 1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Alejandro Abdelnur updated HADOOP-7264:
---------------------------------------
Attachment: HADOOP-7264v5.patch
'v5' follows Luke & Scott recommendation.
Moved the *.avpr and *.avsc files to src/test/avro/.
The plugin config is simpler.
The only thing I've left is that the generate d java code goes to target/generated-test-sources/java (instead leaving the default target/generated-test-sources/avro directory).
The reason for this is that in Eclipse, the generated*sources/<DIR> directories have to be manually added to the IDE directory and by keeping java it is a single dir that has to be added instead multiple.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13086038#comment-13086038 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
Looks like the patch is probably out of date due to HADOOP-7536 and INFRA-3857.
@Scott, yes, avro-ipc should suffice.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087884#comment-13087884 ]
Alejandro Abdelnur commented on HADOOP-7264:
--------------------------------------------
(enter too fast), where are you seeing a failure?
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, HADOOP-7264v5.patch, HADOOP-7264v6.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Luke Lu (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030082#comment-13030082 ]
Luke Lu commented on HADOOP-7264:
---------------------------------
maven shade and jarjar doesn't help in this case for mr2 (it's in the common code in job history), as both client and server needs to be on at least 1.4.1, sans an OSGi containment in the job history server, which would be over-engineering at this point. maven shade/jarjar would be helpful for future client libraries.
BTW, we initially actually used maven shade for various servers, but had to remove it as the added processing time, the annoying warnings and the resulting bulk really don't worth the trouble.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Hadoop QA (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087389#comment-13087389 ]
Hadoop QA commented on HADOOP-7264:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12490887/HADOOP-7264v4.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 20 new or modified tests.
+1 javadoc. The javadoc tool did not generate any warning messages.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 core tests. The patch passed core unit tests.
+1 contrib tests. The patch passed contrib unit tests.
+1 system test framework. The patch passed system test framework compile.
Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/50//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/50//artifact/trunk/target/newPatchFindbugsWarningshadoop-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/50//artifact/trunk/target/newPatchFindbugsWarningshadoop-annotations.html
Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/50//console
This message is automatically generated.
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7264) Bump avro version to at least
1.4.1
Posted by "Alejandro Abdelnur (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13087429#comment-13087429 ]
Alejandro Abdelnur commented on HADOOP-7264:
--------------------------------------------
Scott, I like that better. Still, we want to do this as part of this patch or as a follow up that puts things in better places not only for avro but for protobuffs, etc, etc?
> Bump avro version to at least 1.4.1
> -----------------------------------
>
> Key: HADOOP-7264
> URL: https://issues.apache.org/jira/browse/HADOOP-7264
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io
> Affects Versions: 0.21.0
> Reporter: Luke Lu
> Assignee: Luke Lu
> Labels: avro
> Fix For: 0.23.0
>
> Attachments: HADOOP-7264v2.patch, HADOOP-7264v3.patch, HADOOP-7264v4.patch, hadoop-7264-mr2-v1.patch, hadoop-7264-v1.patch
>
>
> Needed by mapreduce 2.0 avro support. Maybe we could jump to Avro 1.5. There is incompatible API changes from 1.3x to 1.4x (Utf8 to CharSequence in user facing APIs) not sure about 1.5x though.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira