You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@pig.apache.org by "Prashant Kommireddi (JIRA)" <ji...@apache.org> on 2013/03/02 03:31:14 UTC
[jira] [Commented] (PIG-3194) Changes to ObjectSerializer.java
break compatibility with Hadoop 0.20.2
[ https://issues.apache.org/jira/browse/PIG-3194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591233#comment-13591233 ]
Prashant Kommireddi commented on PIG-3194:
------------------------------------------
Base64 seems to have been refactored a bit for codec 1.4. It now extends a base class (BaseNCodec) and uses a util (StringUtils). There's not much of dependency on StringUtils. Though BaseNCodec and Base64 are slightly tied up (even though it's just the 2 methods we use in pig). We could do a few things here
1. Bring Base* classes into a pig util
2. Eliminate call to encodeBase64URLSafeString and use byte[] encodeBase64 (available in 1.3) instead.
Looking at the Base64 code, there isn't much difference in the 2 methods except that '-' is used for '+' and '_' is used for '/' with URLSafe. Doesn't look like we need the encoding be url safe either?
And as Kai suggested, we can use Base64.decodeBase64(str.getBytes()) from 1.3 for decoding.
Thoughts?
> Changes to ObjectSerializer.java break compatibility with Hadoop 0.20.2
> -----------------------------------------------------------------------
>
> Key: PIG-3194
> URL: https://issues.apache.org/jira/browse/PIG-3194
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.11
> Reporter: Kai Londenberg
>
> The changes to ObjectSerializer.java in the following commit
> http://svn.apache.org/viewvc?view=revision&revision=1403934 break compatibility with Hadoop 0.20.2 Clusters.
> The reason is, that the code uses methods from Apache Commons Codec 1.4 - which are not available in Apache Commons Codec 1.3 which is shipping with Hadoop 0.20.2.
> The offending methods are Base64.decodeBase64(String) and Base64.encodeBase64URLSafeString(byte[])
> If I revert these changes, Pig 0.11.0 candidate 2 works well with our Hadoop 0.20.2 Clusters.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira