You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by "Joep Rottinghuis (JIRA)" <ji...@apache.org> on 2011/02/15 02:12:58 UTC
[jira] Created: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Hive Hadoop20SShims depends on removed HadoopArchives
-----------------------------------------------------
Key: HADOOP-7143
URL: https://issues.apache.org/jira/browse/HADOOP-7143
Project: Hadoop Common
Issue Type: Bug
Components: fs
Affects Versions: 0.20.100
Environment: RHEL5u4
Reporter: Joep Rottinghuis
Fix For: 0.20.100
Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
/hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley reassigned HADOOP-7143:
-------------------------------------
Assignee: Joep Rottinghuis
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.203.0
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Assignee: Joep Rottinghuis
> Fix For: 0.20.203.0
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Updated: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Joep Rottinghuis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Joep Rottinghuis updated HADOOP-7143:
-------------------------------------
Attachment: hadoop-7143.patch
Patch adding back in HadoopArchives and HarFileSystem.
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Owen O'Malley (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Owen O'Malley resolved HADOOP-7143.
-----------------------------------
Resolution: Fixed
Hadoop Flags: [Reviewed]
I just committed this to 203.
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.203.0
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Assignee: Joep Rottinghuis
> Fix For: 0.20.203.0
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994625#comment-12994625 ]
Mahadev konar commented on HADOOP-7143:
---------------------------------------
Joep, We had removed archives to allow folks to use multiple versions of it without being deployed on the cluster. With a patch we can add it back. Where are your getting the har classes from in your patch?
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Joep Rottinghuis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994950#comment-12994950 ]
Joep Rottinghuis commented on HADOOP-7143:
------------------------------------------
The intent was to get them from branch-0.20-security just before HADOOP-7108 was applied.
When double-checking I did find fs/HarFileSystem.java is indeed the same.
HadoopArchives however does show differences:
{code}
678,681c678,680
< int ret = 0;
<
< try{
< ret = ToolRunner.run(harchives, args);
---
> try {
> int res = harchives.run(args);
> System.exit(res);
683,684d681
< LOG.debug("Exception in archives ", e);
< System.err.println("Exception in archives");
686d682
< System.exit(1);
688d683
< System.exit(ret);
690c685
< }
---
> }
\ No newline at end of file
{code}
I'll look into that and rectify with new version of patch.
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Joep Rottinghuis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995150#comment-12995150 ]
Joep Rottinghuis commented on HADOOP-7143:
------------------------------------------
I take that back. Both files are branch-0.20-security just before HADOOP-7108 was applied.
Mahadev, the different versions you refer to are from HADOOP-6591 and MAPREDUCE-1585.
How do you think we should move forward with this?
1) Add original version of HadoopArchives back in as per this patch?
2) Move HadoopArchives to Hive so that it has its own copy with its own version?
3) something else
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Joep Rottinghuis (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994613#comment-12994613 ]
Joep Rottinghuis commented on HADOOP-7143:
------------------------------------------
Not sure why HadoopArchives was removed, nor why HarFileSystem was removed. Was the latter replaced by something else?
As a stop-gap I've just resusitated both classes (see patch). Is that a good idea, or is there an alternative to HadoopArchives and/or HarFileSystem that should be used instead.
Wrt. unit test, I could not locate the accompanying test class for HadoopArchives just before HADOOP-7108 was applied, although I did find a copy here:
http://svn.apache.org/viewvc?view=revision&revision=1040414
although the package structure does not match 1-to-1 probably due to mapreduce split from core.
Would it be appropriate to apply this unit test as part of this patch?
Ditto for TestHarFileSystem found on same URL as listed above. Should I add this back in as well as part of patch for this bug?
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (HADOOP-7143) Hive Hadoop20SShims depends on
removed HadoopArchives
Posted by "Mahadev konar (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HADOOP-7143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12995483#comment-12995483 ]
Mahadev konar commented on HADOOP-7143:
---------------------------------------
joep, we should probably add the original version from the branch-0.20-security before hadoop-7108 was applied.
> Hive Hadoop20SShims depends on removed HadoopArchives
> -----------------------------------------------------
>
> Key: HADOOP-7143
> URL: https://issues.apache.org/jira/browse/HADOOP-7143
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 0.20.100
> Environment: RHEL5u4
> Reporter: Joep Rottinghuis
> Fix For: 0.20.100
>
> Attachments: hadoop-7143.patch
>
>
> Compiling (Hive 0.6 + HIVE-1264) or Hive-trunk against 0.20.100 fails compilation.
> /hive/shims/src/0.20S/java/org/apache/hadoop/hive/shims/Hadoop20SShims.java depends on o.a.h.tools.HadoopArchives which was removed from 0.20.100.
> HadoopArchives in turn depends on src/core/o.a.h/fs/HarFileSystem.java which was also removed from 0.20.100.
--
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira