Hive uses hadoop version information which unfortunately is unknown

[Edward Capriolo <ed...@gmail.com>]

I would like to run hive release, unfortunately I have an older 18.3
hadoop that was build without version information.

I am going to check out the 4.1 release and hack at the shell scripts
and the relevant code in hive. Does someone have the output of 'hadoop
version' 18.3 on 18.3, or a better way to handle my problem?
(I know the correct answer is update hadoop but currently I am not
going to follow this course)

Thanks
Edward

Re: Hive uses hadoop version information which unfortunately is unknown

[Edward Capriolo <ed...@gmail.com>]

On Tue, Feb 9, 2010 at 4:24 PM, Jay Booth <ja...@gmail.com> wrote:
> Hey Ed, if you have the option of recompiling hadoop without updating, you
> can simply run "src/saveVersion.sh 0.18.3 ; ant bin-package", and that will
> tag your binaries with the 0.18.3 label.
> If recompiling's not an option..  sorry.  This one's bitten us a couple
> times when rolling out new hadoop versions as well.
>
> On Tue, Feb 9, 2010 at 2:12 PM, Edward Capriolo <ed...@gmail.com>
> wrote:
>>
>> I would like to run hive release, unfortunately I have an older 18.3
>> hadoop that was build without version information.
>>
>> I am going to check out the 4.1 release and hack at the shell scripts
>> and the relevant code in hive. Does someone have the output of 'hadoop
>> version' 18.3 on 18.3, or a better way to handle my problem?
>> (I know the correct answer is update hadoop but currently I am not
>> going to follow this course)
>>
>> Thanks
>> Edward
>
>

Jay,
Thanks. Recompile on hadoop is not an option. 18.3, Its a can of worms :)

Here is what I did, to bypass the unknown. This is bad and breaks the
shim capabilities of hive.
[ecapriolo@hadoopdata10 hive-4.1-rc2]$ svn diff
Index: shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java
===================================================================
--- shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java
 (revision 908160)
+++ shims/src/common/java/org/apache/hadoop/hive/shims/ShimLoader.java
 (working copy)
@@ -94,6 +94,8 @@
    * (e.g "0.18" or "0.20")
    */
   private static String getMajorVersion() {
+    return "0.18";
+    /*
     String vers = VersionInfo.getVersion();

     String parts[] = vers.split("\\.");
@@ -102,5 +104,6 @@
                                  " (expected A.B.* format)");
     }
     return parts[0] + "." + parts[1];
+    */
   }
 }
Index: bin/ext/cli.sh
===================================================================
--- bin/ext/cli.sh      (revision 908160)
+++ bin/ext/cli.sh      (working copy)
@@ -15,7 +15,7 @@
   fi

   version=$($HADOOP version | awk '{if (NR == 1) {print $2;}}');
-
+  version="0.18.3"
   # Save the regex to a var to workaround quoting incompatabilities
   # between Bash 3.1 and 3.2
   version_re="^([[:digit:]]+)\.([[:digit:]]+)(\.([[:digit:]]+))?.*$"
[ecapriolo@nyhadoopdata10 hive-4.1-rc2]$

Re: Hive uses hadoop version information which unfortunately is unknown

[Jay Booth <ja...@gmail.com>]

Hey Ed, if you have the option of recompiling hadoop without updating, you
can simply run "src/saveVersion.sh 0.18.3 ; ant bin-package", and that will
tag your binaries with the 0.18.3 label.

If recompiling's not an option..  sorry.  This one's bitten us a couple
times when rolling out new hadoop versions as well.

On Tue, Feb 9, 2010 at 2:12 PM, Edward Capriolo <ed...@gmail.com>wrote:

> I would like to run hive release, unfortunately I have an older 18.3
> hadoop that was build without version information.
>
> I am going to check out the 4.1 release and hack at the shell scripts
> and the relevant code in hive. Does someone have the output of 'hadoop
> version' 18.3 on 18.3, or a better way to handle my problem?
> (I know the correct answer is update hadoop but currently I am not
> going to follow this course)
>
> Thanks
> Edward
>