You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@drill.apache.org by Li HM <hm...@gmail.com> on 2014/10/07 07:21:27 UTC
Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
Anybody ever made it work?
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Venki Korukanti <ve...@gmail.com>.
Only official Apache Hive release in 0.12.x is 0.12.0 (
http://hive.apache.org/downloads.html). What is Hive 0.12.4? Is it your
customized version of official Hive 0.12.0 release? If that is case it must
be containing patches that changed APIs such as
*ColumnProjectionUtils. appendReadColumnIDs.*
On Tue, Oct 7, 2014 at 10:13 PM, Hmxxyy <hm...@gmail.com> wrote:
> The hive jars I used are 0.12.4 with hive 7620 fixed. Not 0.13.
>
> Sent from my iPhone
>
> > On Oct 7, 2014, at 9:46 PM, Venki Korukanti <ve...@gmail.com>
> wrote:
> >
> > Current storage plugin is using Hive 0.12 APIs. From the build errors it
> > looks like the version of hive jar that you replaced is >= 0.13.0. Hive
> > 0.13 (and above) have few API changes compared Hive 0.12. In order to
> > resolve the kerberos error use jars built from Apache Hive 0.12 source +
> > HIVE-7620 patch.
> >
> >> On Tue, Oct 7, 2014 at 8:57 PM, Li HM <hm...@gmail.com> wrote:
> >>
> >> after I replaced the hive jar in local maven repo, and change the
> >> hadoop version from 2.4.1 to 2.5.0 in pom.xml.
> >>
> >> The build failed with the following error:
> >>
> >> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
> >> drill-storage-hive-core ---
> >> [INFO] Changes detected - recompiling the module!
> >> [INFO] Compiling 34 source files to
> >> /home/lih/incubator-drill/contrib/storage-hive/core/target/classes
> >> [INFO] -------------------------------------------------------------
> >> [ERROR] COMPILATION ERROR :
> >> [INFO] -------------------------------------------------------------
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
> >> error: cannot find symbol
> >> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
> >> location: class ColumnProjectionUtils
> >>
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
> >> error: appendReadColumnNames(Configuration,List<String>) has private
> >> access in ColumnProjectionUtils
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
> >> error: constructor HiveDecimal in class HiveDecimal cannot be applied
> >> to given types;
> >> [ERROR] required: BigDecimal
> >> found: String
> >> reason: actual argument String cannot be converted to BigDecimal by
> >> method invocation conversion
> >>
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
> >> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
> >> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
> >> error: no suitable constructor found for
> >> GenericUDFBridge(String,boolean,Class<CAP#1>)
> >> [INFO] 6 errors
> >> [INFO] -------------------------------------------------------------
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Reactor Summary:
> >> [INFO]
> >> [INFO] Apache Drill Root POM ............................. SUCCESS
> >> [18.888s]
> >> [INFO] Drill Protocol .................................... SUCCESS
> >> [26.133s]
> >> [INFO] Common (Logical Plan, Base expressions) ........... SUCCESS
> >> [26.195s]
> >> [INFO] contrib/Parent Pom ................................ SUCCESS
> [1.563s]
> >> [INFO] contrib/data/Parent Pom ........................... SUCCESS
> [0.970s]
> >> [INFO] contrib/data/tpch-sample-data ..................... SUCCESS
> [9.590s]
> >> [INFO] exec/Parent Pom ................................... SUCCESS
> [1.966s]
> >> [INFO] exec/Java Execution Engine ........................ SUCCESS
> >> [5:18.113s]
> >> [INFO] contrib/hbase-storage-plugin ...................... SUCCESS
> >> [22.751s]
> >> [INFO] contrib/hive-storage-plugin/Parent Pom ............ SUCCESS
> [1.304s]
> >> [INFO] contrib/hive-storage-plugin/hive-exec-shaded ...... SUCCESS
> >> [28.498s]
> >> [INFO] contrib/hive-storage-plugin/core .................. FAILURE
> >> [12.699s]
> >> [INFO] contrib/mongo-storage-plugin ...................... SKIPPED
> >> [INFO] exec/JDBC Driver using dependencies ............... SKIPPED
> >> [INFO] contrib/sqlline ................................... SKIPPED
> >> [INFO] JDBC JAR with all dependencies .................... SKIPPED
> >> [INFO] exec/Drill expression interpreter ................. SKIPPED
> >> [INFO] Packaging and Distribution Assembly ............... SKIPPED
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] BUILD FAILURE
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Total time: 7:50.911s
> >> [INFO] Finished at: Wed Oct 08 03:48:06 UTC 2014
> >> [INFO] Final Memory: 72M/294M
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [ERROR] Failed to execute goal
> >> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
> >> (default-compile) on project drill-storage-hive-core: Compilation
> >> failure: Compilation failure:
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
> >> error: cannot find symbol
> >> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
> >> [ERROR] location: class ColumnProjectionUtils
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
> >> error: appendReadColumnNames(Configuration,List<String>) has private
> >> access in ColumnProjectionUtils
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
> >> error: constructor HiveDecimal in class HiveDecimal cannot be applied
> >> to given types;
> >> [ERROR] required: BigDecimal
> >> [ERROR] found: String
> >> [ERROR] reason: actual argument String cannot be converted to
> >> BigDecimal by method invocation conversion
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
> >> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
> >> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> >> [ERROR]
> >>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
> >> error: no suitable constructor found for
> >> GenericUDFBridge(String,boolean,Class<CAP#1>)
> >> [ERROR] -> [Help 1]
> >> [ERROR]
> >> [ERROR] To see the full stack trace of the errors, re-run Maven with
> >> the -e switch.
> >> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> >> [ERROR]
> >> [ERROR] For more information about the errors and possible solutions,
> >> please read the following articles:
> >> [ERROR] [Help 1]
> >> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> >> [ERROR]
> >> [ERROR] After correcting the problems, you can resume the build with the
> >> command
> >> [ERROR] mvn <goals> -rf :drill-storage-hive-core
> >>
> >> On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
> >> <ve...@gmail.com> wrote:
> >>> Hi,
> >>>
> >>> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
> >>> changes in Hadoop 2.5.0. More details are here:
> >>> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it
> >> work is
> >>> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
> >>> local maven repo, then build drill which picks ups hive jars in local
> >> maven
> >>> repo.
> >>>
> >>> Thanks
> >>> Venki
> >>>
> >>>> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
> >>>>
> >>>> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
> >>>>
> >>>> Anybody ever made it work?
> >>
>
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Hmxxyy <hm...@gmail.com>.
The hive jars I used are 0.12.4 with hive 7620 fixed. Not 0.13.
Sent from my iPhone
> On Oct 7, 2014, at 9:46 PM, Venki Korukanti <ve...@gmail.com> wrote:
>
> Current storage plugin is using Hive 0.12 APIs. From the build errors it
> looks like the version of hive jar that you replaced is >= 0.13.0. Hive
> 0.13 (and above) have few API changes compared Hive 0.12. In order to
> resolve the kerberos error use jars built from Apache Hive 0.12 source +
> HIVE-7620 patch.
>
>> On Tue, Oct 7, 2014 at 8:57 PM, Li HM <hm...@gmail.com> wrote:
>>
>> after I replaced the hive jar in local maven repo, and change the
>> hadoop version from 2.4.1 to 2.5.0 in pom.xml.
>>
>> The build failed with the following error:
>>
>> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
>> drill-storage-hive-core ---
>> [INFO] Changes detected - recompiling the module!
>> [INFO] Compiling 34 source files to
>> /home/lih/incubator-drill/contrib/storage-hive/core/target/classes
>> [INFO] -------------------------------------------------------------
>> [ERROR] COMPILATION ERROR :
>> [INFO] -------------------------------------------------------------
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
>> error: cannot find symbol
>> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
>> location: class ColumnProjectionUtils
>>
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
>> error: appendReadColumnNames(Configuration,List<String>) has private
>> access in ColumnProjectionUtils
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
>> error: constructor HiveDecimal in class HiveDecimal cannot be applied
>> to given types;
>> [ERROR] required: BigDecimal
>> found: String
>> reason: actual argument String cannot be converted to BigDecimal by
>> method invocation conversion
>>
>> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
>> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
>> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
>> error: no suitable constructor found for
>> GenericUDFBridge(String,boolean,Class<CAP#1>)
>> [INFO] 6 errors
>> [INFO] -------------------------------------------------------------
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Reactor Summary:
>> [INFO]
>> [INFO] Apache Drill Root POM ............................. SUCCESS
>> [18.888s]
>> [INFO] Drill Protocol .................................... SUCCESS
>> [26.133s]
>> [INFO] Common (Logical Plan, Base expressions) ........... SUCCESS
>> [26.195s]
>> [INFO] contrib/Parent Pom ................................ SUCCESS [1.563s]
>> [INFO] contrib/data/Parent Pom ........................... SUCCESS [0.970s]
>> [INFO] contrib/data/tpch-sample-data ..................... SUCCESS [9.590s]
>> [INFO] exec/Parent Pom ................................... SUCCESS [1.966s]
>> [INFO] exec/Java Execution Engine ........................ SUCCESS
>> [5:18.113s]
>> [INFO] contrib/hbase-storage-plugin ...................... SUCCESS
>> [22.751s]
>> [INFO] contrib/hive-storage-plugin/Parent Pom ............ SUCCESS [1.304s]
>> [INFO] contrib/hive-storage-plugin/hive-exec-shaded ...... SUCCESS
>> [28.498s]
>> [INFO] contrib/hive-storage-plugin/core .................. FAILURE
>> [12.699s]
>> [INFO] contrib/mongo-storage-plugin ...................... SKIPPED
>> [INFO] exec/JDBC Driver using dependencies ............... SKIPPED
>> [INFO] contrib/sqlline ................................... SKIPPED
>> [INFO] JDBC JAR with all dependencies .................... SKIPPED
>> [INFO] exec/Drill expression interpreter ................. SKIPPED
>> [INFO] Packaging and Distribution Assembly ............... SKIPPED
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD FAILURE
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 7:50.911s
>> [INFO] Finished at: Wed Oct 08 03:48:06 UTC 2014
>> [INFO] Final Memory: 72M/294M
>> [INFO]
>> ------------------------------------------------------------------------
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
>> (default-compile) on project drill-storage-hive-core: Compilation
>> failure: Compilation failure:
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
>> error: cannot find symbol
>> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
>> [ERROR] location: class ColumnProjectionUtils
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
>> error: appendReadColumnNames(Configuration,List<String>) has private
>> access in ColumnProjectionUtils
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
>> error: constructor HiveDecimal in class HiveDecimal cannot be applied
>> to given types;
>> [ERROR] required: BigDecimal
>> [ERROR] found: String
>> [ERROR] reason: actual argument String cannot be converted to
>> BigDecimal by method invocation conversion
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
>> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
>> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
>> [ERROR]
>> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
>> error: no suitable constructor found for
>> GenericUDFBridge(String,boolean,Class<CAP#1>)
>> [ERROR] -> [Help 1]
>> [ERROR]
>> [ERROR] To see the full stack trace of the errors, re-run Maven with
>> the -e switch.
>> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
>> [ERROR]
>> [ERROR] For more information about the errors and possible solutions,
>> please read the following articles:
>> [ERROR] [Help 1]
>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
>> [ERROR]
>> [ERROR] After correcting the problems, you can resume the build with the
>> command
>> [ERROR] mvn <goals> -rf :drill-storage-hive-core
>>
>> On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
>> <ve...@gmail.com> wrote:
>>> Hi,
>>>
>>> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
>>> changes in Hadoop 2.5.0. More details are here:
>>> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it
>> work is
>>> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
>>> local maven repo, then build drill which picks ups hive jars in local
>> maven
>>> repo.
>>>
>>> Thanks
>>> Venki
>>>
>>>> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
>>>>
>>>> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
>>>>
>>>> Anybody ever made it work?
>>
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Venki Korukanti <ve...@gmail.com>.
Current storage plugin is using Hive 0.12 APIs. From the build errors it
looks like the version of hive jar that you replaced is >= 0.13.0. Hive
0.13 (and above) have few API changes compared Hive 0.12. In order to
resolve the kerberos error use jars built from Apache Hive 0.12 source +
HIVE-7620 patch.
On Tue, Oct 7, 2014 at 8:57 PM, Li HM <hm...@gmail.com> wrote:
> after I replaced the hive jar in local maven repo, and change the
> hadoop version from 2.4.1 to 2.5.0 in pom.xml.
>
> The build failed with the following error:
>
> [INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
> drill-storage-hive-core ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 34 source files to
> /home/lih/incubator-drill/contrib/storage-hive/core/target/classes
> [INFO] -------------------------------------------------------------
> [ERROR] COMPILATION ERROR :
> [INFO] -------------------------------------------------------------
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
> error: cannot find symbol
> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
> location: class ColumnProjectionUtils
>
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
> error: appendReadColumnNames(Configuration,List<String>) has private
> access in ColumnProjectionUtils
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
> error: constructor HiveDecimal in class HiveDecimal cannot be applied
> to given types;
> [ERROR] required: BigDecimal
> found: String
> reason: actual argument String cannot be converted to BigDecimal by
> method invocation conversion
>
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
> error: no suitable constructor found for
> GenericUDFBridge(String,boolean,Class<CAP#1>)
> [INFO] 6 errors
> [INFO] -------------------------------------------------------------
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Apache Drill Root POM ............................. SUCCESS
> [18.888s]
> [INFO] Drill Protocol .................................... SUCCESS
> [26.133s]
> [INFO] Common (Logical Plan, Base expressions) ........... SUCCESS
> [26.195s]
> [INFO] contrib/Parent Pom ................................ SUCCESS [1.563s]
> [INFO] contrib/data/Parent Pom ........................... SUCCESS [0.970s]
> [INFO] contrib/data/tpch-sample-data ..................... SUCCESS [9.590s]
> [INFO] exec/Parent Pom ................................... SUCCESS [1.966s]
> [INFO] exec/Java Execution Engine ........................ SUCCESS
> [5:18.113s]
> [INFO] contrib/hbase-storage-plugin ...................... SUCCESS
> [22.751s]
> [INFO] contrib/hive-storage-plugin/Parent Pom ............ SUCCESS [1.304s]
> [INFO] contrib/hive-storage-plugin/hive-exec-shaded ...... SUCCESS
> [28.498s]
> [INFO] contrib/hive-storage-plugin/core .................. FAILURE
> [12.699s]
> [INFO] contrib/mongo-storage-plugin ...................... SKIPPED
> [INFO] exec/JDBC Driver using dependencies ............... SKIPPED
> [INFO] contrib/sqlline ................................... SKIPPED
> [INFO] JDBC JAR with all dependencies .................... SKIPPED
> [INFO] exec/Drill expression interpreter ................. SKIPPED
> [INFO] Packaging and Distribution Assembly ............... SKIPPED
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 7:50.911s
> [INFO] Finished at: Wed Oct 08 03:48:06 UTC 2014
> [INFO] Final Memory: 72M/294M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
> (default-compile) on project drill-storage-hive-core: Compilation
> failure: Compilation failure:
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
> error: cannot find symbol
> [ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
> [ERROR] location: class ColumnProjectionUtils
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
> error: appendReadColumnNames(Configuration,List<String>) has private
> access in ColumnProjectionUtils
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
> error: constructor HiveDecimal in class HiveDecimal cannot be applied
> to given types;
> [ERROR] required: BigDecimal
> [ERROR] found: String
> [ERROR] reason: actual argument String cannot be converted to
> BigDecimal by method invocation conversion
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
> error: HiveDecimal(BigDecimal) has private access in HiveDecimal
> [ERROR]
> /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
> error: no suitable constructor found for
> GenericUDFBridge(String,boolean,Class<CAP#1>)
> [ERROR] -> [Help 1]
> [ERROR]
> [ERROR] To see the full stack trace of the errors, re-run Maven with
> the -e switch.
> [ERROR] Re-run Maven using the -X switch to enable full debug logging.
> [ERROR]
> [ERROR] For more information about the errors and possible solutions,
> please read the following articles:
> [ERROR] [Help 1]
> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
> [ERROR]
> [ERROR] After correcting the problems, you can resume the build with the
> command
> [ERROR] mvn <goals> -rf :drill-storage-hive-core
>
> On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
> <ve...@gmail.com> wrote:
> > Hi,
> >
> > Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
> > changes in Hadoop 2.5.0. More details are here:
> > https://issues.apache.org/jira/browse/HIVE-7620. One way to make it
> work is
> > port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
> > local maven repo, then build drill which picks ups hive jars in local
> maven
> > repo.
> >
> > Thanks
> > Venki
> >
> > On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
> >
> >> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
> >>
> >> Anybody ever made it work?
> >>
>
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Li HM <hm...@gmail.com>.
after I replaced the hive jar in local maven repo, and change the
hadoop version from 2.4.1 to 2.5.0 in pom.xml.
The build failed with the following error:
[INFO] --- maven-compiler-plugin:3.0:compile (default-compile) @
drill-storage-hive-core ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 34 source files to
/home/lih/incubator-drill/contrib/storage-hive/core/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
error: cannot find symbol
[ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
location: class ColumnProjectionUtils
/home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
error: appendReadColumnNames(Configuration,List<String>) has private
access in ColumnProjectionUtils
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
error: constructor HiveDecimal in class HiveDecimal cannot be applied
to given types;
[ERROR] required: BigDecimal
found: String
reason: actual argument String cannot be converted to BigDecimal by
method invocation conversion
/home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
error: HiveDecimal(BigDecimal) has private access in HiveDecimal
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
error: HiveDecimal(BigDecimal) has private access in HiveDecimal
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
error: no suitable constructor found for
GenericUDFBridge(String,boolean,Class<CAP#1>)
[INFO] 6 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Drill Root POM ............................. SUCCESS [18.888s]
[INFO] Drill Protocol .................................... SUCCESS [26.133s]
[INFO] Common (Logical Plan, Base expressions) ........... SUCCESS [26.195s]
[INFO] contrib/Parent Pom ................................ SUCCESS [1.563s]
[INFO] contrib/data/Parent Pom ........................... SUCCESS [0.970s]
[INFO] contrib/data/tpch-sample-data ..................... SUCCESS [9.590s]
[INFO] exec/Parent Pom ................................... SUCCESS [1.966s]
[INFO] exec/Java Execution Engine ........................ SUCCESS [5:18.113s]
[INFO] contrib/hbase-storage-plugin ...................... SUCCESS [22.751s]
[INFO] contrib/hive-storage-plugin/Parent Pom ............ SUCCESS [1.304s]
[INFO] contrib/hive-storage-plugin/hive-exec-shaded ...... SUCCESS [28.498s]
[INFO] contrib/hive-storage-plugin/core .................. FAILURE [12.699s]
[INFO] contrib/mongo-storage-plugin ...................... SKIPPED
[INFO] exec/JDBC Driver using dependencies ............... SKIPPED
[INFO] contrib/sqlline ................................... SKIPPED
[INFO] JDBC JAR with all dependencies .................... SKIPPED
[INFO] exec/Drill expression interpreter ................. SKIPPED
[INFO] Packaging and Distribution Assembly ............... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7:50.911s
[INFO] Finished at: Wed Oct 08 03:48:06 UTC 2014
[INFO] Final Memory: 72M/294M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.0:compile
(default-compile) on project drill-storage-hive-core: Compilation
failure: Compilation failure:
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[192,29]
error: cannot find symbol
[ERROR] symbol: method appendReadColumnIDs(JobConf,List<Integer>)
[ERROR] location: class ColumnProjectionUtils
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[193,29]
error: appendReadColumnNames(Configuration,List<String>) has private
access in ColumnProjectionUtils
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveRecordReader.java:[524,15]
error: constructor HiveDecimal in class HiveDecimal cannot be applied
to given types;
[ERROR] required: BigDecimal
[ERROR] found: String
[ERROR] reason: actual argument String cannot be converted to
BigDecimal by method invocation conversion
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[58,13]
error: HiveDecimal(BigDecimal) has private access in HiveDecimal
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/target/generated-sources/org/apache/drill/exec/expr/fn/impl/hive/DrillDecimal38SparseObjectInspector.java:[66,13]
error: HiveDecimal(BigDecimal) has private access in HiveDecimal
[ERROR] /home/lih/incubator-drill/contrib/storage-hive/core/src/main/java/org/apache/drill/exec/expr/fn/HiveFunctionRegistry.java:[189,31]
error: no suitable constructor found for
GenericUDFBridge(String,boolean,Class<CAP#1>)
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with
the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :drill-storage-hive-core
On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
<ve...@gmail.com> wrote:
> Hi,
>
> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
> changes in Hadoop 2.5.0. More details are here:
> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it work is
> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
> local maven repo, then build drill which picks ups hive jars in local maven
> repo.
>
> Thanks
> Venki
>
> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
>
>> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
>>
>> Anybody ever made it work?
>>
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Li HM <hm...@gmail.com>.
sorry, wrong thread. Please ignore.
On Mon, Oct 6, 2014 at 11:18 PM, Li HM <hm...@gmail.com> wrote:
> Thanks for the replied.
>
> Please refer to my another post entitled "How to make ./bin/spark-sql
> work with hive". It has all the error/exceptions I am getting.
>
> If I understand you correctly, I can build the package with
> mvn -Phive,hadoop-2.4 -Dhadoop.version=2.5.0 clean package
>
> This is what I actually tried.
>
>
> On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
> <ve...@gmail.com> wrote:
>>
>> Hi,
>>
>> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
>> changes in Hadoop 2.5.0. More details are here:
>> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it work is
>> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
>> local maven repo, then build drill which picks ups hive jars in local maven
>> repo.
>>
>> Thanks
>> Venki
>>
>> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
>>
>> > Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
>> >
>> > Anybody ever made it work?
>> >
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Li HM <hm...@gmail.com>.
Thanks for the replied.
Please refer to my another post entitled "How to make ./bin/spark-sql
work with hive". It has all the error/exceptions I am getting.
If I understand you correctly, I can build the package with
mvn -Phive,hadoop-2.4 -Dhadoop.version=2.5.0 clean package
This is what I actually tried.
On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
<ve...@gmail.com> wrote:
>
> Hi,
>
> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
> changes in Hadoop 2.5.0. More details are here:
> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it work is
> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
> local maven repo, then build drill which picks ups hive jars in local maven
> repo.
>
> Thanks
> Venki
>
> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
>
> > Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
> >
> > Anybody ever made it work?
> >
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Li HM <hm...@gmail.com>.
thanks. I already have an existing working version of hive. I will try
to put them into local maven repo and rebuild drill.
On Mon, Oct 6, 2014 at 11:07 PM, Venki Korukanti
<ve...@gmail.com> wrote:
> Hi,
>
> Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
> changes in Hadoop 2.5.0. More details are here:
> https://issues.apache.org/jira/browse/HIVE-7620. One way to make it work is
> port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
> local maven repo, then build drill which picks ups hive jars in local maven
> repo.
>
> Thanks
> Venki
>
> On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
>
>> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
>>
>> Anybody ever made it work?
>>
Re: Drill 0.5.0 with Hive 0.12.0 on hadoop 2.5.0
Posted by Venki Korukanti <ve...@gmail.com>.
Hi,
Hive 0.12 with kerberos is incompatible with Hadoop 2.5.0 due to API
changes in Hadoop 2.5.0. More details are here:
https://issues.apache.org/jira/browse/HIVE-7620. One way to make it work is
port HIVE-7620 patch to hive 0.12, build Hive and install hive jars in
local maven repo, then build drill which picks ups hive jars in local maven
repo.
Thanks
Venki
On Mon, Oct 6, 2014 at 10:21 PM, Li HM <hm...@gmail.com> wrote:
> Does Drill support Hive 0.12.0 on top of hadoop 2.5.0 with kerberos?
>
> Anybody ever made it work?
>