You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Gerald-G <sh...@gmail.com> on 2015/04/30 09:34:28 UTC

HIVE udf explain error

Hi ALL:

  I have develop three UDF and compile them in one jar.  Hive Explainn one
udf to antother class

Dump INFO as Follow:  Hive explain userlost-->shiftAct(), but the return
type is boolean, the right return type I want


0: jdbc:hive2://10-4-32-53:10000> explain select userlost(idayacti,1) from
fat_tbaccount;
+--------------------------------------------------------------+--+
|                           Explain                            |
+--------------------------------------------------------------+--+
| STAGE DEPENDENCIES:                                          |
|   Stage-0 is a root stage                                    |
|                                                              |
| STAGE PLANS:                                                 |
|   Stage: Stage-0                                             |
|     Fetch Operator                                           |
|       limit: -1                                              |
|       Processor Tree:                                        |
|         TableScan                                            |
|           alias: fat_tbaccount                               |
|           Select Operator                                    |
|             expressions: ShiftAct(idayacti) (type: boolean)  |
|             outputColumnNames: _col0                         |
|             ListSink                                         |
|                                                              |
+--------------------------------------------------------------+--+
15 rows selected (0.657 seconds)
0: jdbc:hive2://10-4-32-53:10000> explain select
userlost(idayacti,iworldid) from fat_tbaccount;
Error: Error while compiling statement: FAILED: SemanticException [Error
10004]: Line 1:33 Invalid table alias or column reference 'iworldid':
(possible column names are: dtstatdate, iaccounttype, suin, igameid,
iworld, iroleid, iregtime, ilastacttime, idayacti, iweekacti, imonthacti,
igroup, ilevel, iviplevel, itimes, ionlinetime) (state=42000,code=10004)
0: jdbc:hive2://10-4-32-53:10000> explain select userlost(idayacti,iworld)
from fat_tbaccount;
+--------------------------------------------------------------+--+
|                           Explain                            |
+--------------------------------------------------------------+--+
| STAGE DEPENDENCIES:                                          |
|   Stage-0 is a root stage                                    |
|                                                              |
| STAGE PLANS:                                                 |
|   Stage: Stage-0                                             |
|     Fetch Operator                                           |
|       limit: -1                                              |
|       Processor Tree:                                        |
|         TableScan                                            |
|           alias: fat_tbaccount                               |
|           Select Operator                                    |
|             expressions: ShiftAct(idayacti) (type: boolean)  |
|             outputColumnNames: _col0                         |
|             ListSink                                         |
|                                                              |
+--------------------------------------------------------------+--+
15 rows selected (0.192 seconds)
0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftleft AS
'com.shadowinlife.app.UserACT.ShiftLeft';
No rows affected (0.064 seconds)
0: jdbc:hive2://10-4-32-53:10000> explain select shiftleft(idayacti) from
fat_tbaccount;
+----------------------------------------------------------------------------------------+--+
|                                        Explain
              |
+----------------------------------------------------------------------------------------+--+
| STAGE DEPENDENCIES:
             |
|   Stage-0 is a root stage
             |
|
             |
| STAGE PLANS:
              |
|   Stage: Stage-0
              |
|     Fetch Operator
              |
|       limit: -1
             |
|       Processor Tree:
             |
|         TableScan
             |
|           alias: fat_tbaccount
              |
|           Select Operator
             |
|             expressions: ShiftLeft(idayacti) (type:
struct<header:int,tailer:bigint>)  |
|             outputColumnNames: _col0
              |
|             ListSink
              |
|
             |
+----------------------------------------------------------------------------------------+--+
15 rows selected (0.217 seconds)
0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
fat_tbaccount;
Error: Error while compiling statement: FAILED: SemanticException [Error
10011]: Line 1:15 Invalid function 'shiftact' (state=42000,code=10011)
0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftact AS
'com.shadowinlife.app.UserACT.ShiftAct';
No rows affected (0.064 seconds)
0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
fat_tbaccount;
+---------------------------------------------------------------------------------------+--+
|                                        Explain
             |
+---------------------------------------------------------------------------------------+--+
| STAGE DEPENDENCIES:
            |
|   Stage-0 is a root stage
            |
|
            |
| STAGE PLANS:
             |
|   Stage: Stage-0
             |
|     Fetch Operator
             |
|       limit: -1
            |
|       Processor Tree:
            |
|         TableScan
            |
|           alias: fat_tbaccount
             |
|           Select Operator
            |
|             expressions: ShiftAct(idayacti) (type:
struct<header:int,tailer:bigint>)  |
|             outputColumnNames: _col0
             |
|             ListSink
             |
|
            |
+---------------------------------------------------------------------------------------+--+

Re: HIVE udf explain error

Posted by Gerald-G <sh...@gmail.com>.
Resolved by delete all files under /tmp   hdfs://namenode:/tmp/hive
delete from mysql-->hive-->funcs

recreate all the function & it work

On Thu, Apr 30, 2015 at 3:36 PM, Gerald-G <sh...@gmail.com> wrote:

> HI
>  My hive version is 0.14.0 installed from HDP2.2.4
>
> On Thu, Apr 30, 2015 at 3:34 PM, Gerald-G <sh...@gmail.com> wrote:
>
>> Hi ALL:
>>
>>   I have develop three UDF and compile them in one jar.  Hive Explainn
>> one udf to antother class
>>
>> Dump INFO as Follow:  Hive explain userlost-->shiftAct(), but the return
>> type is boolean, the right return type I want
>>
>>
>> 0: jdbc:hive2://10-4-32-53:10000> explain select userlost(idayacti,1)
>> from fat_tbaccount;
>> +--------------------------------------------------------------+--+
>> |                           Explain                            |
>> +--------------------------------------------------------------+--+
>> | STAGE DEPENDENCIES:                                          |
>> |   Stage-0 is a root stage                                    |
>> |                                                              |
>> | STAGE PLANS:                                                 |
>> |   Stage: Stage-0                                             |
>> |     Fetch Operator                                           |
>> |       limit: -1                                              |
>> |       Processor Tree:                                        |
>> |         TableScan                                            |
>> |           alias: fat_tbaccount                               |
>> |           Select Operator                                    |
>> |             expressions: ShiftAct(idayacti) (type: boolean)  |
>> |             outputColumnNames: _col0                         |
>> |             ListSink                                         |
>> |                                                              |
>> +--------------------------------------------------------------+--+
>> 15 rows selected (0.657 seconds)
>> 0: jdbc:hive2://10-4-32-53:10000> explain select
>> userlost(idayacti,iworldid) from fat_tbaccount;
>> Error: Error while compiling statement: FAILED: SemanticException [Error
>> 10004]: Line 1:33 Invalid table alias or column reference 'iworldid':
>> (possible column names are: dtstatdate, iaccounttype, suin, igameid,
>> iworld, iroleid, iregtime, ilastacttime, idayacti, iweekacti, imonthacti,
>> igroup, ilevel, iviplevel, itimes, ionlinetime) (state=42000,code=10004)
>> 0: jdbc:hive2://10-4-32-53:10000> explain select
>> userlost(idayacti,iworld) from fat_tbaccount;
>> +--------------------------------------------------------------+--+
>> |                           Explain                            |
>> +--------------------------------------------------------------+--+
>> | STAGE DEPENDENCIES:                                          |
>> |   Stage-0 is a root stage                                    |
>> |                                                              |
>> | STAGE PLANS:                                                 |
>> |   Stage: Stage-0                                             |
>> |     Fetch Operator                                           |
>> |       limit: -1                                              |
>> |       Processor Tree:                                        |
>> |         TableScan                                            |
>> |           alias: fat_tbaccount                               |
>> |           Select Operator                                    |
>> |             expressions: ShiftAct(idayacti) (type: boolean)  |
>> |             outputColumnNames: _col0                         |
>> |             ListSink                                         |
>> |                                                              |
>> +--------------------------------------------------------------+--+
>> 15 rows selected (0.192 seconds)
>> 0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftleft AS
>> 'com.shadowinlife.app.UserACT.ShiftLeft';
>> No rows affected (0.064 seconds)
>> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftleft(idayacti) from
>> fat_tbaccount;
>>
>> +----------------------------------------------------------------------------------------+--+
>> |                                        Explain
>>                 |
>>
>> +----------------------------------------------------------------------------------------+--+
>> | STAGE DEPENDENCIES:
>>                |
>> |   Stage-0 is a root stage
>>                |
>> |
>>                |
>> | STAGE PLANS:
>>                 |
>> |   Stage: Stage-0
>>                 |
>> |     Fetch Operator
>>                 |
>> |       limit: -1
>>                |
>> |       Processor Tree:
>>                |
>> |         TableScan
>>                |
>> |           alias: fat_tbaccount
>>                 |
>> |           Select Operator
>>                |
>> |             expressions: ShiftLeft(idayacti) (type:
>> struct<header:int,tailer:bigint>)  |
>> |             outputColumnNames: _col0
>>                 |
>> |             ListSink
>>                 |
>> |
>>                |
>>
>> +----------------------------------------------------------------------------------------+--+
>> 15 rows selected (0.217 seconds)
>> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
>> fat_tbaccount;
>> Error: Error while compiling statement: FAILED: SemanticException [Error
>> 10011]: Line 1:15 Invalid function 'shiftact' (state=42000,code=10011)
>> 0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftact AS
>> 'com.shadowinlife.app.UserACT.ShiftAct';
>> No rows affected (0.064 seconds)
>> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
>> fat_tbaccount;
>>
>> +---------------------------------------------------------------------------------------+--+
>> |                                        Explain
>>                |
>>
>> +---------------------------------------------------------------------------------------+--+
>> | STAGE DEPENDENCIES:
>>               |
>> |   Stage-0 is a root stage
>>               |
>> |
>>               |
>> | STAGE PLANS:
>>                |
>> |   Stage: Stage-0
>>                |
>> |     Fetch Operator
>>                |
>> |       limit: -1
>>               |
>> |       Processor Tree:
>>               |
>> |         TableScan
>>               |
>> |           alias: fat_tbaccount
>>                |
>> |           Select Operator
>>               |
>> |             expressions: ShiftAct(idayacti) (type:
>> struct<header:int,tailer:bigint>)  |
>> |             outputColumnNames: _col0
>>                |
>> |             ListSink
>>                |
>> |
>>               |
>>
>> +---------------------------------------------------------------------------------------+--+
>>
>>
>

Re: HIVE udf explain error

Posted by Gerald-G <sh...@gmail.com>.
HI
 My hive version is 0.14.0 installed from HDP2.2.4

On Thu, Apr 30, 2015 at 3:34 PM, Gerald-G <sh...@gmail.com> wrote:

> Hi ALL:
>
>   I have develop three UDF and compile them in one jar.  Hive Explainn one
> udf to antother class
>
> Dump INFO as Follow:  Hive explain userlost-->shiftAct(), but the return
> type is boolean, the right return type I want
>
>
> 0: jdbc:hive2://10-4-32-53:10000> explain select userlost(idayacti,1) from
> fat_tbaccount;
> +--------------------------------------------------------------+--+
> |                           Explain                            |
> +--------------------------------------------------------------+--+
> | STAGE DEPENDENCIES:                                          |
> |   Stage-0 is a root stage                                    |
> |                                                              |
> | STAGE PLANS:                                                 |
> |   Stage: Stage-0                                             |
> |     Fetch Operator                                           |
> |       limit: -1                                              |
> |       Processor Tree:                                        |
> |         TableScan                                            |
> |           alias: fat_tbaccount                               |
> |           Select Operator                                    |
> |             expressions: ShiftAct(idayacti) (type: boolean)  |
> |             outputColumnNames: _col0                         |
> |             ListSink                                         |
> |                                                              |
> +--------------------------------------------------------------+--+
> 15 rows selected (0.657 seconds)
> 0: jdbc:hive2://10-4-32-53:10000> explain select
> userlost(idayacti,iworldid) from fat_tbaccount;
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10004]: Line 1:33 Invalid table alias or column reference 'iworldid':
> (possible column names are: dtstatdate, iaccounttype, suin, igameid,
> iworld, iroleid, iregtime, ilastacttime, idayacti, iweekacti, imonthacti,
> igroup, ilevel, iviplevel, itimes, ionlinetime) (state=42000,code=10004)
> 0: jdbc:hive2://10-4-32-53:10000> explain select userlost(idayacti,iworld)
> from fat_tbaccount;
> +--------------------------------------------------------------+--+
> |                           Explain                            |
> +--------------------------------------------------------------+--+
> | STAGE DEPENDENCIES:                                          |
> |   Stage-0 is a root stage                                    |
> |                                                              |
> | STAGE PLANS:                                                 |
> |   Stage: Stage-0                                             |
> |     Fetch Operator                                           |
> |       limit: -1                                              |
> |       Processor Tree:                                        |
> |         TableScan                                            |
> |           alias: fat_tbaccount                               |
> |           Select Operator                                    |
> |             expressions: ShiftAct(idayacti) (type: boolean)  |
> |             outputColumnNames: _col0                         |
> |             ListSink                                         |
> |                                                              |
> +--------------------------------------------------------------+--+
> 15 rows selected (0.192 seconds)
> 0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftleft AS
> 'com.shadowinlife.app.UserACT.ShiftLeft';
> No rows affected (0.064 seconds)
> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftleft(idayacti) from
> fat_tbaccount;
>
> +----------------------------------------------------------------------------------------+--+
> |                                        Explain
>               |
>
> +----------------------------------------------------------------------------------------+--+
> | STAGE DEPENDENCIES:
>                |
> |   Stage-0 is a root stage
>                |
> |
>                |
> | STAGE PLANS:
>               |
> |   Stage: Stage-0
>               |
> |     Fetch Operator
>               |
> |       limit: -1
>                |
> |       Processor Tree:
>                |
> |         TableScan
>                |
> |           alias: fat_tbaccount
>               |
> |           Select Operator
>                |
> |             expressions: ShiftLeft(idayacti) (type:
> struct<header:int,tailer:bigint>)  |
> |             outputColumnNames: _col0
>               |
> |             ListSink
>               |
> |
>                |
>
> +----------------------------------------------------------------------------------------+--+
> 15 rows selected (0.217 seconds)
> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
> fat_tbaccount;
> Error: Error while compiling statement: FAILED: SemanticException [Error
> 10011]: Line 1:15 Invalid function 'shiftact' (state=42000,code=10011)
> 0: jdbc:hive2://10-4-32-53:10000> CREATE FUNCTION shiftact AS
> 'com.shadowinlife.app.UserACT.ShiftAct';
> No rows affected (0.064 seconds)
> 0: jdbc:hive2://10-4-32-53:10000> explain select shiftact(idayacti) from
> fat_tbaccount;
>
> +---------------------------------------------------------------------------------------+--+
> |                                        Explain
>              |
>
> +---------------------------------------------------------------------------------------+--+
> | STAGE DEPENDENCIES:
>               |
> |   Stage-0 is a root stage
>               |
> |
>               |
> | STAGE PLANS:
>              |
> |   Stage: Stage-0
>              |
> |     Fetch Operator
>              |
> |       limit: -1
>               |
> |       Processor Tree:
>               |
> |         TableScan
>               |
> |           alias: fat_tbaccount
>              |
> |           Select Operator
>               |
> |             expressions: ShiftAct(idayacti) (type:
> struct<header:int,tailer:bigint>)  |
> |             outputColumnNames: _col0
>              |
> |             ListSink
>              |
> |
>               |
>
> +---------------------------------------------------------------------------------------+--+
>
>