[jira] [Created] (HIVEMALL-302) Is there any way to let the function accept string value instead of const string value

["Bob (Jira)" <ji...@apache.org>]

Bob created HIVEMALL-302:
----------------------------

             Summary: Is there any way to let the function accept string value instead of const string value
                 Key: HIVEMALL-302
                 URL: https://issues.apache.org/jira/browse/HIVEMALL-302
             Project: Hivemall
          Issue Type: Wish
    Affects Versions: 0.6.0
            Reporter: Bob


 

I am using hivemall in Aliyun MaxCompute environment and get below error message once I was running the SQL from [http://hivemall.incubator.apache.org/userguide/recommend/movielens_cf.html]

 
{code:java}
org.apache.hadoop.hive.ql.exec.UDFArgumentException: argument must be a constant value: string
        at hivemall.utils.hadoop.HiveUtils.getConstValue(HiveUtils.java:587)
        at hivemall.utils.hadoop.HiveUtils.getConstString(HiveUtils.java:661)
        at hivemall.knn.similarity.DIMSUMMapperUDTF.processOptions(DIMSUMMapperUDTF.java:100)
        at hivemall.knn.similarity.DIMSUMMapperUDTF.initialize(DIMSUMMapperUDTF.java:135)
        at org.apache.hadoop.hive.ql.udf.generic.GenericUDTF.initialize(GenericUDTF.java:56)
        at com.aliyun.odps.compiler.hive.adapter.Adapter.resolveGenericUdtf(Adapter.java:92)
        at com.aliyun.odps.compiler.hive.adapter.Adapter.invokeResolve(Adapter.java:38)
        at com.aliyun.odps.compiler.function.udf.hive.java.HiveJavaUdfResolveSandbox.invokeResolve(HiveJavaUdfResolveSandbox.java:20)
        at com.aliyun.odps.compiler.function.udf.JavaUdfResolveSandbox$MainClass.main(JavaUdfResolveSandbox.java:237)
{code}
 

Related SQL statements are:

 
{code:java}
with movie_magnitude as ( -- compute magnitude of each movie vector
  select
    to_map(j, mag) as mags,
    1 as con
  from (
    select 
      movieid as j,
      l2_norm(rating) as mag
    from 
      training
    group by
      movieid
  ) t0
),
movie_features as (
  select
    userid as i,
    1 as con,
    collect_list(
      feature(movieid, rating)
    ) as feature_vector
  from
    training
  group by
    userid
)
select
dimsum_mapper(f.feature_vector, m.mags, "-disable_symmetric_output -threshold 0.1")
    as (movieid, other, s)
from
movie_features f
left outer join movie_magnitude m on f.con = m.con
;
{code}
 

 

As I have use the same SQL statement in pure Hive environment and it works fine, so I have to say this is not the problem of hivemall. But I still wonder if there is some way to disable the validation of const value or force the hive to pass the const value. Otherwise I have to update the HiveUtils.java to handle this, and I do not think this should be the suitable way.

Thanks a lot for your time and please kindly let me know if any configs with hive or hivemall could make this happen. I have found the same error under [HIVEMALL-255|https://issues.apache.org/jira/browse/HIVEMALL-255] but it is closed for no following responses.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)