You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hive.apache.org by Ravi Shetye <ra...@gmail.com> on 2012/09/04 16:42:21 UTC
Hive UDF intialization
Hi
I am trying to register a java udf which looks like
public final class IP_2_GEO extends UDF {
String geo_file;
String geo_type;
public IP_2_GEO(String geo_file, String geo_type) {
this.geo_file = geo_file;
this.geo_type = geo_type;
}
public Text evaluate(final Text ip) {
//IP to geo conversion based on the level passed in 'geo_type' and by look
up in 'geo_file'
return new Text(geoString);
}
}
I had similar udf in pig and could register it by using
define ip_2_city com.udfs.common.IP_2_GEO('$geo_file', 'city');
define ip_2_country com.udfs.common.IP_2_GEO('$geo_file', 'country');
but in hive I am not able to register and initialize the udf
hive> create temporary function ip_2_city as
"udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')";
FAILED: Class udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city') not
found
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.FunctionTask
hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO()";
FAILED: Class udfs.common.IP_2_GEO() not found
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.FunctionTask
hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO";
OK
Time taken: 0.0080 seconds
1) Is there an alternate way to achieve what I expect from the command
hive> create temporary function ip_2_city as
"udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')"; ?
2) What is the common practice of converting ip to City and Country in hive?
--
RAVI SHETYE
Re: Hive UDF intialization
Posted by Ruslan Al-Fakikh <ru...@jalent.ru>.
Ravi,
It looks like you are missing the
ADD JAR ...
command
Ruslan
On Tue, Sep 4, 2012 at 6:45 PM, Edward Capriolo <ed...@gmail.com> wrote:
> You could start with this:
>
> https://github.com/edwardcapriolo/hive-geoip
>
> On Tue, Sep 4, 2012 at 10:42 AM, Ravi Shetye <ra...@gmail.com> wrote:
>> Hi
>> I am trying to register a java udf which looks like
>>
>> public final class IP_2_GEO extends UDF {
>> String geo_file;
>> String geo_type;
>> public IP_2_GEO(String geo_file, String geo_type) {
>> this.geo_file = geo_file;
>> this.geo_type = geo_type;
>> }
>>
>> public Text evaluate(final Text ip) {
>> //IP to geo conversion based on the level passed in 'geo_type' and by look
>> up in 'geo_file'
>>
>> return new Text(geoString);
>> }
>> }
>>
>> I had similar udf in pig and could register it by using
>> define ip_2_city com.udfs.common.IP_2_GEO('$geo_file', 'city');
>> define ip_2_country com.udfs.common.IP_2_GEO('$geo_file', 'country');
>>
>> but in hive I am not able to register and initialize the udf
>>
>> hive> create temporary function ip_2_city as
>> "udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')";
>> FAILED: Class udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city') not
>> found
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.FunctionTask
>> hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO()";
>> FAILED: Class udfs.common.IP_2_GEO() not found
>> FAILED: Execution Error, return code 1 from
>> org.apache.hadoop.hive.ql.exec.FunctionTask
>> hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO";
>> OK
>> Time taken: 0.0080 seconds
>>
>>
>> 1) Is there an alternate way to achieve what I expect from the command
>>
>> hive> create temporary function ip_2_city as
>> "udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')"; ?
>>
>> 2) What is the common practice of converting ip to City and Country in hive?
>> --
>> RAVI SHETYE
--
Best Regards,
Ruslan Al-Fakikh
Re: Hive UDF intialization
Posted by Edward Capriolo <ed...@gmail.com>.
You could start with this:
https://github.com/edwardcapriolo/hive-geoip
On Tue, Sep 4, 2012 at 10:42 AM, Ravi Shetye <ra...@gmail.com> wrote:
> Hi
> I am trying to register a java udf which looks like
>
> public final class IP_2_GEO extends UDF {
> String geo_file;
> String geo_type;
> public IP_2_GEO(String geo_file, String geo_type) {
> this.geo_file = geo_file;
> this.geo_type = geo_type;
> }
>
> public Text evaluate(final Text ip) {
> //IP to geo conversion based on the level passed in 'geo_type' and by look
> up in 'geo_file'
>
> return new Text(geoString);
> }
> }
>
> I had similar udf in pig and could register it by using
> define ip_2_city com.udfs.common.IP_2_GEO('$geo_file', 'city');
> define ip_2_country com.udfs.common.IP_2_GEO('$geo_file', 'country');
>
> but in hive I am not able to register and initialize the udf
>
> hive> create temporary function ip_2_city as
> "udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')";
> FAILED: Class udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city') not
> found
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.FunctionTask
> hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO()";
> FAILED: Class udfs.common.IP_2_GEO() not found
> FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.FunctionTask
> hive> create temporary function ip_2_city as "udfs.common.IP_2_GEO";
> OK
> Time taken: 0.0080 seconds
>
>
> 1) Is there an alternate way to achieve what I expect from the command
>
> hive> create temporary function ip_2_city as
> "udfs.common.IP_2_GEO('/mnt/ravi/GeoLiteCity.dat','city')"; ?
>
> 2) What is the common practice of converting ip to City and Country in hive?
> --
> RAVI SHETYE