You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by Stephen Sprague <sp...@gmail.com> on 2012/05/29 17:46:39 UTC
perl code generated from thrift
Hi Good People,
I am totally stumped on this one. Wondered if a "perl" bindings chap on
this list might be able to help me.
I generated the ThrftHive.pm class as follows (see below) from my download
area. I've tested some basic examples of querying - and it works well.
The one thing that i can't get to work, though, is a call to
"$client->getSchema()" in order to ascertain the schema of a result set.
* this built ThriftHIve.pm
$ thrift -r \
-I $HOME/downloads/hive-0.9.0/src/ql \
-I $HOME/downloads/hive-0.9.0/src \
-I $HOME/downloads/thrift-0.8.0 \
\
-gen perl \
\
$HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
* in hive_service.thrift here are the relevant lines (that i can tell)
#the includes
include "contrib/fb303/if/fb303.thrift"
include "metastore/if/hive_metastore.thrift"
include "ql/if/queryplan.thrift"
namespace java org.apache.hadoop.hive.service
namespace cpp Apache.Hadoop.Hive
<snip>
#the getSchema() reference
service ThriftHive extends hive_metastore.ThriftHiveMetastore {
<snip>
# Get a schema object with fields represented with native Hive types
hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
* the relevant line in ThriftHive.pm that throws an undefined method call:
Undefined subroutine &ThriftHive_getSchema_result::Schema called at
/home/spragues/downloads/thrift-0.8.0/gen-perl/ThriftHive.pm line 672.
$self->{success} = new Schema();
$xfer += $self->{success}->read($input);
Where 'o where is this "Schema" class supposed to be? Any pointers, clues,
suggestions, bread crumbs most welcome!
Thanks
Stephen
Re: perl code generated from thrift
Posted by Stephen Sprague <sp...@gmail.com>.
Thanks Ed. I'll be glad to bark up another tree - just want to make sure
its the most relevant tree! :) In my case i'm not sure if its a hive or
thrift domain issue i have.
Ultimately though like you suggest if "thrift -gen perl" and "thrift -gen
py" _given the same input_ don't quite generate the analogous code it sure
wouldn't hurt to ask 'em. :)
Thanks again for your help on this - much appreciated.
Cheers,
Stephen
On Tue, May 29, 2012 at 1:40 PM, Edward Capriolo <ed...@gmail.com>wrote:
> I never tried to this exactly but are you sure your thrift version
> matches what hive is using? If not I would consider talking to the
> #thrift guys because thrift SHOULD_BE generating bindings for every
> language roughly the same. (How is that for passing the buck?)
>
> Edward
>
> On Tue, May 29, 2012 at 1:34 PM, Stephen Sprague <sp...@gmail.com>
> wrote:
> > thanks Ed. Something just isn't quite right.
> >
> > in .$HOME/downloads/hive-0.9.0/src/metastore/if/hive_metastore.thrift
> > there is a 'struct Schema' entry but all attempts to generate the perl
> > code for that seem to come up empty.
> >
> > looking at -gen py and the associated code i see this code frag in
> > ThriftHive.py
> >
> > if ftype == TType.STRUCT:
> > self.success = hive_metastore.ttypes.Schema()
> > self.success.read(iprot)
> > else:
> >
> > for perl i only have this:
> >
> > if ($ftype == TType::STRUCT) {
> > $self->{success} = new Schema();
> > $xfer += $self->{success}->read($input);
> >
> > and hence "new Schema" throws the exception of not found.
> >
> >
> > comparing to the generated python code -- it has a whole big ttype.py
> under
> > hive_metastore while the generated perl code has zilch. the perl stuff
> only
> > has ThriftHiveMetastore.pm but no subclasses. :(
> >
> > thoughts, ideas, ruminations?
> >
> > Thanks again Ed.
> >
> > Cheers,
> > Stephen
> >
> > On Tue, May 29, 2012 at 9:00 AM, Edward Capriolo <edlinuxguru@gmail.com
> >wrote:
> >
> >> Hive uses thrift in two places 1) HiveServer 2)Metastore . Search in
> >> the metastore directory for the thrift interface file and compile
> >> that.
> >>
> >> Then you should get a class generated for hive_metastore.Schema
> >>
> >> Edward
> >>
> >>
> >>
> >> On Tue, May 29, 2012 at 11:46 AM, Stephen Sprague <sp...@gmail.com>
> >> wrote:
> >> > Hi Good People,
> >> > I am totally stumped on this one. Wondered if a "perl" bindings chap
> on
> >> > this list might be able to help me.
> >> >
> >> > I generated the ThrftHive.pm class as follows (see below) from my
> >> download
> >> > area. I've tested some basic examples of querying - and it works
> well.
> >> > The one thing that i can't get to work, though, is a call to
> >> > "$client->getSchema()" in order to ascertain the schema of a result
> set.
> >> >
> >> > * this built ThriftHIve.pm
> >> >
> >> > $ thrift -r \
> >> > -I $HOME/downloads/hive-0.9.0/src/ql \
> >> > -I $HOME/downloads/hive-0.9.0/src \
> >> > -I $HOME/downloads/thrift-0.8.0 \
> >> > \
> >> > -gen perl \
> >> > \
> >> > $HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
> >> >
> >> >
> >> >
> >> >
> >> > * in hive_service.thrift here are the relevant lines (that i can tell)
> >> >
> >> > #the includes
> >> > include "contrib/fb303/if/fb303.thrift"
> >> > include "metastore/if/hive_metastore.thrift"
> >> > include "ql/if/queryplan.thrift"
> >> > namespace java org.apache.hadoop.hive.service
> >> > namespace cpp Apache.Hadoop.Hive
> >> >
> >> > <snip>
> >> > #the getSchema() reference
> >> > service ThriftHive extends hive_metastore.ThriftHiveMetastore {
> >> > <snip>
> >> > # Get a schema object with fields represented with native Hive
> types
> >> > hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
> >> >
> >> >
> >> > * the relevant line in ThriftHive.pm that throws an undefined method
> >> call:
> >> >
> >> > Undefined subroutine &ThriftHive_getSchema_result::Schema called at
> >> > /home/spragues/downloads/thrift-0.8.0/gen-perl/ThriftHive.pm line 672.
> >> >
> >> > $self->{success} = new Schema();
> >> > $xfer += $self->{success}->read($input);
> >> >
> >> >
> >> > Where 'o where is this "Schema" class supposed to be? Any pointers,
> >> clues,
> >> > suggestions, bread crumbs most welcome!
> >> >
> >> > Thanks
> >> > Stephen
> >>
>
Re: perl code generated from thrift
Posted by Edward Capriolo <ed...@gmail.com>.
I never tried to this exactly but are you sure your thrift version
matches what hive is using? If not I would consider talking to the
#thrift guys because thrift SHOULD_BE generating bindings for every
language roughly the same. (How is that for passing the buck?)
Edward
On Tue, May 29, 2012 at 1:34 PM, Stephen Sprague <sp...@gmail.com> wrote:
> thanks Ed. Something just isn't quite right.
>
> in .$HOME/downloads/hive-0.9.0/src/metastore/if/hive_metastore.thrift
> there is a 'struct Schema' entry but all attempts to generate the perl
> code for that seem to come up empty.
>
> looking at -gen py and the associated code i see this code frag in
> ThriftHive.py
>
> if ftype == TType.STRUCT:
> self.success = hive_metastore.ttypes.Schema()
> self.success.read(iprot)
> else:
>
> for perl i only have this:
>
> if ($ftype == TType::STRUCT) {
> $self->{success} = new Schema();
> $xfer += $self->{success}->read($input);
>
> and hence "new Schema" throws the exception of not found.
>
>
> comparing to the generated python code -- it has a whole big ttype.py under
> hive_metastore while the generated perl code has zilch. the perl stuff only
> has ThriftHiveMetastore.pm but no subclasses. :(
>
> thoughts, ideas, ruminations?
>
> Thanks again Ed.
>
> Cheers,
> Stephen
>
> On Tue, May 29, 2012 at 9:00 AM, Edward Capriolo <ed...@gmail.com>wrote:
>
>> Hive uses thrift in two places 1) HiveServer 2)Metastore . Search in
>> the metastore directory for the thrift interface file and compile
>> that.
>>
>> Then you should get a class generated for hive_metastore.Schema
>>
>> Edward
>>
>>
>>
>> On Tue, May 29, 2012 at 11:46 AM, Stephen Sprague <sp...@gmail.com>
>> wrote:
>> > Hi Good People,
>> > I am totally stumped on this one. Wondered if a "perl" bindings chap on
>> > this list might be able to help me.
>> >
>> > I generated the ThrftHive.pm class as follows (see below) from my
>> download
>> > area. I've tested some basic examples of querying - and it works well.
>> > The one thing that i can't get to work, though, is a call to
>> > "$client->getSchema()" in order to ascertain the schema of a result set.
>> >
>> > * this built ThriftHIve.pm
>> >
>> > $ thrift -r \
>> > -I $HOME/downloads/hive-0.9.0/src/ql \
>> > -I $HOME/downloads/hive-0.9.0/src \
>> > -I $HOME/downloads/thrift-0.8.0 \
>> > \
>> > -gen perl \
>> > \
>> > $HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
>> >
>> >
>> >
>> >
>> > * in hive_service.thrift here are the relevant lines (that i can tell)
>> >
>> > #the includes
>> > include "contrib/fb303/if/fb303.thrift"
>> > include "metastore/if/hive_metastore.thrift"
>> > include "ql/if/queryplan.thrift"
>> > namespace java org.apache.hadoop.hive.service
>> > namespace cpp Apache.Hadoop.Hive
>> >
>> > <snip>
>> > #the getSchema() reference
>> > service ThriftHive extends hive_metastore.ThriftHiveMetastore {
>> > <snip>
>> > # Get a schema object with fields represented with native Hive types
>> > hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
>> >
>> >
>> > * the relevant line in ThriftHive.pm that throws an undefined method
>> call:
>> >
>> > Undefined subroutine &ThriftHive_getSchema_result::Schema called at
>> > /home/spragues/downloads/thrift-0.8.0/gen-perl/ThriftHive.pm line 672.
>> >
>> > $self->{success} = new Schema();
>> > $xfer += $self->{success}->read($input);
>> >
>> >
>> > Where 'o where is this "Schema" class supposed to be? Any pointers,
>> clues,
>> > suggestions, bread crumbs most welcome!
>> >
>> > Thanks
>> > Stephen
>>
Re: perl code generated from thrift
Posted by Stephen Sprague <sp...@gmail.com>.
thanks Ed. Something just isn't quite right.
in .$HOME/downloads/hive-0.9.0/src/metastore/if/hive_metastore.thrift
there is a 'struct Schema' entry but all attempts to generate the perl
code for that seem to come up empty.
looking at -gen py and the associated code i see this code frag in
ThriftHive.py
if ftype == TType.STRUCT:
self.success = hive_metastore.ttypes.Schema()
self.success.read(iprot)
else:
for perl i only have this:
if ($ftype == TType::STRUCT) {
$self->{success} = new Schema();
$xfer += $self->{success}->read($input);
and hence "new Schema" throws the exception of not found.
comparing to the generated python code -- it has a whole big ttype.py under
hive_metastore while the generated perl code has zilch. the perl stuff only
has ThriftHiveMetastore.pm but no subclasses. :(
thoughts, ideas, ruminations?
Thanks again Ed.
Cheers,
Stephen
On Tue, May 29, 2012 at 9:00 AM, Edward Capriolo <ed...@gmail.com>wrote:
> Hive uses thrift in two places 1) HiveServer 2)Metastore . Search in
> the metastore directory for the thrift interface file and compile
> that.
>
> Then you should get a class generated for hive_metastore.Schema
>
> Edward
>
>
>
> On Tue, May 29, 2012 at 11:46 AM, Stephen Sprague <sp...@gmail.com>
> wrote:
> > Hi Good People,
> > I am totally stumped on this one. Wondered if a "perl" bindings chap on
> > this list might be able to help me.
> >
> > I generated the ThrftHive.pm class as follows (see below) from my
> download
> > area. I've tested some basic examples of querying - and it works well.
> > The one thing that i can't get to work, though, is a call to
> > "$client->getSchema()" in order to ascertain the schema of a result set.
> >
> > * this built ThriftHIve.pm
> >
> > $ thrift -r \
> > -I $HOME/downloads/hive-0.9.0/src/ql \
> > -I $HOME/downloads/hive-0.9.0/src \
> > -I $HOME/downloads/thrift-0.8.0 \
> > \
> > -gen perl \
> > \
> > $HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
> >
> >
> >
> >
> > * in hive_service.thrift here are the relevant lines (that i can tell)
> >
> > #the includes
> > include "contrib/fb303/if/fb303.thrift"
> > include "metastore/if/hive_metastore.thrift"
> > include "ql/if/queryplan.thrift"
> > namespace java org.apache.hadoop.hive.service
> > namespace cpp Apache.Hadoop.Hive
> >
> > <snip>
> > #the getSchema() reference
> > service ThriftHive extends hive_metastore.ThriftHiveMetastore {
> > <snip>
> > # Get a schema object with fields represented with native Hive types
> > hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
> >
> >
> > * the relevant line in ThriftHive.pm that throws an undefined method
> call:
> >
> > Undefined subroutine &ThriftHive_getSchema_result::Schema called at
> > /home/spragues/downloads/thrift-0.8.0/gen-perl/ThriftHive.pm line 672.
> >
> > $self->{success} = new Schema();
> > $xfer += $self->{success}->read($input);
> >
> >
> > Where 'o where is this "Schema" class supposed to be? Any pointers,
> clues,
> > suggestions, bread crumbs most welcome!
> >
> > Thanks
> > Stephen
>
Re: perl code generated from thrift
Posted by Edward Capriolo <ed...@gmail.com>.
Hive uses thrift in two places 1) HiveServer 2)Metastore . Search in
the metastore directory for the thrift interface file and compile
that.
Then you should get a class generated for hive_metastore.Schema
Edward
On Tue, May 29, 2012 at 11:46 AM, Stephen Sprague <sp...@gmail.com> wrote:
> Hi Good People,
> I am totally stumped on this one. Wondered if a "perl" bindings chap on
> this list might be able to help me.
>
> I generated the ThrftHive.pm class as follows (see below) from my download
> area. I've tested some basic examples of querying - and it works well.
> The one thing that i can't get to work, though, is a call to
> "$client->getSchema()" in order to ascertain the schema of a result set.
>
> * this built ThriftHIve.pm
>
> $ thrift -r \
> -I $HOME/downloads/hive-0.9.0/src/ql \
> -I $HOME/downloads/hive-0.9.0/src \
> -I $HOME/downloads/thrift-0.8.0 \
> \
> -gen perl \
> \
> $HOME/downloads/hive-0.9.0/src/service/if/hive_service.thrift
>
>
>
>
> * in hive_service.thrift here are the relevant lines (that i can tell)
>
> #the includes
> include "contrib/fb303/if/fb303.thrift"
> include "metastore/if/hive_metastore.thrift"
> include "ql/if/queryplan.thrift"
> namespace java org.apache.hadoop.hive.service
> namespace cpp Apache.Hadoop.Hive
>
> <snip>
> #the getSchema() reference
> service ThriftHive extends hive_metastore.ThriftHiveMetastore {
> <snip>
> # Get a schema object with fields represented with native Hive types
> hive_metastore.Schema getSchema() throws(1:HiveServerException ex)
>
>
> * the relevant line in ThriftHive.pm that throws an undefined method call:
>
> Undefined subroutine &ThriftHive_getSchema_result::Schema called at
> /home/spragues/downloads/thrift-0.8.0/gen-perl/ThriftHive.pm line 672.
>
> $self->{success} = new Schema();
> $xfer += $self->{success}->read($input);
>
>
> Where 'o where is this "Schema" class supposed to be? Any pointers, clues,
> suggestions, bread crumbs most welcome!
>
> Thanks
> Stephen