You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hive.apache.org by "Stephen R. Scaffidi" <ss...@tripadvisor.com> on 2012/08/06 23:15:13 UTC
Possible patch to fix column comments with non-native SerDe
My team and I have been trying, with limited success, to use the COMMENT
feature of hive columns to maintain documentation for the tables and
columns in our data-warehouse built on hive. However, we use a number of
custom and non-native SerDes, and what happens to those tables is that
the comments always get overwritten with the string "from deserializer".
I've possibly found a way to work around this from within hive but I
want to get some insight from the hive-dev community to figure out
whether or not this is a patently bad idea and we are just setting
ourselves up for pain later on.
I won't go into all the details but it seems to work in our (so far)
limited testing. However, we are using hive 0.7.1 and the patch I am
sending is against master/HEAD.
Please let me know if this is an acceptable approach to preserving
column comments with non-native SerDes or not!
Re: Possible patch to fix column comments with non-native SerDe
Posted by Jakob Homan <jg...@gmail.com>.
You'll need to update your serde to use the method call that takes comments. See https://github.com/jghoman/haivvreo/commit/29ead1fe101baafa8e9844eaf92022cbe4846c6f for an example.
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
On Tuesday, August 7, 2012 at 7:21 AM, Stephen R. Scaffidi wrote:
> So, the patch doesn not seem to fix the problem we are having, but it
> combined with the one I sent to the list seems to take care of it. I
> will continue to study this issue and report back on any issues.
>
> Thanks again!
>
> On 08/06/2012 06:59 PM, Stephen Scaffid wrote:
> > Thanks! I'll see how it goes!
> >
> > (better yet, this could be what it takes to convince the team to upgrade!)
> >
> > On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:
> >
> > > This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171). Can you just apply that patch?
> > >
> > > On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <sscaffidi@tripadvisor.com (mailto:sscaffidi@tripadvisor.com)> wrote:
> > > My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
> > >
> > > I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
> > >
> > > I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
> > >
> > > Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!
Re: Possible patch to fix column comments with non-native SerDe
Posted by "Stephen R. Scaffidi" <ss...@tripadvisor.com>.
So, the patch doesn not seem to fix the problem we are having, but it
combined with the one I sent to the list seems to take care of it. I
will continue to study this issue and report back on any issues.
Thanks again!
On 08/06/2012 06:59 PM, Stephen Scaffid wrote:
> Thanks! I'll see how it goes!
>
> (better yet, this could be what it takes to convince the team to upgrade!)
>
> On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:
>
>> This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171). Can you just apply that patch?
>>
>> On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <ss...@tripadvisor.com> wrote:
>> My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
>>
>> I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
>>
>> I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
>>
>> Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!
>>
>>
>>
>
>
Re: Possible patch to fix column comments with non-native SerDe
Posted by Stephen Scaffid <ss...@tripadvisor.com>.
Thanks! I'll see how it goes!
(better yet, this could be what it takes to convince the team to upgrade!)
On Aug 6, 2012, at 6:47 PM, Jakob Homan wrote:
> This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171). Can you just apply that patch?
>
> On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <ss...@tripadvisor.com> wrote:
> My team and I have been trying, with limited success, to use the COMMENT feature of hive columns to maintain documentation for the tables and columns in our data-warehouse built on hive. However, we use a number of custom and non-native SerDes, and what happens to those tables is that the comments always get overwritten with the string "from deserializer".
>
> I've possibly found a way to work around this from within hive but I want to get some insight from the hive-dev community to figure out whether or not this is a patently bad idea and we are just setting ourselves up for pain later on.
>
> I won't go into all the details but it seems to work in our (so far) limited testing. However, we are using hive 0.7.1 and the patch I am sending is against master/HEAD.
>
> Please let me know if this is an acceptable approach to preserving column comments with non-native SerDes or not!
>
>
>
Re: Possible patch to fix column comments with non-native SerDe
Posted by Jakob Homan <jg...@gmail.com>.
This was fixed in Hive 8 (https://issues.apache.org/jira/browse/HIVE-2171).
Can you just apply that patch?
On Mon, Aug 6, 2012 at 2:15 PM, Stephen R. Scaffidi <
sscaffidi@tripadvisor.com> wrote:
> My team and I have been trying, with limited success, to use the COMMENT
> feature of hive columns to maintain documentation for the tables and
> columns in our data-warehouse built on hive. However, we use a number of
> custom and non-native SerDes, and what happens to those tables is that the
> comments always get overwritten with the string "from deserializer".
>
> I've possibly found a way to work around this from within hive but I want
> to get some insight from the hive-dev community to figure out whether or
> not this is a patently bad idea and we are just setting ourselves up for
> pain later on.
>
> I won't go into all the details but it seems to work in our (so far)
> limited testing. However, we are using hive 0.7.1 and the patch I am
> sending is against master/HEAD.
>
> Please let me know if this is an acceptable approach to preserving column
> comments with non-native SerDes or not!
>
>
>