You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@hbase.apache.org by lars hofhansl <lh...@yahoo.com> on 2012/11/27 19:54:49 UTC
State of protobufs in hbase-0.96
HBASE-7215 has a rant about the current state of the protobuf work in 0.96... Continuing here...
HBASE-7215 came about because I simply wanted to add a field to Mutation (Put, Delete, etc) for HBASE-5954.
Should be easy now with protobufs, right? Nope!
- Both Put and Delete (and Result, Action, MultiAction, MultiResponse, MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable and are still used that way.
- After I dug in I found that Writable is literally all over the place still.
- In some cases we even serialize an Object as Writable inside a protobuf message.
- HBaseObjectWritable is still around and still used all over the place
So what we have now has Writables and Protobuf code, worse than it was before (lots of extra code, two ways to serialize stuff, and still no wire compatibility).
HBASE-7215 will fix some of the stuff, but there's a lot more to do.
-- Lars
Re: State of protobufs in hbase-0.96
Posted by lars hofhansl <lh...@yahoo.com>.
I'd be happy if KeyValue is the only class implementing Writable :)
________________________________
From: Jimmy Xiang <jx...@cloudera.com>
To: dev@hbase.apache.org
Cc: lars hofhansl <lh...@yahoo.com>
Sent: Tuesday, November 27, 2012 1:46 PM
Subject: Re: State of protobufs in hbase-0.96
We can't completely purge Writables since we are still using KV in HFile.
Are we going to convert the file format too? If so, we need some
migration strategy.
Thanks,
Jimmy
On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <gh...@gmail.com> wrote:
> Yep, the initial conversion work (HBASE-5448) only deprecated
> CoprocessorProtocol, but in subsequent discussion I think the universal
> consensus was to remove it completely for 0.96 and go PB-only. Otherwise
> we can't completely purge Writables for 0.96.
>
>
>
> On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org> wrote:
>
>> We agreed to remove Writables from coprocessors and go PB-only for 0.96.
>> This way, to avoid building up any transitional cruft.
>>
>>
>> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
>> wrote:
>>
>> > I thought for coprocessors we had decided to only deprecate the old
>> > protocol. I'm +1 removing it, though, because it let's us potentially get
>> > rid of all Writables.
>> >
>> >
>> >
>> > ________________________________
>> > From: Gary Helmling <gh...@gmail.com>
>> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
>> > lhofhansl@yahoo.com>
>> > Sent: Tuesday, November 27, 2012 11:45 AM
>> > Subject: Re: State of protobufs in hbase-0.96
>> >
>> >
>> > For coprocessor endpoints, we have
>> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue
>> for
>> > removing the Writable based CoprocessorProtocol support.
>> >
>> > There are still a few subtasks to work through to get there, but once
>> done
>> > we can pull out CoprocessorProtocol, Exec, ExecResult,
>> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
>> > supporting code.
>> >
>> >
>> >
>> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com>
>> > wrote:
>> >
>> > HBASE-7215 has a rant about the current state of the protobuf work in
>> > 0.96... Continuing here...
>> > >
>> > >HBASE-7215 came about because I simply wanted to add a field to Mutation
>> > (Put, Delete, etc) for HBASE-5954.
>> > >
>> > >Should be easy now with protobufs, right? Nope!
>> > >
>> > >- Both Put and Delete (and Result, Action, MultiAction, MultiResponse,
>> > MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable
>> and
>> > are still used that way.
>> > >- After I dug in I found that Writable is literally all over the place
>> > still.
>> > >- In some cases we even serialize an Object as Writable inside a
>> protobuf
>> > message.
>> > >- HBaseObjectWritable is still around and still used all over the place
>> > >
>> > >So what we have now has Writables and Protobuf code, worse than it was
>> > before (lots of extra code, two ways to serialize stuff, and still no
>> wire
>> > compatibility).
>> > >
>> > >HBASE-7215 will fix some of the stuff, but there's a lot more to do.
>> > >
>> > >-- Lars
>> > >
>>
>>
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
Re: State of protobufs in hbase-0.96
Posted by Enis Söztutar <en...@gmail.com>.
I had opened https://issues.apache.org/jira/browse/HBASE-7201 for some of
the discussions for HFile/Hlog. Converting HFile and HLog meta fields are
relatively easy, and if we are still going to keep KV as writable HFile is
mostly covered. However, I think we should convert HLogKey and WALEdit as
well.
On Tue, Nov 27, 2012 at 3:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> KeyValue is a special case. They're not really Writables either, in that
> readFields() and writeFields() are not used half the time for
> marshalling/unmarshalling.
>
>
> On Tue, Nov 27, 2012 at 1:46 PM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > We can't completely purge Writables since we are still using KV in HFile.
> >
> > Are we going to convert the file format too? If so, we need some
> > migration strategy.
> >
> > Thanks,
> > Jimmy
> >
> > On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <gh...@gmail.com>
> > wrote:
> > > Yep, the initial conversion work (HBASE-5448) only deprecated
> > > CoprocessorProtocol, but in subsequent discussion I think the universal
> > > consensus was to remove it completely for 0.96 and go PB-only.
> Otherwise
> > > we can't completely purge Writables for 0.96.
> > >
> > >
> > >
> > > On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> > >
> > >> We agreed to remove Writables from coprocessors and go PB-only for
> 0.96.
> > >> This way, to avoid building up any transitional cruft.
> > >>
> > >>
> > >> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
> > >> wrote:
> > >>
> > >> > I thought for coprocessors we had decided to only deprecate the old
> > >> > protocol. I'm +1 removing it, though, because it let's us
> potentially
> > get
> > >> > rid of all Writables.
> > >> >
> > >> >
> > >> >
> > >> > ________________________________
> > >> > From: Gary Helmling <gh...@gmail.com>
> > >> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> > >> > lhofhansl@yahoo.com>
> > >> > Sent: Tuesday, November 27, 2012 11:45 AM
> > >> > Subject: Re: State of protobufs in hbase-0.96
> > >> >
> > >> >
> > >> > For coprocessor endpoints, we have
> > >> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella
> issue
> > >> for
> > >> > removing the Writable based CoprocessorProtocol support.
> > >> >
> > >> > There are still a few subtasks to work through to get there, but
> once
> > >> done
> > >> > we can pull out CoprocessorProtocol, Exec, ExecResult,
> > >> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> > >> > supporting code.
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <
> lhofhansl@yahoo.com>
> > >> > wrote:
> > >> >
> > >> > HBASE-7215 has a rant about the current state of the protobuf work
> in
> > >> > 0.96... Continuing here...
> > >> > >
> > >> > >HBASE-7215 came about because I simply wanted to add a field to
> > Mutation
> > >> > (Put, Delete, etc) for HBASE-5954.
> > >> > >
> > >> > >Should be easy now with protobufs, right? Nope!
> > >> > >
> > >> > >- Both Put and Delete (and Result, Action, MultiAction,
> > MultiResponse,
> > >> > MultiPut, MultiResponse, etc, etc, etc) are still implementing
> > Writable
> > >> and
> > >> > are still used that way.
> > >> > >- After I dug in I found that Writable is literally all over the
> > place
> > >> > still.
> > >> > >- In some cases we even serialize an Object as Writable inside a
> > >> protobuf
> > >> > message.
> > >> > >- HBaseObjectWritable is still around and still used all over the
> > place
> > >> > >
> > >> > >So what we have now has Writables and Protobuf code, worse than it
> > was
> > >> > before (lots of extra code, two ways to serialize stuff, and still
> no
> > >> wire
> > >> > compatibility).
> > >> > >
> > >> > >HBASE-7215 will fix some of the stuff, but there's a lot more to
> do.
> > >> > >
> > >> > >-- Lars
> > >> > >
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: State of protobufs in hbase-0.96
Posted by Devaraj Das <dd...@hortonworks.com>.
Have raised https://issues.apache.org/jira/browse/HBASE-7224 for removing
all references to Writable class in the ipc package. Just FYI, please link
all new PB issues up in HBASE-5305 (and it currently has three unassigned
tasks). I see Stack's comments on HBASE-6887 and on HBASE-6789 (indicative
that Stack is looking at them). Stack, in case you are working on these,
please assign them to yourself..
Thanks,
Devaraj.
On Tue, Nov 27, 2012 at 3:26 PM, Andrew Purtell <ap...@apache.org> wrote:
> KeyValue is a special case. They're not really Writables either, in that
> readFields() and writeFields() are not used half the time for
> marshalling/unmarshalling.
>
>
> On Tue, Nov 27, 2012 at 1:46 PM, Jimmy Xiang <jx...@cloudera.com> wrote:
>
> > We can't completely purge Writables since we are still using KV in HFile.
> >
> > Are we going to convert the file format too? If so, we need some
> > migration strategy.
> >
> > Thanks,
> > Jimmy
> >
> > On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <gh...@gmail.com>
> > wrote:
> > > Yep, the initial conversion work (HBASE-5448) only deprecated
> > > CoprocessorProtocol, but in subsequent discussion I think the universal
> > > consensus was to remove it completely for 0.96 and go PB-only.
> Otherwise
> > > we can't completely purge Writables for 0.96.
> > >
> > >
> > >
> > > On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org>
> > wrote:
> > >
> > >> We agreed to remove Writables from coprocessors and go PB-only for
> 0.96.
> > >> This way, to avoid building up any transitional cruft.
> > >>
> > >>
> > >> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
> > >> wrote:
> > >>
> > >> > I thought for coprocessors we had decided to only deprecate the old
> > >> > protocol. I'm +1 removing it, though, because it let's us
> potentially
> > get
> > >> > rid of all Writables.
> > >> >
> > >> >
> > >> >
> > >> > ________________________________
> > >> > From: Gary Helmling <gh...@gmail.com>
> > >> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> > >> > lhofhansl@yahoo.com>
> > >> > Sent: Tuesday, November 27, 2012 11:45 AM
> > >> > Subject: Re: State of protobufs in hbase-0.96
> > >> >
> > >> >
> > >> > For coprocessor endpoints, we have
> > >> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella
> issue
> > >> for
> > >> > removing the Writable based CoprocessorProtocol support.
> > >> >
> > >> > There are still a few subtasks to work through to get there, but
> once
> > >> done
> > >> > we can pull out CoprocessorProtocol, Exec, ExecResult,
> > >> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> > >> > supporting code.
> > >> >
> > >> >
> > >> >
> > >> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <
> lhofhansl@yahoo.com>
> > >> > wrote:
> > >> >
> > >> > HBASE-7215 has a rant about the current state of the protobuf work
> in
> > >> > 0.96... Continuing here...
> > >> > >
> > >> > >HBASE-7215 came about because I simply wanted to add a field to
> > Mutation
> > >> > (Put, Delete, etc) for HBASE-5954.
> > >> > >
> > >> > >Should be easy now with protobufs, right? Nope!
> > >> > >
> > >> > >- Both Put and Delete (and Result, Action, MultiAction,
> > MultiResponse,
> > >> > MultiPut, MultiResponse, etc, etc, etc) are still implementing
> > Writable
> > >> and
> > >> > are still used that way.
> > >> > >- After I dug in I found that Writable is literally all over the
> > place
> > >> > still.
> > >> > >- In some cases we even serialize an Object as Writable inside a
> > >> protobuf
> > >> > message.
> > >> > >- HBaseObjectWritable is still around and still used all over the
> > place
> > >> > >
> > >> > >So what we have now has Writables and Protobuf code, worse than it
> > was
> > >> > before (lots of extra code, two ways to serialize stuff, and still
> no
> > >> wire
> > >> > compatibility).
> > >> > >
> > >> > >HBASE-7215 will fix some of the stuff, but there's a lot more to
> do.
> > >> > >
> > >> > >-- Lars
> > >> > >
> > >>
> > >>
> > >>
> > >>
> > >> --
> > >> Best regards,
> > >>
> > >> - Andy
> > >>
> > >> Problems worthy of attack prove their worth by hitting back. - Piet
> Hein
> > >> (via Tom White)
> > >>
> >
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: State of protobufs in hbase-0.96
Posted by Andrew Purtell <ap...@apache.org>.
KeyValue is a special case. They're not really Writables either, in that
readFields() and writeFields() are not used half the time for
marshalling/unmarshalling.
On Tue, Nov 27, 2012 at 1:46 PM, Jimmy Xiang <jx...@cloudera.com> wrote:
> We can't completely purge Writables since we are still using KV in HFile.
>
> Are we going to convert the file format too? If so, we need some
> migration strategy.
>
> Thanks,
> Jimmy
>
> On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <gh...@gmail.com>
> wrote:
> > Yep, the initial conversion work (HBASE-5448) only deprecated
> > CoprocessorProtocol, but in subsequent discussion I think the universal
> > consensus was to remove it completely for 0.96 and go PB-only. Otherwise
> > we can't completely purge Writables for 0.96.
> >
> >
> >
> > On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org>
> wrote:
> >
> >> We agreed to remove Writables from coprocessors and go PB-only for 0.96.
> >> This way, to avoid building up any transitional cruft.
> >>
> >>
> >> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
> >> wrote:
> >>
> >> > I thought for coprocessors we had decided to only deprecate the old
> >> > protocol. I'm +1 removing it, though, because it let's us potentially
> get
> >> > rid of all Writables.
> >> >
> >> >
> >> >
> >> > ________________________________
> >> > From: Gary Helmling <gh...@gmail.com>
> >> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> >> > lhofhansl@yahoo.com>
> >> > Sent: Tuesday, November 27, 2012 11:45 AM
> >> > Subject: Re: State of protobufs in hbase-0.96
> >> >
> >> >
> >> > For coprocessor endpoints, we have
> >> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue
> >> for
> >> > removing the Writable based CoprocessorProtocol support.
> >> >
> >> > There are still a few subtasks to work through to get there, but once
> >> done
> >> > we can pull out CoprocessorProtocol, Exec, ExecResult,
> >> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> >> > supporting code.
> >> >
> >> >
> >> >
> >> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com>
> >> > wrote:
> >> >
> >> > HBASE-7215 has a rant about the current state of the protobuf work in
> >> > 0.96... Continuing here...
> >> > >
> >> > >HBASE-7215 came about because I simply wanted to add a field to
> Mutation
> >> > (Put, Delete, etc) for HBASE-5954.
> >> > >
> >> > >Should be easy now with protobufs, right? Nope!
> >> > >
> >> > >- Both Put and Delete (and Result, Action, MultiAction,
> MultiResponse,
> >> > MultiPut, MultiResponse, etc, etc, etc) are still implementing
> Writable
> >> and
> >> > are still used that way.
> >> > >- After I dug in I found that Writable is literally all over the
> place
> >> > still.
> >> > >- In some cases we even serialize an Object as Writable inside a
> >> protobuf
> >> > message.
> >> > >- HBaseObjectWritable is still around and still used all over the
> place
> >> > >
> >> > >So what we have now has Writables and Protobuf code, worse than it
> was
> >> > before (lots of extra code, two ways to serialize stuff, and still no
> >> wire
> >> > compatibility).
> >> > >
> >> > >HBASE-7215 will fix some of the stuff, but there's a lot more to do.
> >> > >
> >> > >-- Lars
> >> > >
> >>
> >>
> >>
> >>
> >> --
> >> Best regards,
> >>
> >> - Andy
> >>
> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> >> (via Tom White)
> >>
>
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: State of protobufs in hbase-0.96
Posted by Jimmy Xiang <jx...@cloudera.com>.
We can't completely purge Writables since we are still using KV in HFile.
Are we going to convert the file format too? If so, we need some
migration strategy.
Thanks,
Jimmy
On Tue, Nov 27, 2012 at 1:41 PM, Gary Helmling <gh...@gmail.com> wrote:
> Yep, the initial conversion work (HBASE-5448) only deprecated
> CoprocessorProtocol, but in subsequent discussion I think the universal
> consensus was to remove it completely for 0.96 and go PB-only. Otherwise
> we can't completely purge Writables for 0.96.
>
>
>
> On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org> wrote:
>
>> We agreed to remove Writables from coprocessors and go PB-only for 0.96.
>> This way, to avoid building up any transitional cruft.
>>
>>
>> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
>> wrote:
>>
>> > I thought for coprocessors we had decided to only deprecate the old
>> > protocol. I'm +1 removing it, though, because it let's us potentially get
>> > rid of all Writables.
>> >
>> >
>> >
>> > ________________________________
>> > From: Gary Helmling <gh...@gmail.com>
>> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
>> > lhofhansl@yahoo.com>
>> > Sent: Tuesday, November 27, 2012 11:45 AM
>> > Subject: Re: State of protobufs in hbase-0.96
>> >
>> >
>> > For coprocessor endpoints, we have
>> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue
>> for
>> > removing the Writable based CoprocessorProtocol support.
>> >
>> > There are still a few subtasks to work through to get there, but once
>> done
>> > we can pull out CoprocessorProtocol, Exec, ExecResult,
>> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
>> > supporting code.
>> >
>> >
>> >
>> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com>
>> > wrote:
>> >
>> > HBASE-7215 has a rant about the current state of the protobuf work in
>> > 0.96... Continuing here...
>> > >
>> > >HBASE-7215 came about because I simply wanted to add a field to Mutation
>> > (Put, Delete, etc) for HBASE-5954.
>> > >
>> > >Should be easy now with protobufs, right? Nope!
>> > >
>> > >- Both Put and Delete (and Result, Action, MultiAction, MultiResponse,
>> > MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable
>> and
>> > are still used that way.
>> > >- After I dug in I found that Writable is literally all over the place
>> > still.
>> > >- In some cases we even serialize an Object as Writable inside a
>> protobuf
>> > message.
>> > >- HBaseObjectWritable is still around and still used all over the place
>> > >
>> > >So what we have now has Writables and Protobuf code, worse than it was
>> > before (lots of extra code, two ways to serialize stuff, and still no
>> wire
>> > compatibility).
>> > >
>> > >HBASE-7215 will fix some of the stuff, but there's a lot more to do.
>> > >
>> > >-- Lars
>> > >
>>
>>
>>
>>
>> --
>> Best regards,
>>
>> - Andy
>>
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>
Re: State of protobufs in hbase-0.96
Posted by Gary Helmling <gh...@gmail.com>.
Yep, the initial conversion work (HBASE-5448) only deprecated
CoprocessorProtocol, but in subsequent discussion I think the universal
consensus was to remove it completely for 0.96 and go PB-only. Otherwise
we can't completely purge Writables for 0.96.
On Tue, Nov 27, 2012 at 1:37 PM, Andrew Purtell <ap...@apache.org> wrote:
> We agreed to remove Writables from coprocessors and go PB-only for 0.96.
> This way, to avoid building up any transitional cruft.
>
>
> On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com>
> wrote:
>
> > I thought for coprocessors we had decided to only deprecate the old
> > protocol. I'm +1 removing it, though, because it let's us potentially get
> > rid of all Writables.
> >
> >
> >
> > ________________________________
> > From: Gary Helmling <gh...@gmail.com>
> > To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> > lhofhansl@yahoo.com>
> > Sent: Tuesday, November 27, 2012 11:45 AM
> > Subject: Re: State of protobufs in hbase-0.96
> >
> >
> > For coprocessor endpoints, we have
> > https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue
> for
> > removing the Writable based CoprocessorProtocol support.
> >
> > There are still a few subtasks to work through to get there, but once
> done
> > we can pull out CoprocessorProtocol, Exec, ExecResult,
> > HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> > supporting code.
> >
> >
> >
> > On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com>
> > wrote:
> >
> > HBASE-7215 has a rant about the current state of the protobuf work in
> > 0.96... Continuing here...
> > >
> > >HBASE-7215 came about because I simply wanted to add a field to Mutation
> > (Put, Delete, etc) for HBASE-5954.
> > >
> > >Should be easy now with protobufs, right? Nope!
> > >
> > >- Both Put and Delete (and Result, Action, MultiAction, MultiResponse,
> > MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable
> and
> > are still used that way.
> > >- After I dug in I found that Writable is literally all over the place
> > still.
> > >- In some cases we even serialize an Object as Writable inside a
> protobuf
> > message.
> > >- HBaseObjectWritable is still around and still used all over the place
> > >
> > >So what we have now has Writables and Protobuf code, worse than it was
> > before (lots of extra code, two ways to serialize stuff, and still no
> wire
> > compatibility).
> > >
> > >HBASE-7215 will fix some of the stuff, but there's a lot more to do.
> > >
> > >-- Lars
> > >
>
>
>
>
> --
> Best regards,
>
> - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>
Re: State of protobufs in hbase-0.96
Posted by Andrew Purtell <ap...@apache.org>.
We agreed to remove Writables from coprocessors and go PB-only for 0.96.
This way, to avoid building up any transitional cruft.
On Tue, Nov 27, 2012 at 1:29 PM, lars hofhansl <lh...@yahoo.com> wrote:
> I thought for coprocessors we had decided to only deprecate the old
> protocol. I'm +1 removing it, though, because it let's us potentially get
> rid of all Writables.
>
>
>
> ________________________________
> From: Gary Helmling <gh...@gmail.com>
> To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <
> lhofhansl@yahoo.com>
> Sent: Tuesday, November 27, 2012 11:45 AM
> Subject: Re: State of protobufs in hbase-0.96
>
>
> For coprocessor endpoints, we have
> https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue for
> removing the Writable based CoprocessorProtocol support.
>
> There are still a few subtasks to work through to get there, but once done
> we can pull out CoprocessorProtocol, Exec, ExecResult,
> HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
> supporting code.
>
>
>
> On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com>
> wrote:
>
> HBASE-7215 has a rant about the current state of the protobuf work in
> 0.96... Continuing here...
> >
> >HBASE-7215 came about because I simply wanted to add a field to Mutation
> (Put, Delete, etc) for HBASE-5954.
> >
> >Should be easy now with protobufs, right? Nope!
> >
> >- Both Put and Delete (and Result, Action, MultiAction, MultiResponse,
> MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable and
> are still used that way.
> >- After I dug in I found that Writable is literally all over the place
> still.
> >- In some cases we even serialize an Object as Writable inside a protobuf
> message.
> >- HBaseObjectWritable is still around and still used all over the place
> >
> >So what we have now has Writables and Protobuf code, worse than it was
> before (lots of extra code, two ways to serialize stuff, and still no wire
> compatibility).
> >
> >HBASE-7215 will fix some of the stuff, but there's a lot more to do.
> >
> >-- Lars
> >
--
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)
Re: State of protobufs in hbase-0.96
Posted by lars hofhansl <lh...@yahoo.com>.
I thought for coprocessors we had decided to only deprecate the old protocol. I'm +1 removing it, though, because it let's us potentially get rid of all Writables.
________________________________
From: Gary Helmling <gh...@gmail.com>
To: "dev@hbase.apache.org" <de...@hbase.apache.org>; lars hofhansl <lh...@yahoo.com>
Sent: Tuesday, November 27, 2012 11:45 AM
Subject: Re: State of protobufs in hbase-0.96
For coprocessor endpoints, we have https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue for removing the Writable based CoprocessorProtocol support.
There are still a few subtasks to work through to get there, but once done we can pull out CoprocessorProtocol, Exec, ExecResult, HTable.coprocessorProxy(), HTable.coprocessorExec() and any other supporting code.
On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com> wrote:
HBASE-7215 has a rant about the current state of the protobuf work in 0.96... Continuing here...
>
>HBASE-7215 came about because I simply wanted to add a field to Mutation (Put, Delete, etc) for HBASE-5954.
>
>Should be easy now with protobufs, right? Nope!
>
>- Both Put and Delete (and Result, Action, MultiAction, MultiResponse, MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable and are still used that way.
>- After I dug in I found that Writable is literally all over the place still.
>- In some cases we even serialize an Object as Writable inside a protobuf message.
>- HBaseObjectWritable is still around and still used all over the place
>
>So what we have now has Writables and Protobuf code, worse than it was before (lots of extra code, two ways to serialize stuff, and still no wire compatibility).
>
>HBASE-7215 will fix some of the stuff, but there's a lot more to do.
>
>-- Lars
>
Re: State of protobufs in hbase-0.96
Posted by Gary Helmling <gh...@gmail.com>.
For coprocessor endpoints, we have
https://issues.apache.org/jira/browse/HBASE-6895 as an umbrella issue for
removing the Writable based CoprocessorProtocol support.
There are still a few subtasks to work through to get there, but once done
we can pull out CoprocessorProtocol, Exec, ExecResult,
HTable.coprocessorProxy(), HTable.coprocessorExec() and any other
supporting code.
On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com> wrote:
> HBASE-7215 has a rant about the current state of the protobuf work in
> 0.96... Continuing here...
>
> HBASE-7215 came about because I simply wanted to add a field to Mutation
> (Put, Delete, etc) for HBASE-5954.
>
> Should be easy now with protobufs, right? Nope!
>
> - Both Put and Delete (and Result, Action, MultiAction, MultiResponse,
> MultiPut, MultiResponse, etc, etc, etc) are still implementing Writable and
> are still used that way.
> - After I dug in I found that Writable is literally all over the place
> still.
> - In some cases we even serialize an Object as Writable inside a protobuf
> message.
> - HBaseObjectWritable is still around and still used all over the place
>
> So what we have now has Writables and Protobuf code, worse than it was
> before (lots of extra code, two ways to serialize stuff, and still no wire
> compatibility).
>
> HBASE-7215 will fix some of the stuff, but there's a lot more to do.
>
> -- Lars
>
Re: State of protobufs in hbase-0.96
Posted by Stack <st...@duboce.net>.
On Tue, Nov 27, 2012 at 10:54 AM, lars hofhansl <lh...@yahoo.com> wrote:
> HBASE-7215 has a rant about the current state of the protobuf work in 0.96... Continuing here...
>
Thanks Lars for the rant (You could have gone on way longer!).
Let me take on removing HbaseObjectWritable. Will file blockers for
the others (Gary has a CP convertion issue going on and Devaraj has
been hereabouts too).
In general, if folks are having trouble figuring how to help along
0.96, just take any of the blockers and criticals from here:
https://issues.apache.org/jira/browse/HBASE/fixforversion/12320040#selectedTab=com.atlassian.jira.plugin.system.project%3Aversion-issues-panel
There is still loads to do....
St.Ack