You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Jonathan Ellis <jb...@gmail.com> on 2012/11/30 21:12:35 UTC
2.0
The more I think about it, the more I think we should call 1.2-next,
2.0. I'd like to spend some time paying off our technical debt:
- replace supercolumns with composites (CASSANDRA-3237)
- rewrite counters (CASSANDRA-4775)
- improve storage engine support for wide rows
- better stage management to improve latency (disruptor? lightweight
threads? custom executor + queue?)
- improved repair (CASSANDRA-3362, 2699)
Of course, we're planning some new features as well:
- triggers (CASSANDRA-1311)
- improved query fault tolerance (CASSANDRA-4705)
- row size limits (CASSANDRA-3929)
- cql3 integration for hadoop (CASSANDRA-4421)
- improved caching (CASSANDRA-1956, 2864)
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced
Re: 2.0
Posted by Jason Brown <ja...@gmail.com>.
My hope is that after 1.2 (i.e. by the time we're 2.0'ing), the binary CQL
protocol is out of beta :).
On Fri, Nov 30, 2012 at 2:39 PM, Edward Capriolo <ed...@gmail.com>wrote:
> Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to
> upgrade to a version that removes thrift. Then when all our clients can not
> connect they will be forced to get with the program
>
Re: 2.0
Posted by Edward Capriolo <ed...@gmail.com>.
Good idea. Lets remove thrift, CQL3 is still beta, but I am willing to
upgrade to a version that removes thrift. Then when all our clients can not
connect they will be forced to get with the program.
On Fri, Nov 30, 2012 at 5:33 PM, Jason Brown <ja...@gmail.com> wrote:
> Hi Jonathan,
>
> I'm in favor of paying off the technical debt, as well, and I wonder if
> there is value in removing support for thrift with 2.0? We're currently in
> 'do as little as possible' mode with thrift, so should we aggressively cast
> it off and push the binary CQL protocol? Seems like a jump to '2.0', along
> with the other initiatives, would be a reasonable time/milestone to do so.
>
> Thanks,
>
> -Jason
>
>
> On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
>
> > The more I think about it, the more I think we should call 1.2-next,
> > 2.0. I'd like to spend some time paying off our technical debt:
> >
> > - replace supercolumns with composites (CASSANDRA-3237)
> > - rewrite counters (CASSANDRA-4775)
> > - improve storage engine support for wide rows
> > - better stage management to improve latency (disruptor? lightweight
> > threads? custom executor + queue?)
> > - improved repair (CASSANDRA-3362, 2699)
> >
> > Of course, we're planning some new features as well:
> > - triggers (CASSANDRA-1311)
> > - improved query fault tolerance (CASSANDRA-4705)
> > - row size limits (CASSANDRA-3929)
> > - cql3 integration for hadoop (CASSANDRA-4421)
> > - improved caching (CASSANDRA-1956, 2864)
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder, http://www.datastax.com
> > @spyced
> >
>
Re: 2.0
Posted by Bill de hÓra <bi...@dehora.net>.
I'm not thrilled with Thrift, but I'd like to see and hear more real
world use of CQL first (Avro all the things is not that long ago).
That said, a major rev /could/ do this - not start the thrift server by
default. It's then a hoop jump to enable it via nodetool/yaml, and
signals to the client communities and user base, where the project
intends to go.
Bill
On 30/11/12 22:49, Jonathan Ellis wrote:
> As attractive as it would be to clean house, I think we owe it to our
> users to keep Thrift around for the forseeable future rather than
> orphan all Thrift-using applications (which is virtually everyone) on
> 1.2.
>
> On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com> wrote:
>> Hi Jonathan,
>>
>> I'm in favor of paying off the technical debt, as well, and I wonder if
>> there is value in removing support for thrift with 2.0? We're currently in
>> 'do as little as possible' mode with thrift, so should we aggressively cast
>> it off and push the binary CQL protocol? Seems like a jump to '2.0', along
>> with the other initiatives, would be a reasonable time/milestone to do so.
>>
>> Thanks,
>>
>> -Jason
>>
>>
>> On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>>
>>> The more I think about it, the more I think we should call 1.2-next,
>>> 2.0. I'd like to spend some time paying off our technical debt:
>>>
>>> - replace supercolumns with composites (CASSANDRA-3237)
>>> - rewrite counters (CASSANDRA-4775)
>>> - improve storage engine support for wide rows
>>> - better stage management to improve latency (disruptor? lightweight
>>> threads? custom executor + queue?)
>>> - improved repair (CASSANDRA-3362, 2699)
>>>
>>> Of course, we're planning some new features as well:
>>> - triggers (CASSANDRA-1311)
>>> - improved query fault tolerance (CASSANDRA-4705)
>>> - row size limits (CASSANDRA-3929)
>>> - cql3 integration for hadoop (CASSANDRA-4421)
>>> - improved caching (CASSANDRA-1956, 2864)
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder, http://www.datastax.com
>>> @spyced
>>>
>
>
>
RE: Stable Hector version with cassandra 1.1.6
Posted by "Bisht, Jaikrit" <bi...@visa.com>.
Thanks for the response, I will use the following version of hector client for cassandra 1.1.6.
<dependency>
<groupId>org.hectorclient</groupId>
<artifactId>hector-core</artifactId>
<version>1.1-1</version>
</dependency>
-----Original Message-----
From: Edward Capriolo [mailto:edlinuxguru@gmail.com]
Sent: 04 December 2012 14:46
To: dev@cassandra.apache.org
Subject: Re: Stable Hector version with cassandra 1.1.6
One thing to note. The maven repo has moved from me.prettyprint to
org.Hector-client so that should aid in your searches of the maven repo.
On Tuesday, December 4, 2012, Bisht, Jaikrit <bi...@visa.com> wrote:
>
> Hi,
>
> Could someone recommend the stable version of Hector libraries for
Cassandra 1.1.6?
>
> Regards
> Jay
>
> VISA EUROPE is a technology business that provides the brand, systems,
services and rules that make electronic payments between millions of
consumers, retailers, businesses and governments happen. Visa Europe is a
membership association of more than 3,700 members that includes banks and
other payment service providers from 36 countries across Europe. We
continually invest and innovate to create new and better ways to pay and be
paid. For more information, please visit www.visaeurope.com.
>
> Please consider the environment before printing this email.
>
> This email (including attachments) is confidential and is solely intended
for the addressee. Unless you are the addressee, you may not read, use or
store this email in any way, or permit others to. If you have received it
in error, please contact Visa Europe on +44 (0)20 7937 8111.
>
> Visa Europe Services Inc. is a company incorporated in Delaware USA,
acting through its UK Establishment (UK Establishment number BR007632)
whose registered office is at 1 Sheldon Square, London, W2 6TT.
>
VISA EUROPE is a technology business that provides the brand, systems, services and rules that make electronic payments between millions of consumers, retailers, businesses and governments happen. Visa Europe is a membership association of more than 3,700 members that includes banks and other payment service providers from 36 countries across Europe. We continually invest and innovate to create new and better ways to pay and be paid. For more information, please visit www.visaeurope.com.
Please consider the environment before printing this email.
This email (including attachments) is confidential and is solely intended for the addressee. Unless you are the addressee, you may not read, use or store this email in any way, or permit others to. If you have received it in error, please contact Visa Europe on +44 (0)20 7937 8111.
Visa Europe Services Inc. is a company incorporated in Delaware USA, acting through its UK Establishment (UK Establishment number BR007632) whose registered office is at 1 Sheldon Square, London, W2 6TT.
Re: Stable Hector version with cassandra 1.1.6
Posted by Edward Capriolo <ed...@gmail.com>.
One thing to note. The maven repo has moved from me.prettyprint to
org.Hector-client so that should aid in your searches of the maven repo.
On Tuesday, December 4, 2012, Bisht, Jaikrit <bi...@visa.com> wrote:
>
> Hi,
>
> Could someone recommend the stable version of Hector libraries for
Cassandra 1.1.6?
>
> Regards
> Jay
>
> VISA EUROPE is a technology business that provides the brand, systems,
services and rules that make electronic payments between millions of
consumers, retailers, businesses and governments happen. Visa Europe is a
membership association of more than 3,700 members that includes banks and
other payment service providers from 36 countries across Europe. We
continually invest and innovate to create new and better ways to pay and be
paid. For more information, please visit www.visaeurope.com.
>
> Please consider the environment before printing this email.
>
> This email (including attachments) is confidential and is solely intended
for the addressee. Unless you are the addressee, you may not read, use or
store this email in any way, or permit others to. If you have received it
in error, please contact Visa Europe on +44 (0)20 7937 8111.
>
> Visa Europe Services Inc. is a company incorporated in Delaware USA,
acting through its UK Establishment (UK Establishment number BR007632)
whose registered office is at 1 Sheldon Square, London, W2 6TT.
>
Stable Hector version with cassandra 1.1.6
Posted by "Bisht, Jaikrit" <bi...@visa.com>.
Hi,
Could someone recommend the stable version of Hector libraries for Cassandra 1.1.6?
Regards
Jay
VISA EUROPE is a technology business that provides the brand, systems, services and rules that make electronic payments between millions of consumers, retailers, businesses and governments happen. Visa Europe is a membership association of more than 3,700 members that includes banks and other payment service providers from 36 countries across Europe. We continually invest and innovate to create new and better ways to pay and be paid. For more information, please visit www.visaeurope.com.
Please consider the environment before printing this email.
This email (including attachments) is confidential and is solely intended for the addressee. Unless you are the addressee, you may not read, use or store this email in any way, or permit others to. If you have received it in error, please contact Visa Europe on +44 (0)20 7937 8111.
Visa Europe Services Inc. is a company incorporated in Delaware USA, acting through its UK Establishment (UK Establishment number BR007632) whose registered office is at 1 Sheldon Square, London, W2 6TT.
Re: 2.0
Posted by Jason Brown <ja...@gmail.com>.
Oops, meant to address this specifically to Jonathan, but since I've
confused 'reply' with 'forward'. my apologies for any extra noise on this
topic.
On Mon, Dec 3, 2012 at 9:25 AM, Jason Brown <ja...@gmail.com> wrote:
> - world
>
> Hi Jonathan,
>
> This topic may have been discussed elsewhere, or my memory is worse off
> than I thought, but what is our long term vision for thrift support?
> Admittedly, I need to learn much more about the binary CQL protocol, and I
> understand Ed's concerns, as well (more acutely now) about existing
> installations, but we probably wouldn't have dreamt up a new client
> interface/protocol if we went planning, at some point, on retiring the old
> one. And, also, I missed the Avro debate from the past, so I'm not sure how
> much that affects current and future thinking.
>
> After raising the issue here on the dev list, it certainly seems like 2.0
> is premature for a full-on switch over, and Ed raised some interesting
> metrics to consider when we could declare the CQL protocol as 'accepted'.
> I'm curious as to how you are seeing it roll out.
>
> Thanks for your time,
>
> -Jason
>
>
>
>
>
> On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> As attractive as it would be to clean house, I think we owe it to our
>> users to keep Thrift around for the forseeable future rather than
>> orphan all Thrift-using applications (which is virtually everyone) on
>> 1.2.
>>
>> On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com> wrote:
>> > Hi Jonathan,
>> >
>> > I'm in favor of paying off the technical debt, as well, and I wonder if
>> > there is value in removing support for thrift with 2.0? We're currently
>> in
>> > 'do as little as possible' mode with thrift, so should we aggressively
>> cast
>> > it off and push the binary CQL protocol? Seems like a jump to '2.0',
>> along
>> > with the other initiatives, would be a reasonable time/milestone to do
>> so.
>> >
>> > Thanks,
>> >
>> > -Jason
>> >
>> >
>> > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com>
>> wrote:
>> >
>> >> The more I think about it, the more I think we should call 1.2-next,
>> >> 2.0. I'd like to spend some time paying off our technical debt:
>> >>
>> >> - replace supercolumns with composites (CASSANDRA-3237)
>> >> - rewrite counters (CASSANDRA-4775)
>> >> - improve storage engine support for wide rows
>> >> - better stage management to improve latency (disruptor? lightweight
>> >> threads? custom executor + queue?)
>> >> - improved repair (CASSANDRA-3362, 2699)
>> >>
>> >> Of course, we're planning some new features as well:
>> >> - triggers (CASSANDRA-1311)
>> >> - improved query fault tolerance (CASSANDRA-4705)
>> >> - row size limits (CASSANDRA-3929)
>> >> - cql3 integration for hadoop (CASSANDRA-4421)
>> >> - improved caching (CASSANDRA-1956, 2864)
>> >>
>> >> --
>> >> Jonathan Ellis
>> >> Project Chair, Apache Cassandra
>> >> co-founder, http://www.datastax.com
>> >> @spyced
>> >>
>>
>>
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder, http://www.datastax.com
>> @spyced
>>
>
>
Re: 2.0
Posted by Jason Brown <ja...@gmail.com>.
- world
Hi Jonathan,
This topic may have been discussed elsewhere, or my memory is worse off
than I thought, but what is our long term vision for thrift support?
Admittedly, I need to learn much more about the binary CQL protocol, and I
understand Ed's concerns, as well (more acutely now) about existing
installations, but we probably wouldn't have dreamt up a new client
interface/protocol if we went planning, at some point, on retiring the old
one. And, also, I missed the Avro debate from the past, so I'm not sure how
much that affects current and future thinking.
After raising the issue here on the dev list, it certainly seems like 2.0
is premature for a full-on switch over, and Ed raised some interesting
metrics to consider when we could declare the CQL protocol as 'accepted'.
I'm curious as to how you are seeing it roll out.
Thanks for your time,
-Jason
On Fri, Nov 30, 2012 at 2:49 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> As attractive as it would be to clean house, I think we owe it to our
> users to keep Thrift around for the forseeable future rather than
> orphan all Thrift-using applications (which is virtually everyone) on
> 1.2.
>
> On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com> wrote:
> > Hi Jonathan,
> >
> > I'm in favor of paying off the technical debt, as well, and I wonder if
> > there is value in removing support for thrift with 2.0? We're currently
> in
> > 'do as little as possible' mode with thrift, so should we aggressively
> cast
> > it off and push the binary CQL protocol? Seems like a jump to '2.0',
> along
> > with the other initiatives, would be a reasonable time/milestone to do
> so.
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> >
> >> The more I think about it, the more I think we should call 1.2-next,
> >> 2.0. I'd like to spend some time paying off our technical debt:
> >>
> >> - replace supercolumns with composites (CASSANDRA-3237)
> >> - rewrite counters (CASSANDRA-4775)
> >> - improve storage engine support for wide rows
> >> - better stage management to improve latency (disruptor? lightweight
> >> threads? custom executor + queue?)
> >> - improved repair (CASSANDRA-3362, 2699)
> >>
> >> Of course, we're planning some new features as well:
> >> - triggers (CASSANDRA-1311)
> >> - improved query fault tolerance (CASSANDRA-4705)
> >> - row size limits (CASSANDRA-3929)
> >> - cql3 integration for hadoop (CASSANDRA-4421)
> >> - improved caching (CASSANDRA-1956, 2864)
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder, http://www.datastax.com
> >> @spyced
> >>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
Re: 2.0
Posted by Drew Kutcharian <dr...@venarc.com>.
I agree with Edward here. We use Thrift too and we haven't really found a good enough reason to move to CQL3.
-- Drew
On Dec 1, 2012, at 10:24 AM, Edward Capriolo <ed...@gmail.com> wrote:
> I do not understand why everyone wants to force this issue on removing
> thrift. If cql, cql sparse tables and the new transport are better people
> will naturally begin to use them, but as it stands now I see the it
> this way:
>
> Thrift still has more clients for more languages, thrift has more higher
> level clients for more languages.
> Thrift has Hadoop support hive support and pig support in the wild.
> Thrift has third party tools like Orm tools, support for tools like flume.
>
> Most of cql3 features like collections do not work with compact tables,
> and compact tables are much more space efficient then their cql3 sparse
> counterparts, composite rows with UTf column names, blank rows, etc.
> Cql3 binary client is only available for in beta stage for a few languages.
>
> So the project can easily remove thrift today but until a majority of the
> tooling by the community adopts the transport and for the most part cqls
> sparse tables it is not going to mean anything. Many people already have
> code live in production working fine with the old toolset and will be
> unwilling to convert something just because....
>
> Think about it like this a company like mine that already has something in
> production. Even if you could convince me us that cql native transport was
> better, which by the way no one has showed me a vast performance reason to
> this point, they still may not want to invest the resources to convert
> their app. Many companies endured the painful transition from Cassandra 0.6
> to Cassandra 0.7 conversion and they are not eagerly going to entertain
> another change which is mostly cosmetic.
>
> Also I find issues like this extremely frustrating.
> https://issues.apache.org/jira/browse/CASSANDRA-4924
>
> It seems like the project is drawing a hard line in the sand dividing
> people. Is it the case that cql3's sparse tables can't be accessed
> by thrift, or is it the case that no one wants to make this happen? Like is
> it technically impossible? It seems not to me in Cassandra
> Row key, column, and value are all still byte arrays right? So I do not see
> why thrift users need to be locked out of them. Just like composites we
> will figure out how to pack the bytes.
>
> I hope that we can stop talking about removing thrift until there is some
> consensus between active users that it is not in use anymore.
> This consensus is not as simple as n committers saying that something is
> technically not needed anymore. It has to look at the users, the number of
> clients, the number of languages, the number of high level tools available.
> In the mean time when issues like 4924 pop up it would be better if people
> tried to find solutions for maximum forward and backward compatibility
> instead of drawing a line and trying to shut thrift users out of things.
>
> Avro was much the same way . I had a spirited debate on irc and got
> basicallly insulted because i belived thrift was not dead. The glory of
> avro never came true because it really did not work for clients outside a
> few languages. Cql and the binary transport has to pass this same litmus
> test. Let it gain momentum and have rock solid clients for 5 languages and
> have higher level tools written on top of it then its easy to say thrift is
> not needed anymore.
>
>
> On Saturday, December 1, 2012, Sylvain Lebresne wrote:
>
>> I agree on 2.0.
>>
>> For the thrift part, we've said clearly that we wouldn't remove it any time
>> soon so let's stick to that. Besides, I would agree it's too soon anyway.
>> What we can do however in the relatively short term on that front, is to
>> pull thrift in it's own jar (we've almost removed all internal dependencies
>> on thrift, and the few remaining ones will be easy to kill) and make that
>> jar optional if you don't want to use it.
>>
>> --
>> Sylvain
>>
>>
>> On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski <ray.slakinski@gmail.com<javascript:;>
>>> wrote:
>>
>>> I agree, I don't think its a great idea to drop thrift until the back
>>> end tools are 100% compatible and have some level of agreement from the
>>> major users of
>>> Cassandra.
>>>
>>> Paying off technical dept though I'm all for, and I think its key to the
>>> long term success of the application. Right now Supercolumns to someone
>>> new coming to the system might think "Hey, these things look great. Lets
>>> use them" and in a few months time hate all things that are cassandra.
>>>
>>> Ray Slakinski
>>>
>>> On 12/01, Jonathan Ellis wrote:
>>>> As attractive as it would be to clean house, I think we owe it to our
>>>> users to keep Thrift around for the forseeable future rather than
>>>> orphan all Thrift-using applications (which is virtually everyone) on
>>>> 1.2.
>>>>
>>>> On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <jasedbrown@gmail.com<javascript:;>
>>>
>>> wrote:
>>>>> Hi Jonathan,
>>>>>
>>>>> I'm in favor of paying off the technical debt, as well, and I wonder
>> if
>>>>> there is value in removing support for thrift with 2.0? We're
>>> currently in
>>>>> 'do as little as possible' mode with thrift, so should we
>> aggressively
>>> cast
>>>>> it off and push the binary CQL protocol? Seems like a jump to '2.0',
>>> along
>>>>> with the other initiatives, would be a reasonable time/milestone to
>> do
>>> so.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> -Jason
>>>>>
>>>>>
>>>>> On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jbellis@gmail.com<javascript:;>
>>>
>>> wrote:
>>>>>
>>>>>> The more I think about it, the more I think we should call 1.2-next,
>>>>>> 2.0. I'd like to spend some time paying off our technical debt:
>>>>>>
>>>>>> - replace supercolumns with composites (CASSANDRA-3237)
>>>>>> - rewrite counters (CASSANDRA-4775)
>>>>>> - improve storage engine support for wide rows
>>>>>> - better stage management to improve latency (disruptor? lightweight
>>>>>> threads? custom executor + queue?)
>>>>>> - improved repair (CASSANDRA-3362, 2699)
>>>>>>
>>>>>> Of course, we're planning some new features as well:
>>>>>> - triggers (CASSANDRA-1311)
>>>>>> - improved query fault tolerance (CASSANDRA-4705)
>>>>>> - row size limits (CASSANDRA-3929)
>>>>>> - cql3 integration for hadoop (CASSANDRA-4421)
>>>>>> - improved caching (CASSANDRA-1956, 2864)
>>>>>>
>>>>>> --
>>>>>> Jonathan Ellis
>>>>>> Project Chair, Apache Cassandra
>>>>>> co-founder, http://www.datastax.com
>>>>>> @spyced
>>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Jonathan Ellis
>>>> Project Chair, Apache Cassandra
>>>> co-founder, http://www.datastax.com
>>>> @spyced
>>>
>>
Re: 2.0
Posted by Brandon Williams <dr...@gmail.com>.
On Sat, Dec 1, 2012 at 12:24 PM, Edward Capriolo <ed...@gmail.com> wrote:
> I do not understand why everyone wants to force this issue on removing
> thrift.
I'm -1 on removing thrift, and by my count, that would put us at -3
binding if it ever came to vote, so let's consider this proposition
closed and move on to other more constructive points.
-Brandon
Re: 2.0
Posted by Edward Capriolo <ed...@gmail.com>.
I do not understand why everyone wants to force this issue on removing
thrift. If cql, cql sparse tables and the new transport are better people
will naturally begin to use them, but as it stands now I see the it
this way:
Thrift still has more clients for more languages, thrift has more higher
level clients for more languages.
Thrift has Hadoop support hive support and pig support in the wild.
Thrift has third party tools like Orm tools, support for tools like flume.
Most of cql3 features like collections do not work with compact tables,
and compact tables are much more space efficient then their cql3 sparse
counterparts, composite rows with UTf column names, blank rows, etc.
Cql3 binary client is only available for in beta stage for a few languages.
So the project can easily remove thrift today but until a majority of the
tooling by the community adopts the transport and for the most part cqls
sparse tables it is not going to mean anything. Many people already have
code live in production working fine with the old toolset and will be
unwilling to convert something just because....
Think about it like this a company like mine that already has something in
production. Even if you could convince me us that cql native transport was
better, which by the way no one has showed me a vast performance reason to
this point, they still may not want to invest the resources to convert
their app. Many companies endured the painful transition from Cassandra 0.6
to Cassandra 0.7 conversion and they are not eagerly going to entertain
another change which is mostly cosmetic.
Also I find issues like this extremely frustrating.
https://issues.apache.org/jira/browse/CASSANDRA-4924
It seems like the project is drawing a hard line in the sand dividing
people. Is it the case that cql3's sparse tables can't be accessed
by thrift, or is it the case that no one wants to make this happen? Like is
it technically impossible? It seems not to me in Cassandra
Row key, column, and value are all still byte arrays right? So I do not see
why thrift users need to be locked out of them. Just like composites we
will figure out how to pack the bytes.
I hope that we can stop talking about removing thrift until there is some
consensus between active users that it is not in use anymore.
This consensus is not as simple as n committers saying that something is
technically not needed anymore. It has to look at the users, the number of
clients, the number of languages, the number of high level tools available.
In the mean time when issues like 4924 pop up it would be better if people
tried to find solutions for maximum forward and backward compatibility
instead of drawing a line and trying to shut thrift users out of things.
Avro was much the same way . I had a spirited debate on irc and got
basicallly insulted because i belived thrift was not dead. The glory of
avro never came true because it really did not work for clients outside a
few languages. Cql and the binary transport has to pass this same litmus
test. Let it gain momentum and have rock solid clients for 5 languages and
have higher level tools written on top of it then its easy to say thrift is
not needed anymore.
On Saturday, December 1, 2012, Sylvain Lebresne wrote:
> I agree on 2.0.
>
> For the thrift part, we've said clearly that we wouldn't remove it any time
> soon so let's stick to that. Besides, I would agree it's too soon anyway.
> What we can do however in the relatively short term on that front, is to
> pull thrift in it's own jar (we've almost removed all internal dependencies
> on thrift, and the few remaining ones will be easy to kill) and make that
> jar optional if you don't want to use it.
>
> --
> Sylvain
>
>
> On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski <ray.slakinski@gmail.com<javascript:;>
> >wrote:
>
> > I agree, I don't think its a great idea to drop thrift until the back
> > end tools are 100% compatible and have some level of agreement from the
> > major users of
> > Cassandra.
> >
> > Paying off technical dept though I'm all for, and I think its key to the
> > long term success of the application. Right now Supercolumns to someone
> > new coming to the system might think "Hey, these things look great. Lets
> > use them" and in a few months time hate all things that are cassandra.
> >
> > Ray Slakinski
> >
> > On 12/01, Jonathan Ellis wrote:
> > > As attractive as it would be to clean house, I think we owe it to our
> > > users to keep Thrift around for the forseeable future rather than
> > > orphan all Thrift-using applications (which is virtually everyone) on
> > > 1.2.
> > >
> > > On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <jasedbrown@gmail.com<javascript:;>
> >
> > wrote:
> > > > Hi Jonathan,
> > > >
> > > > I'm in favor of paying off the technical debt, as well, and I wonder
> if
> > > > there is value in removing support for thrift with 2.0? We're
> > currently in
> > > > 'do as little as possible' mode with thrift, so should we
> aggressively
> > cast
> > > > it off and push the binary CQL protocol? Seems like a jump to '2.0',
> > along
> > > > with the other initiatives, would be a reasonable time/milestone to
> do
> > so.
> > > >
> > > > Thanks,
> > > >
> > > > -Jason
> > > >
> > > >
> > > > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jbellis@gmail.com<javascript:;>
> >
> > wrote:
> > > >
> > > >> The more I think about it, the more I think we should call 1.2-next,
> > > >> 2.0. I'd like to spend some time paying off our technical debt:
> > > >>
> > > >> - replace supercolumns with composites (CASSANDRA-3237)
> > > >> - rewrite counters (CASSANDRA-4775)
> > > >> - improve storage engine support for wide rows
> > > >> - better stage management to improve latency (disruptor? lightweight
> > > >> threads? custom executor + queue?)
> > > >> - improved repair (CASSANDRA-3362, 2699)
> > > >>
> > > >> Of course, we're planning some new features as well:
> > > >> - triggers (CASSANDRA-1311)
> > > >> - improved query fault tolerance (CASSANDRA-4705)
> > > >> - row size limits (CASSANDRA-3929)
> > > >> - cql3 integration for hadoop (CASSANDRA-4421)
> > > >> - improved caching (CASSANDRA-1956, 2864)
> > > >>
> > > >> --
> > > >> Jonathan Ellis
> > > >> Project Chair, Apache Cassandra
> > > >> co-founder, http://www.datastax.com
> > > >> @spyced
> > > >>
> > >
> > >
> > >
> > > --
> > > Jonathan Ellis
> > > Project Chair, Apache Cassandra
> > > co-founder, http://www.datastax.com
> > > @spyced
> >
>
Re: 2.0
Posted by Sylvain Lebresne <sy...@datastax.com>.
I agree on 2.0.
For the thrift part, we've said clearly that we wouldn't remove it any time
soon so let's stick to that. Besides, I would agree it's too soon anyway.
What we can do however in the relatively short term on that front, is to
pull thrift in it's own jar (we've almost removed all internal dependencies
on thrift, and the few remaining ones will be easy to kill) and make that
jar optional if you don't want to use it.
--
Sylvain
On Sat, Dec 1, 2012 at 2:52 AM, Ray Slakinski <ra...@gmail.com>wrote:
> I agree, I don't think its a great idea to drop thrift until the back
> end tools are 100% compatible and have some level of agreement from the
> major users of
> Cassandra.
>
> Paying off technical dept though I'm all for, and I think its key to the
> long term success of the application. Right now Supercolumns to someone
> new coming to the system might think "Hey, these things look great. Lets
> use them" and in a few months time hate all things that are cassandra.
>
> Ray Slakinski
>
> On 12/01, Jonathan Ellis wrote:
> > As attractive as it would be to clean house, I think we owe it to our
> > users to keep Thrift around for the forseeable future rather than
> > orphan all Thrift-using applications (which is virtually everyone) on
> > 1.2.
> >
> > On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com>
> wrote:
> > > Hi Jonathan,
> > >
> > > I'm in favor of paying off the technical debt, as well, and I wonder if
> > > there is value in removing support for thrift with 2.0? We're
> currently in
> > > 'do as little as possible' mode with thrift, so should we aggressively
> cast
> > > it off and push the binary CQL protocol? Seems like a jump to '2.0',
> along
> > > with the other initiatives, would be a reasonable time/milestone to do
> so.
> > >
> > > Thanks,
> > >
> > > -Jason
> > >
> > >
> > > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com>
> wrote:
> > >
> > >> The more I think about it, the more I think we should call 1.2-next,
> > >> 2.0. I'd like to spend some time paying off our technical debt:
> > >>
> > >> - replace supercolumns with composites (CASSANDRA-3237)
> > >> - rewrite counters (CASSANDRA-4775)
> > >> - improve storage engine support for wide rows
> > >> - better stage management to improve latency (disruptor? lightweight
> > >> threads? custom executor + queue?)
> > >> - improved repair (CASSANDRA-3362, 2699)
> > >>
> > >> Of course, we're planning some new features as well:
> > >> - triggers (CASSANDRA-1311)
> > >> - improved query fault tolerance (CASSANDRA-4705)
> > >> - row size limits (CASSANDRA-3929)
> > >> - cql3 integration for hadoop (CASSANDRA-4421)
> > >> - improved caching (CASSANDRA-1956, 2864)
> > >>
> > >> --
> > >> Jonathan Ellis
> > >> Project Chair, Apache Cassandra
> > >> co-founder, http://www.datastax.com
> > >> @spyced
> > >>
> >
> >
> >
> > --
> > Jonathan Ellis
> > Project Chair, Apache Cassandra
> > co-founder, http://www.datastax.com
> > @spyced
>
Re: Hector 0.8.0-2 update fails with : "All host pools marked down.
Retry burden pushed out to client."
Posted by Radim Kolar <hs...@filez.com>.
Dne 3.12.2012 9:15, Bisht, Jaikrit napsal(a):
> Hi there,
>
> What have been the problems with Hector?
problems with improper detection of down nodes
problems with improper detection of timeouts
some lost updates due to bad timestamp generation, spliting into more
mutators helped.
lack of support for all data types
RE: Hector 0.8.0-2 update fails with : "All host pools marked
down. Retry burden pushed out to client."
Posted by "Bisht, Jaikrit" <bi...@visa.com>.
Hi there,
What have been the problems with Hector? I assume you are using Java client, so which client are you using now to use Cassandra?
Regards
Jay
-----Original Message-----
From: Radim Kolar [mailto:hsn@filez.com]
Sent: 01 December 2012 19:06
To: dev@cassandra.apache.org
Subject: Re: Hector 0.8.0-2 update fails with : "All host pools marked down. Retry burden pushed out to client."
after 1 year experience with Hector, i would recommend: stay away from
hector if possible.
VISA EUROPE is a technology business that provides the brand, systems, services and rules that make electronic payments between millions of consumers, retailers, businesses and governments happen. Visa Europe is a membership association of more than 3,700 members that includes banks and other payment service providers from 36 countries across Europe. We continually invest and innovate to create new and better ways to pay and be paid. For more information, please visit www.visaeurope.com.
Please consider the environment before printing this email.
This email (including attachments) is confidential and is solely intended for the addressee. Unless you are the addressee, you may not read, use or store this email in any way, or permit others to. If you have received it in error, please contact Visa Europe on +44 (0)20 7937 8111.
Visa Europe Services Inc. is a company incorporated in Delaware USA, acting through its UK Establishment (UK Establishment number BR007632) whose registered office is at 1 Sheldon Square, London, W2 6TT.
Re: Hector 0.8.0-2 update fails with : "All host pools marked down.
Retry burden pushed out to client."
Posted by Radim Kolar <hs...@filez.com>.
after 1 year experience with Hector, i would recommend: stay away from
hector if possible.
Hector 0.8.0-2 update fails with : "All host pools marked down.
Retry burden pushed out to client."
Posted by "Bisht, Jaikrit" <bi...@visa.com>.
Hi,
I am working on a POC for cassandra 1.1.6 with hector 0.8.0-2.
It fails on a update with the following exception. Anyone has seen this error before? Please could you help..
Hector throws an HectorException: Exception message : "All host pools marked down. Retry burden pushed out to client."
java.lang.NullPointerException
at me.prettyprint.cassandra.model.MutatorImpl.execute(MutatorImpl.java:217)
at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeBatch(AbstractColumnFamilyTemplate.java:127)
at me.prettyprint.cassandra.service.template.AbstractColumnFamilyTemplate.executeIfNotBatched(AbstractColumnFamilyTemplate.java:162)
at me.prettyprint.cassandra.service.template.SuperCfTemplate.update(SuperCfTemplate.java:198)
at com.visa.shared.cache.poc.dao.AbstractSuperColumnDataAccess.update(AbstractSuperColumnDataAccess.java:188)
at com.visa.shared.cache.poc.domain.InterfaceStationProcessor.markLinkUp(InterfaceStationProcessor.java:47)
at com.visa.shared.cache.poc.domain.InterfaceStationProcessor.processRequest(InterfaceStationProcessor.java:78)
at com.visa.shared.cache.poc.domain.SwitchService.processRequest(SwitchService.java:78)
at com.visa.shared.cache.poc.domain.SwitchService.access$000(SwitchService.java:13)
at com.visa.shared.cache.poc.domain.SwitchService$1.run(SwitchService.java:59)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Exception in thread "switchServiceTaskExceutor-1242" java.lang.NullPointerException
at me.prettyprint.cassandra.model.MutatorImpl.addInsertion(MutatorImpl.java:158)
at me.prettyprint.cassandra.service.template.SuperCfUpdater.updateInternal(SuperCfUpdater.java:115)
at me.prettyprint.cassandra.service.template.SuperCfTemplate.update(SuperCfTemplate.java:196)
Cassandra logs in /var/log/cassandra/system.log:
INFO [FlushWriter:8] 2012-12-01 17:01:13,200 Memtable.java (line 264) Writing Memtable-jpp_0001@1896150523(17573964/21967455 serialized/live bytes, 1055 ops)
ERROR [Thrift:1030] 2012-12-01 17:01:14,324 CustomTThreadPoolServer.java (line 200) Thrift error occurred during processing of message.
org.apache.thrift.TException: Negative length: -2147418111
at org.apache.thrift.protocol.TBinaryProtocol.checkReadLength(TBinaryProtocol.java:388)
at org.apache.thrift.protocol.TBinaryProtocol.readBinary(TBinaryProtocol.java:363)
at org.apache.cassandra.thrift.Cassandra$batch_mutate_args.read(Cassandra.java:19724)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:21)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:34)
at org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:186)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
Regards
Jay
VISA EUROPE is a technology business that provides the brand, systems, services and rules that make electronic payments between millions of consumers, retailers, businesses and governments happen. Visa Europe is a membership association of more than 3,700 members that includes banks and other payment service providers from 36 countries across Europe. We continually invest and innovate to create new and better ways to pay and be paid. For more information, please visit www.visaeurope.com.
Please consider the environment before printing this email.
This email (including attachments) is confidential and is solely intended for the addressee. Unless you are the addressee, you may not read, use or store this email in any way, or permit others to. If you have received it in error, please contact Visa Europe on +44 (0)20 7937 8111.
Visa Europe Services Inc. is a company incorporated in Delaware USA, acting through its UK Establishment (UK Establishment number BR007632) whose registered office is at 1 Sheldon Square, London, W2 6TT.
Re: 2.0
Posted by Ray Slakinski <ra...@gmail.com>.
I agree, I don't think its a great idea to drop thrift until the back
end tools are 100% compatible and have some level of agreement from the major users of
Cassandra.
Paying off technical dept though I'm all for, and I think its key to the
long term success of the application. Right now Supercolumns to someone
new coming to the system might think "Hey, these things look great. Lets
use them" and in a few months time hate all things that are cassandra.
Ray Slakinski
On 12/01, Jonathan Ellis wrote:
> As attractive as it would be to clean house, I think we owe it to our
> users to keep Thrift around for the forseeable future rather than
> orphan all Thrift-using applications (which is virtually everyone) on
> 1.2.
>
> On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com> wrote:
> > Hi Jonathan,
> >
> > I'm in favor of paying off the technical debt, as well, and I wonder if
> > there is value in removing support for thrift with 2.0? We're currently in
> > 'do as little as possible' mode with thrift, so should we aggressively cast
> > it off and push the binary CQL protocol? Seems like a jump to '2.0', along
> > with the other initiatives, would be a reasonable time/milestone to do so.
> >
> > Thanks,
> >
> > -Jason
> >
> >
> > On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> >
> >> The more I think about it, the more I think we should call 1.2-next,
> >> 2.0. I'd like to spend some time paying off our technical debt:
> >>
> >> - replace supercolumns with composites (CASSANDRA-3237)
> >> - rewrite counters (CASSANDRA-4775)
> >> - improve storage engine support for wide rows
> >> - better stage management to improve latency (disruptor? lightweight
> >> threads? custom executor + queue?)
> >> - improved repair (CASSANDRA-3362, 2699)
> >>
> >> Of course, we're planning some new features as well:
> >> - triggers (CASSANDRA-1311)
> >> - improved query fault tolerance (CASSANDRA-4705)
> >> - row size limits (CASSANDRA-3929)
> >> - cql3 integration for hadoop (CASSANDRA-4421)
> >> - improved caching (CASSANDRA-1956, 2864)
> >>
> >> --
> >> Jonathan Ellis
> >> Project Chair, Apache Cassandra
> >> co-founder, http://www.datastax.com
> >> @spyced
> >>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
Re: 2.0
Posted by Jonathan Ellis <jb...@gmail.com>.
As attractive as it would be to clean house, I think we owe it to our
users to keep Thrift around for the forseeable future rather than
orphan all Thrift-using applications (which is virtually everyone) on
1.2.
On Sat, Dec 1, 2012 at 7:33 AM, Jason Brown <ja...@gmail.com> wrote:
> Hi Jonathan,
>
> I'm in favor of paying off the technical debt, as well, and I wonder if
> there is value in removing support for thrift with 2.0? We're currently in
> 'do as little as possible' mode with thrift, so should we aggressively cast
> it off and push the binary CQL protocol? Seems like a jump to '2.0', along
> with the other initiatives, would be a reasonable time/milestone to do so.
>
> Thanks,
>
> -Jason
>
>
> On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
>
>> The more I think about it, the more I think we should call 1.2-next,
>> 2.0. I'd like to spend some time paying off our technical debt:
>>
>> - replace supercolumns with composites (CASSANDRA-3237)
>> - rewrite counters (CASSANDRA-4775)
>> - improve storage engine support for wide rows
>> - better stage management to improve latency (disruptor? lightweight
>> threads? custom executor + queue?)
>> - improved repair (CASSANDRA-3362, 2699)
>>
>> Of course, we're planning some new features as well:
>> - triggers (CASSANDRA-1311)
>> - improved query fault tolerance (CASSANDRA-4705)
>> - row size limits (CASSANDRA-3929)
>> - cql3 integration for hadoop (CASSANDRA-4421)
>> - improved caching (CASSANDRA-1956, 2864)
>>
>> --
>> Jonathan Ellis
>> Project Chair, Apache Cassandra
>> co-founder, http://www.datastax.com
>> @spyced
>>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced
Re: 2.0
Posted by Jason Brown <ja...@gmail.com>.
Hi Jonathan,
I'm in favor of paying off the technical debt, as well, and I wonder if
there is value in removing support for thrift with 2.0? We're currently in
'do as little as possible' mode with thrift, so should we aggressively cast
it off and push the binary CQL protocol? Seems like a jump to '2.0', along
with the other initiatives, would be a reasonable time/milestone to do so.
Thanks,
-Jason
On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> The more I think about it, the more I think we should call 1.2-next,
> 2.0. I'd like to spend some time paying off our technical debt:
>
> - replace supercolumns with composites (CASSANDRA-3237)
> - rewrite counters (CASSANDRA-4775)
> - improve storage engine support for wide rows
> - better stage management to improve latency (disruptor? lightweight
> threads? custom executor + queue?)
> - improved repair (CASSANDRA-3362, 2699)
>
> Of course, we're planning some new features as well:
> - triggers (CASSANDRA-1311)
> - improved query fault tolerance (CASSANDRA-4705)
> - row size limits (CASSANDRA-3929)
> - cql3 integration for hadoop (CASSANDRA-4421)
> - improved caching (CASSANDRA-1956, 2864)
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>
Re: 2.0
Posted by Jonathan Ellis <jb...@gmail.com>.
With no objections I have renamed 1.3 to 2.0 in JIRA.
On Fri, Nov 30, 2012 at 2:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> The more I think about it, the more I think we should call 1.2-next,
> 2.0. I'd like to spend some time paying off our technical debt:
>
> - replace supercolumns with composites (CASSANDRA-3237)
> - rewrite counters (CASSANDRA-4775)
> - improve storage engine support for wide rows
> - better stage management to improve latency (disruptor? lightweight
> threads? custom executor + queue?)
> - improved repair (CASSANDRA-3362, 2699)
>
> Of course, we're planning some new features as well:
> - triggers (CASSANDRA-1311)
> - improved query fault tolerance (CASSANDRA-4705)
> - row size limits (CASSANDRA-3929)
> - cql3 integration for hadoop (CASSANDRA-4421)
> - improved caching (CASSANDRA-1956, 2864)
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder, http://www.datastax.com
@spyced
Re: 2.0
Posted by Vijay <vi...@gmail.com>.
+1
Regards,
</VJ>
On Fri, Nov 30, 2012 at 12:12 PM, Jonathan Ellis <jb...@gmail.com> wrote:
> The more I think about it, the more I think we should call 1.2-next,
> 2.0. I'd like to spend some time paying off our technical debt:
>
> - replace supercolumns with composites (CASSANDRA-3237)
> - rewrite counters (CASSANDRA-4775)
> - improve storage engine support for wide rows
> - better stage management to improve latency (disruptor? lightweight
> threads? custom executor + queue?)
> - improved repair (CASSANDRA-3362, 2699)
>
> Of course, we're planning some new features as well:
> - triggers (CASSANDRA-1311)
> - improved query fault tolerance (CASSANDRA-4705)
> - row size limits (CASSANDRA-3929)
> - cql3 integration for hadoop (CASSANDRA-4421)
> - improved caching (CASSANDRA-1956, 2864)
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder, http://www.datastax.com
> @spyced
>