You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@cassandra.apache.org by Joseph Stein <cr...@gmail.com> on 2011/06/28 01:13:08 UTC
CounterColumn as a double
So has anyone considered using the CounterColumn for summing?
I wanted to-do this over the weekend until I realized it was only a long :(
so using it for things like duration (as an example for me this would have
been great to keep track of aggregate durations of ad impressions) are not
possible (or total costs when processing business workflows, etc,etc).
I thought this might be a little more the speed of a first contribution too
:) and also helps out with more functionality since a lot of real time
analytics will need double.
Let me know, I think it is a good feature.
Implementing it not sure we would want to break the thrift interface I would
suggest that I would create another interface for the double value?
Under the hood of the thrift interface I was thinking of creating a
CounterValue class and then setting the lValue or the dValue depending on
which thrift function was called. I can update the thrift, add a sister
function and re-work the entire code path of long CounterColumn.value into
CounterValue CounterColumn.value.
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
unsubscribe
Posted by 김준영 <ju...@naver.com>.
unsubscribe
Re: CounterColumn as a double
Posted by David Boxenhorn <da...@citypath.com>.
How about BigDoubles and BigIntegers?
On Tue, Jun 28, 2011 at 3:16 AM, Jonathan Ellis <jb...@gmail.com> wrote:
> I don't think you can avoid that.
>
> I'd suggest making it CQL-only if we do doubles -- no backwards
> incompatibility required there.
>
> On Mon, Jun 27, 2011 at 7:07 PM, Jason <jf...@gmail.com> wrote:
>> Sorry, I should have been more clear; I was speaking to the question of how to avoid modifying the thrift interface.
>>
>> - Jason
>>
>> On Jun 27, 2011, at 7:58 PM, Joseph Stein <cr...@gmail.com> wrote:
>>
>>> hmmm, well Jason it is not as accurate as I would have thought at first and
>>> the increments on the long are whacked (which now that I think about it more
>>> makes sense since a +1 of the bits as long for the double would not
>>> necessarly represent the +1 on the double).
>>>
>>> So I am setting the increment to be
>>>
>>> 1.23
>>>
>>> which comes back as 1.3213044541238036E308
>>>
>>> and then another increment of 1.23 comes back as 10.359999999999987
>>>
>>> so for the increment (which is just +value to the long which would not
>>> provide the right shift)
>>>
>>> even if under the hood I keep the thrift interface as long somehow the 1.23
>>> vs 1.321 is a big difference when you have billions of them
>>>
>>> so I think it goes back to my original idea/proposition. Any one have
>>> issue? any +1 ? should I create a JIRA and get to it?
>>>
>>> On Mon, Jun 27, 2011 at 7:39 PM, Joseph Stein <cr...@gmail.com> wrote:
>>>
>>>> I will give that a shot, seems that it will work fantastically, thanks!
>>>>
>>>> I will keep trolling JIRA then for something I feel I can get my feet wet
>>>> with and contribute then.
>>>>
>>>> On Mon, Jun 27, 2011 at 7:33 PM, Jason Fager <jf...@gmail.com> wrote:
>>>>
>>>>> Longs and Doubles are both 64-bit values and are pretty easily
>>>>> convertible. Check out Double.doubleToLongBits and
>>>>> Double.longBitsToDouble in the JDK; you can also read more about the
>>>>> details of the conversion and get some pointers to some code in a post
>>>>> I wrote last year:
>>>>> http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
>>>>> is on using doubles in key strings, but it should cover what you
>>>>> need).
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
>>>>>> So has anyone considered using the CounterColumn for summing?
>>>>>>
>>>>>> I wanted to-do this over the weekend until I realized it was only a long
>>>>> :(
>>>>>> so using it for things like duration (as an example for me this would
>>>>> have
>>>>>> been great to keep track of aggregate durations of ad impressions) are
>>>>> not
>>>>>> possible (or total costs when processing business workflows, etc,etc).
>>>>>>
>>>>>> I thought this might be a little more the speed of a first contribution
>>>>> too
>>>>>> :) and also helps out with more functionality since a lot of real time
>>>>>> analytics will need double.
>>>>>>
>>>>>> Let me know, I think it is a good feature.
>>>>>>
>>>>>> Implementing it not sure we would want to break the thrift interface I
>>>>> would
>>>>>> suggest that I would create another interface for the double value?
>>>>>>
>>>>>> Under the hood of the thrift interface I was thinking of creating a
>>>>>> CounterValue class and then setting the lValue or the dValue depending
>>>>> on
>>>>>> which thrift function was called. I can update the thrift, add a sister
>>>>>> function and re-work the entire code path of long CounterColumn.value
>>>>> into
>>>>>> CounterValue CounterColumn.value.
>>>>>>
>>>>>> /*
>>>>>> Joe Stein
>>>>>> http://www.linkedin.com/in/charmalloc
>>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>>>>> */
>>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> /*
>>>> Joe Stein
>>>> http://www.linkedin.com/in/charmalloc
>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>>> */
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> /*
>>> Joe Stein
>>> http://www.linkedin.com/in/charmalloc
>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> */
>>
>
>
>
> --
> Jonathan Ellis
> Project Chair, Apache Cassandra
> co-founder of DataStax, the source for professional Cassandra support
> http://www.datastax.com
>
Re: CounterColumn as a double
Posted by Jonathan Ellis <jb...@gmail.com>.
I don't think you can avoid that.
I'd suggest making it CQL-only if we do doubles -- no backwards
incompatibility required there.
On Mon, Jun 27, 2011 at 7:07 PM, Jason <jf...@gmail.com> wrote:
> Sorry, I should have been more clear; I was speaking to the question of how to avoid modifying the thrift interface.
>
> - Jason
>
> On Jun 27, 2011, at 7:58 PM, Joseph Stein <cr...@gmail.com> wrote:
>
>> hmmm, well Jason it is not as accurate as I would have thought at first and
>> the increments on the long are whacked (which now that I think about it more
>> makes sense since a +1 of the bits as long for the double would not
>> necessarly represent the +1 on the double).
>>
>> So I am setting the increment to be
>>
>> 1.23
>>
>> which comes back as 1.3213044541238036E308
>>
>> and then another increment of 1.23 comes back as 10.359999999999987
>>
>> so for the increment (which is just +value to the long which would not
>> provide the right shift)
>>
>> even if under the hood I keep the thrift interface as long somehow the 1.23
>> vs 1.321 is a big difference when you have billions of them
>>
>> so I think it goes back to my original idea/proposition. Any one have
>> issue? any +1 ? should I create a JIRA and get to it?
>>
>> On Mon, Jun 27, 2011 at 7:39 PM, Joseph Stein <cr...@gmail.com> wrote:
>>
>>> I will give that a shot, seems that it will work fantastically, thanks!
>>>
>>> I will keep trolling JIRA then for something I feel I can get my feet wet
>>> with and contribute then.
>>>
>>> On Mon, Jun 27, 2011 at 7:33 PM, Jason Fager <jf...@gmail.com> wrote:
>>>
>>>> Longs and Doubles are both 64-bit values and are pretty easily
>>>> convertible. Check out Double.doubleToLongBits and
>>>> Double.longBitsToDouble in the JDK; you can also read more about the
>>>> details of the conversion and get some pointers to some code in a post
>>>> I wrote last year:
>>>> http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
>>>> is on using doubles in key strings, but it should cover what you
>>>> need).
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
>>>>> So has anyone considered using the CounterColumn for summing?
>>>>>
>>>>> I wanted to-do this over the weekend until I realized it was only a long
>>>> :(
>>>>> so using it for things like duration (as an example for me this would
>>>> have
>>>>> been great to keep track of aggregate durations of ad impressions) are
>>>> not
>>>>> possible (or total costs when processing business workflows, etc,etc).
>>>>>
>>>>> I thought this might be a little more the speed of a first contribution
>>>> too
>>>>> :) and also helps out with more functionality since a lot of real time
>>>>> analytics will need double.
>>>>>
>>>>> Let me know, I think it is a good feature.
>>>>>
>>>>> Implementing it not sure we would want to break the thrift interface I
>>>> would
>>>>> suggest that I would create another interface for the double value?
>>>>>
>>>>> Under the hood of the thrift interface I was thinking of creating a
>>>>> CounterValue class and then setting the lValue or the dValue depending
>>>> on
>>>>> which thrift function was called. I can update the thrift, add a sister
>>>>> function and re-work the entire code path of long CounterColumn.value
>>>> into
>>>>> CounterValue CounterColumn.value.
>>>>>
>>>>> /*
>>>>> Joe Stein
>>>>> http://www.linkedin.com/in/charmalloc
>>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>>>> */
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> /*
>>> Joe Stein
>>> http://www.linkedin.com/in/charmalloc
>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>> */
>>>
>>
>>
>>
>> --
>>
>> /*
>> Joe Stein
>> http://www.linkedin.com/in/charmalloc
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> */
>
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
Re: CounterColumn as a double
Posted by Jason <jf...@gmail.com>.
Sorry, I should have been more clear; I was speaking to the question of how to avoid modifying the thrift interface.
- Jason
On Jun 27, 2011, at 7:58 PM, Joseph Stein <cr...@gmail.com> wrote:
> hmmm, well Jason it is not as accurate as I would have thought at first and
> the increments on the long are whacked (which now that I think about it more
> makes sense since a +1 of the bits as long for the double would not
> necessarly represent the +1 on the double).
>
> So I am setting the increment to be
>
> 1.23
>
> which comes back as 1.3213044541238036E308
>
> and then another increment of 1.23 comes back as 10.359999999999987
>
> so for the increment (which is just +value to the long which would not
> provide the right shift)
>
> even if under the hood I keep the thrift interface as long somehow the 1.23
> vs 1.321 is a big difference when you have billions of them
>
> so I think it goes back to my original idea/proposition. Any one have
> issue? any +1 ? should I create a JIRA and get to it?
>
> On Mon, Jun 27, 2011 at 7:39 PM, Joseph Stein <cr...@gmail.com> wrote:
>
>> I will give that a shot, seems that it will work fantastically, thanks!
>>
>> I will keep trolling JIRA then for something I feel I can get my feet wet
>> with and contribute then.
>>
>> On Mon, Jun 27, 2011 at 7:33 PM, Jason Fager <jf...@gmail.com> wrote:
>>
>>> Longs and Doubles are both 64-bit values and are pretty easily
>>> convertible. Check out Double.doubleToLongBits and
>>> Double.longBitsToDouble in the JDK; you can also read more about the
>>> details of the conversion and get some pointers to some code in a post
>>> I wrote last year:
>>> http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
>>> is on using doubles in key strings, but it should cover what you
>>> need).
>>>
>>>
>>>
>>>
>>>
>>> On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
>>>> So has anyone considered using the CounterColumn for summing?
>>>>
>>>> I wanted to-do this over the weekend until I realized it was only a long
>>> :(
>>>> so using it for things like duration (as an example for me this would
>>> have
>>>> been great to keep track of aggregate durations of ad impressions) are
>>> not
>>>> possible (or total costs when processing business workflows, etc,etc).
>>>>
>>>> I thought this might be a little more the speed of a first contribution
>>> too
>>>> :) and also helps out with more functionality since a lot of real time
>>>> analytics will need double.
>>>>
>>>> Let me know, I think it is a good feature.
>>>>
>>>> Implementing it not sure we would want to break the thrift interface I
>>> would
>>>> suggest that I would create another interface for the double value?
>>>>
>>>> Under the hood of the thrift interface I was thinking of creating a
>>>> CounterValue class and then setting the lValue or the dValue depending
>>> on
>>>> which thrift function was called. I can update the thrift, add a sister
>>>> function and re-work the entire code path of long CounterColumn.value
>>> into
>>>> CounterValue CounterColumn.value.
>>>>
>>>> /*
>>>> Joe Stein
>>>> http://www.linkedin.com/in/charmalloc
>>>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>>>> */
>>>>
>>>
>>
>>
>>
>> --
>>
>> /*
>> Joe Stein
>> http://www.linkedin.com/in/charmalloc
>> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> */
>>
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
Re: CounterColumn as a double
Posted by Joseph Stein <cr...@gmail.com>.
hmmm, well Jason it is not as accurate as I would have thought at first and
the increments on the long are whacked (which now that I think about it more
makes sense since a +1 of the bits as long for the double would not
necessarly represent the +1 on the double).
So I am setting the increment to be
1.23
which comes back as 1.3213044541238036E308
and then another increment of 1.23 comes back as 10.359999999999987
so for the increment (which is just +value to the long which would not
provide the right shift)
even if under the hood I keep the thrift interface as long somehow the 1.23
vs 1.321 is a big difference when you have billions of them
so I think it goes back to my original idea/proposition. Any one have
issue? any +1 ? should I create a JIRA and get to it?
On Mon, Jun 27, 2011 at 7:39 PM, Joseph Stein <cr...@gmail.com> wrote:
> I will give that a shot, seems that it will work fantastically, thanks!
>
> I will keep trolling JIRA then for something I feel I can get my feet wet
> with and contribute then.
>
> On Mon, Jun 27, 2011 at 7:33 PM, Jason Fager <jf...@gmail.com> wrote:
>
>> Longs and Doubles are both 64-bit values and are pretty easily
>> convertible. Check out Double.doubleToLongBits and
>> Double.longBitsToDouble in the JDK; you can also read more about the
>> details of the conversion and get some pointers to some code in a post
>> I wrote last year:
>> http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
>> is on using doubles in key strings, but it should cover what you
>> need).
>>
>>
>>
>>
>>
>> On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
>> > So has anyone considered using the CounterColumn for summing?
>> >
>> > I wanted to-do this over the weekend until I realized it was only a long
>> :(
>> > so using it for things like duration (as an example for me this would
>> have
>> > been great to keep track of aggregate durations of ad impressions) are
>> not
>> > possible (or total costs when processing business workflows, etc,etc).
>> >
>> > I thought this might be a little more the speed of a first contribution
>> too
>> > :) and also helps out with more functionality since a lot of real time
>> > analytics will need double.
>> >
>> > Let me know, I think it is a good feature.
>> >
>> > Implementing it not sure we would want to break the thrift interface I
>> would
>> > suggest that I would create another interface for the double value?
>> >
>> > Under the hood of the thrift interface I was thinking of creating a
>> > CounterValue class and then setting the lValue or the dValue depending
>> on
>> > which thrift function was called. I can update the thrift, add a sister
>> > function and re-work the entire code path of long CounterColumn.value
>> into
>> > CounterValue CounterColumn.value.
>> >
>> > /*
>> > Joe Stein
>> > http://www.linkedin.com/in/charmalloc
>> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
>> > */
>> >
>>
>
>
>
> --
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: CounterColumn as a double
Posted by Joseph Stein <cr...@gmail.com>.
I will give that a shot, seems that it will work fantastically, thanks!
I will keep trolling JIRA then for something I feel I can get my feet wet
with and contribute then.
On Mon, Jun 27, 2011 at 7:33 PM, Jason Fager <jf...@gmail.com> wrote:
> Longs and Doubles are both 64-bit values and are pretty easily
> convertible. Check out Double.doubleToLongBits and
> Double.longBitsToDouble in the JDK; you can also read more about the
> details of the conversion and get some pointers to some code in a post
> I wrote last year:
> http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
> is on using doubles in key strings, but it should cover what you
> need).
>
>
>
>
>
> On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
> > So has anyone considered using the CounterColumn for summing?
> >
> > I wanted to-do this over the weekend until I realized it was only a long
> :(
> > so using it for things like duration (as an example for me this would
> have
> > been great to keep track of aggregate durations of ad impressions) are
> not
> > possible (or total costs when processing business workflows, etc,etc).
> >
> > I thought this might be a little more the speed of a first contribution
> too
> > :) and also helps out with more functionality since a lot of real time
> > analytics will need double.
> >
> > Let me know, I think it is a good feature.
> >
> > Implementing it not sure we would want to break the thrift interface I
> would
> > suggest that I would create another interface for the double value?
> >
> > Under the hood of the thrift interface I was thinking of creating a
> > CounterValue class and then setting the lValue or the dValue depending on
> > which thrift function was called. I can update the thrift, add a sister
> > function and re-work the entire code path of long CounterColumn.value
> into
> > CounterValue CounterColumn.value.
> >
> > /*
> > Joe Stein
> > http://www.linkedin.com/in/charmalloc
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > */
> >
>
--
/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/
Re: CounterColumn as a double
Posted by Jason Fager <jf...@gmail.com>.
Longs and Doubles are both 64-bit values and are pretty easily
convertible. Check out Double.doubleToLongBits and
Double.longBitsToDouble in the JDK; you can also read more about the
details of the conversion and get some pointers to some code in a post
I wrote last year:
http://jasonfager.com/770-lexi-sortable-number-strings/ (the emphasis
is on using doubles in key strings, but it should cover what you
need).
On Mon, Jun 27, 2011 at 7:13 PM, Joseph Stein <cr...@gmail.com> wrote:
> So has anyone considered using the CounterColumn for summing?
>
> I wanted to-do this over the weekend until I realized it was only a long :(
> so using it for things like duration (as an example for me this would have
> been great to keep track of aggregate durations of ad impressions) are not
> possible (or total costs when processing business workflows, etc,etc).
>
> I thought this might be a little more the speed of a first contribution too
> :) and also helps out with more functionality since a lot of real time
> analytics will need double.
>
> Let me know, I think it is a good feature.
>
> Implementing it not sure we would want to break the thrift interface I would
> suggest that I would create another interface for the double value?
>
> Under the hood of the thrift interface I was thinking of creating a
> CounterValue class and then setting the lValue or the dValue depending on
> which thrift function was called. I can update the thrift, add a sister
> function and re-work the entire code path of long CounterColumn.value into
> CounterValue CounterColumn.value.
>
> /*
> Joe Stein
> http://www.linkedin.com/in/charmalloc
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> */
>