You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Joe Stein (JIRA)" <ji...@apache.org> on 2011/06/28 03:11:17 UTC
[jira] [Created] (CASSANDRA-2833) CounterColumn should have an
optional binary field so that double can be incremented/decremented along
with long
CounterColumn should have an optional binary field so that double can be incremented/decremented along with long
----------------------------------------------------------------------------------------------------------------
Key: CASSANDRA-2833
URL: https://issues.apache.org/jira/browse/CASSANDRA-2833
Project: Cassandra
Issue Type: Improvement
Components: Core
Reporter: Joe Stein
Currently CounterColumn only has a long making it not feasible to track increment/decrement of durations or other values common to analytics represented as a double
The change I am proposing to implement, after some discussions/advice in the irc to issues raised, is to add a new optional binary field to CounterColumn (thrift). I was thinking we could call it *operand*
Under the hood (src/java/org/apache/cassandra/db/CounterColumn.java) I would handle things with byte[] moving between long and double as internal helper functions with case switch on type of operand we are setting.... might also need an optional enum for type perhaps too so the client can let the server know how it should materialize the bytes for when it += the value stored
The clients should continue to work as expected and folks looking to use this can just do so.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2833) CounterColumn should have an
optional binary field so that double can be incremented/decremented along
with long
Posted by "Rick Shaw (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056522#comment-13056522 ]
Rick Shaw commented on CASSANDRA-2833:
--------------------------------------
A wrinkle that might be worth consideration is that the JDBC driver will eventually need an {{AbstractType}} implementation of {{BigDecimal}} ({{NumericType}}?) in order to complete it data type support in {{PreparedStatement}} and {{ResultSet}}.
This data type represents arbitrary-precision signed decimal numbers in a generalized way. It has attributes similar to {{BigInteger}} ({{IntegerType}}) It is really a {{BigInteger}} representing the unscaled part with a long representing the scaling.
This may be a better alternative than Double as a generalized counter type targeted at analytics data.
> CounterColumn should have an optional binary field so that double can be incremented/decremented along with long
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2833
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2833
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Joe Stein
>
> Currently CounterColumn only has a long making it not feasible to track increment/decrement of durations or other values common to analytics represented as a double
> The change I am proposing to implement, after some discussions/advice in the irc to issues raised, is to add a new optional binary field to CounterColumn (thrift). I was thinking we could call it *operand*
> Under the hood (src/java/org/apache/cassandra/db/CounterColumn.java) I would handle things with byte[] moving between long and double as internal helper functions with case switch on type of operand we are setting.... might also need an optional enum for type perhaps too so the client can let the server know how it should materialize the bytes for when it += the value stored
> The clients should continue to work as expected and folks looking to use this can just do so.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2833) CounterColumn should have an
optional binary field so that double can be incremented/decremented along
with long
Posted by "Sylvain Lebresne (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056425#comment-13056425 ]
Sylvain Lebresne commented on CASSANDRA-2833:
---------------------------------------------
Do people really still use double nowadays ?! :)
I mean I'll admit that I don't see right away why you can't use long to track durations or "other values common to analytics". Don't get me wrong, that may require the client to multiply its values by some power of 10, but *not feasible* seems a bit of a strong word to me. I guess what I'm saying is that I think we should avoid being in the "let's have it because it sounds cool" (a.k.a feature creep) and make sure we stay in "let's have it because it is generally useful and solve a problem that can't be solved otherwise easily" land. And I'm not saying that adding double support is of the former kind, but I guess I'm not yet convinced it is completely of the latter kind either.
Initial ranting being done, this is technically doable and fairly easily so. This will add a bit of clutter internally (imho, you'd want to add a DoubleCounterType (or RealCounterType so people don't think it doubles the value each time) and subclass/refactor CounterContext somehow) but probably not too much.
> CounterColumn should have an optional binary field so that double can be incremented/decremented along with long
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2833
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2833
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Joe Stein
>
> Currently CounterColumn only has a long making it not feasible to track increment/decrement of durations or other values common to analytics represented as a double
> The change I am proposing to implement, after some discussions/advice in the irc to issues raised, is to add a new optional binary field to CounterColumn (thrift). I was thinking we could call it *operand*
> Under the hood (src/java/org/apache/cassandra/db/CounterColumn.java) I would handle things with byte[] moving between long and double as internal helper functions with case switch on type of operand we are setting.... might also need an optional enum for type perhaps too so the client can let the server know how it should materialize the bytes for when it += the value stored
> The clients should continue to work as expected and folks looking to use this can just do so.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (CASSANDRA-2833) CounterColumn should have an
optional binary field so that double can be incremented/decremented along
with long
Posted by "Joe Stein (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/CASSANDRA-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13056513#comment-13056513 ]
Joe Stein commented on CASSANDRA-2833:
--------------------------------------
the phrasing *not feasible* was a bit strong.
In general I would argue that the up take of a system would want to have the edges smoothed out as it continued so that the flexibility of it being able to have workarounds are fantastic but as it evolved those workarounds can be internalized to the system. This burden is not always completely on the programming side but in essence the development team and having to explain to every developer that touches your system why you are multiplying by 10 and when they need to divide by 10 to get data out in the format they want. This just makes for an unclean implementation, granted functional for now with your nice workaround, by having to-do some type of naming conventions "realPricing, longInventory" so that folks do not incorrectly infer the wrong implementation of the counter in their own work effort.
Not using double as a name anywhere is a really good point that is just going to add possible confusion.
And, yes. floats are used a lot in Analytics in Mobile and Healthcare (at least two I can speak about directly) in addition to Advertising. In mobile the duration of users using application is a metric very much sought after and used and in Healthcare (as would also hold true for lots of other business verticals) when providing analyses of reimbursement between payer and provider. Most often these durations are coupled with a duration count because you do not always have 1:1 duration between uses and times you have that data though this complex type of counter I think representing as 2 columns (one long and one double) so the caller can create their average themselves which is how it is done now.
What do you think about the idea to not upset existing users and support new ones with the thrift CounterColumn additional optional binary operand field? It was not my idea but I think a good one (thanks Stu Hood)?
So the change could then be:
1) create a new CF RealCounterType - this would let the underlying code know to use the double instead of the long
2) change CounterColumn thrift and db accordingly so that old clients still work but new ones can use this new CF RealCounterType
a) either with a optional double realvalue
or
b) optional binary operand
3) CounterContext seems it needs to have the support between long and double in how it wants to pass the data inside for the += in total as well as some overloading of writeElement
looking at the context it seems cleaner to make the double in addition to long rather than storing it in bytes, granted this is the first time I am looking at :) or if we store it in bytes the conversion
If you and others think this is a good change I am happy to start on it granted while it is probably a simple change it will take me some bumps and bruises as I go through it to get it working.
And I do agree that change for the sake of change or "because it is cool" is never a good way to proceed which can be argued for this change, yes. But I am arguing that it is useful
+1
but would like to hear from others so this change is not only for me maybe everyone else is ok the way it is
> CounterColumn should have an optional binary field so that double can be incremented/decremented along with long
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-2833
> URL: https://issues.apache.org/jira/browse/CASSANDRA-2833
> Project: Cassandra
> Issue Type: Improvement
> Components: Core
> Reporter: Joe Stein
>
> Currently CounterColumn only has a long making it not feasible to track increment/decrement of durations or other values common to analytics represented as a double
> The change I am proposing to implement, after some discussions/advice in the irc to issues raised, is to add a new optional binary field to CounterColumn (thrift). I was thinking we could call it *operand*
> Under the hood (src/java/org/apache/cassandra/db/CounterColumn.java) I would handle things with byte[] moving between long and double as internal helper functions with case switch on type of operand we are setting.... might also need an optional enum for type perhaps too so the client can let the server know how it should materialize the bytes for when it += the value stored
> The clients should continue to work as expected and folks looking to use this can just do so.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira