You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@cassandra.apache.org by "Tyler Hobbs (JIRA)" <ji...@apache.org> on 2015/04/21 22:57:00 UTC
[jira] [Created] (CASSANDRA-9224) Figure out a better default float
precision rule for cqlsh
Tyler Hobbs created CASSANDRA-9224:
--------------------------------------
Summary: Figure out a better default float precision rule for cqlsh
Key: CASSANDRA-9224
URL: https://issues.apache.org/jira/browse/CASSANDRA-9224
Project: Cassandra
Issue Type: Improvement
Components: Tools
Reporter: Tyler Hobbs
Assignee: Stefania
Fix For: 3.0, 2.1.5
We currently use a {{DEFAULT_FLOAT_PRECISION}} of 5 in cqlsh with formatting {{'%.*g' % (float_precision, val)}}. In practice, this is way too low. For example, 12345.5 will show up as 123456. Since the float precision is used for cqlsh's COPY TO, it's particularly important that we maintain as much precision as is practical by default.
There are some other tricky considerations, though. If the precision is too high, python will do something like this:
{noformat}
> '%.25g' % (12345.5555555555555555,)
'12345.55555555555474711582'
{noformat}
That's not terrible, but it would be nice to avoid if we can.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)