You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@carbondata.apache.org by ashokblend <gi...@git.apache.org> on 2016/08/25 10:45:48 UTC
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
GitHub user ashokblend opened a pull request:
https://github.com/apache/incubator-carbondata/pull/95
[CARBONDATA-152]Double min max difference compression issue.
1.While compressing maxdifference typecast with proper DataType
2.While writting we follow like
a. diff with max value , say d = max - x
b. multiply with 10^decimal i.e w = d*10^decimal
3.Similarly while reading we should follow like
a. divide written value 10^decimal i.e d = w/10^decimal
b. diff with max i.e x = max -d
4. There is problem in java with double subtraction
i.e double value = 3.141818 - 0.000610
java will result 3.1412080000000002
To avoid this we are converting to BigDecimal and then doing subtraction
5.Changed logic to calculate no of integer after decimal.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ashokblend/incubator-carbondata doubleminmaxcompression
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-carbondata/pull/95.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #95
----
commit 6c1c959a2c75d4e1d297f8ce2ac9056cdacf44c0
Author: Ashok Kumar <as...@gmail.com>
Date: 2016-08-25T10:40:29Z
Double min max difference compression issue.
1.While compressing maxdifference typecast with proper DataType
2.While writting we follow like
a. diff with max value , say d = max - x
b. multiply with 10^decimal i.e w = d*10^decimal
3.Similarly while reading we should follow like
a. divide written value 10^decimal i.e d = w/10^decimal
b. diff with max i.e x = max -d
4. There is problem in java with double subtraction
i.e double value = 3.141818 - 0.000610
java will result 3.1412080000000002
To avoid this we are converting to BigDecimal and then doing subtraction
5.Changed logic to calculate no of integer after decimal.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
Posted by Vimal-Das <gi...@git.apache.org>.
Github user Vimal-Das commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/95#discussion_r76512969
--- Diff: core/src/main/java/org/apache/carbondata/core/datastorage/store/compression/type/UnCompressNonDecimalMaxMinByte.java ---
@@ -93,7 +95,9 @@
if (value[i] == 0) {
vals[i] = maxValue;
} else {
- vals[i] = (maxValue - value[i]) / Math.pow(10, decimalVal);
+ BigDecimal diff = new BigDecimal(Double.toString(value[i] / Math.pow(10, decimalVal)));
--- End diff --
use BigDecimal.valueOf(double) instead of converting to string for performance and dataloss concerns.refer http://www.javaworld.com/article/2073176/caution--double-to-bigdecimal-in-java.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
Posted by Vimal-Das <gi...@git.apache.org>.
Github user Vimal-Das commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/95#discussion_r76512976
--- Diff: core/src/main/java/org/apache/carbondata/core/datastorage/store/compression/type/UnCompressNonDecimalMaxMinFloat.java ---
@@ -98,7 +99,9 @@
if (value[i] == 0) {
vals[i] = maxValue;
} else {
- vals[i] = (maxValue - value[i]) / Math.pow(10, decimal);
+ BigDecimal diff = new BigDecimal(Double.toString(value[i] / Math.pow(10, decimal)));
--- End diff --
use BigDecimal.valueOf(double) instead of converting to string for performance and dataloss concerns.refer http://www.javaworld.com/article/2073176/caution--double-to-bigdecimal-in-java.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
Posted by Vimal-Das <gi...@git.apache.org>.
Github user Vimal-Das commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/95#discussion_r76512974
--- Diff: core/src/main/java/org/apache/carbondata/core/datastorage/store/compression/type/UnCompressNonDecimalMaxMinDefault.java ---
@@ -95,7 +96,9 @@
if (value[i] == 0) {
vals[i] = maxVal;
} else {
- vals[i] = (maxVal - value[i]) / Math.pow(10, decimal);
+ BigDecimal diff = new BigDecimal(Double.toString(value[i] / Math.pow(10, decimal)));
--- End diff --
use BigDecimal.valueOf(double) instead of converting to string for performance and dataloss concerns.refer http://www.javaworld.com/article/2073176/caution--double-to-bigdecimal-in-java.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
Posted by Vimal-Das <gi...@git.apache.org>.
Github user Vimal-Das commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/95#discussion_r76512979
--- Diff: core/src/main/java/org/apache/carbondata/core/datastorage/store/compression/type/UnCompressNonDecimalMaxMinInt.java ---
@@ -93,7 +94,9 @@
if (value[i] == 0) {
vals[i] = maxValue;
} else {
- vals[i] = (maxValue - value[i]) / Math.pow(10, decimal);
+ BigDecimal diff = new BigDecimal(Double.toString(value[i] / Math.pow(10, decimal)));
--- End diff --
use BigDecimal.valueOf(double) instead of converting to string for performance and dataloss concerns.refer http://www.javaworld.com/article/2073176/caution--double-to-bigdecimal-in-java.html
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] incubator-carbondata pull request #95: [CARBONDATA-152]Double min max differ...
Posted by asfgit <gi...@git.apache.org>.
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/95
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---