[jira] [Created] (CARBONDATA-187) when using Decimal type as dictionary the generated surrogate key would mismatch for the same values during increment load

["zhangshunyu (JIRA)" <ji...@apache.org>]

zhangshunyu created CARBONDATA-187:
--------------------------------------

             Summary: when using Decimal type as dictionary the generated surrogate key would mismatch for the same values during increment load
                 Key: CARBONDATA-187
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-187
             Project: CarbonData
          Issue Type: Bug
            Reporter: zhangshunyu
            Assignee: zhangshunyu


Fix bug: when using Decimal type as dictionary gen surrogate key will mismatch for the same values during increment load.
For example, when we specify Decimal type column using dictionary, as the using of DataTypeUtil.normalizeColumnValueForItsDataType, deciaml data for example 45, if we specify the precision of this column as 3, parsedValue would be 45.000, and this 45.000 would be written into dic file by writer.write(parsedValue). As a result, the second time we load the same data 45, dictionary.getSurrogateKey(value) would compare the value with dic value, but here the value is 45, our dic value is 45.000 stored as string, so dic would think that i don not have 45, this would lead to repeated values in dic.
The dic would be like this: @NU#LL$!100.050 100.055 �45.000� 100.050 �45.000, this is a bug.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)