You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@kylin.apache.org by "Shi, Shaofeng" <sh...@ebay.com> on 2015/05/05 16:08:22 UTC
Re: 答复: 答复: Max on negative double values is not working
Kylin persists the aggregated values in hbase (which should be correct as
you observed), and returns the records to Calcite as a data source;
Calcite will do a round of aggregation (min, max, …) even if it has
already been aggregated by Kylin; What you seeing (resultSet.getString) is
Kylin get the final result from Calcite to compose the REST response, at
this moment the result is wrong;
Hua, have you opened a JIRA for this issue? If not, I suggest you create
such one with your findings in
https://issues.apache.org/jira/secure/Dashboard.jspa, we will track the
problem with JIRA. Thanks!
On 5/5/15, 8:52 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
>Hi Shaofeng,
>
>I don't think it's the problem of Math.max because the value of cube(after
>building) in hbase are in right values such as -59.
>The problem occurs when we do sql query and this piece of code
>"oneRow.add((resultSet.getString(i + 1))); " packages wrong values like
>4.9E-324 while the underlying hbase storage can get correct negative
>values.
>
>Best Regards,
>Hua
>> -----邮件原件-----
>> 发件人: dev-return-1458-
>> huanghua=mininglamp.com@kylin.incubator.apache.org [mailto:dev-return-
>> 1458-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Shi,
>> Shaofeng
>> 发送时间: 2015年5月5日 20:01
>> 收件人: dev@kylin.incubator.apache.org; 'Li Yang'
>> 主题: Re: 答复: Max on negative double values is not working
>>
>> 4.9E-324 is the Double.MIN_VALUE, while it is still positive, so when
>compare
>> it with -59, Math.max returned 4.9E-324; see this post:
>>
>> https://stackoverflow.com/questions/3717655/math-max-seems-to-be-
>> returning-
>> the-wrong-answer
>>
>>
>> I’m not sure whether it is a bug in Calcite; Yang, hongbin, do you have
>any
>> idea?
>>
>> On 5/5/15, 7:33 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
>>
>> >I spent some time to debug the kylin code and I think it's very likely
>> >a bug in Query module.
>> >
>> >The cube and the fact table was illustrated before in the last couple
>> >emails. The query I used was "select id, sum(trans_at), max(trans_at)
>> >from max_neg group by id;".
>> >When the query hit the rest server, it will be passed to the function
>> >"private SQLResponse execute(String sql, SQLRequest sqlRequest)" of
>> >"QueryService" class in kylin-server module.
>> >And in that function, it utilizes the calcite to parse the sql and the
>> >kylin-storage module to actually fetch data from hbase. However, the
>> >following codes of the function packages wrong results.
>> > while (resultSet.next()) {
>> > for (int i = 0; i < columnCount; i++) {
>> > oneRow.add((resultSet.getString(i + 1))); -- This
>> >produce wrong result like [2c9083862fe1d8f00130062167480ce8-213582789,
>> >-86.4, 4.9E-324]
>> > }
>> >
>> > results.add(new LinkedList<String>(oneRow));
>> > oneRow.clear();
>> > }
>> >
>> >I also checked the related functions of kylin-storage, and the results
>> >fetched from hbase were all good, for example, the function "public
>> >Tuple next()" of "CubeSegmentTupleIterator" class returned
>> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0] and the
>> >function "private Object[] convertCurrentRow(ITuple tuple)" of
>> >"CubeEnumerator" class also returned
>> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0] which I
>> >think both are correct.
>> >
>> >Unfortunately I am still looking for the exact piece of codes which
>> >cause the problem, can anyone help look into this issue? Thanks.
>> >
>> >Hua
>> >
>> >> -----邮件原件-----
>> >> 发件人: Huang Hua [mailto:huanghua@mininglamp.com]
>> >> 发送时间: 2015年4月30日 12:10
>> >> 收件人: 'Li Yang'; dev@kylin.incubator.apache.org
>> >> 主题: 答复: Max on negative double values is not working
>> >>
>> >> We defined the max(trans_at) as one of the measures of the cube.
>> >>
>> >> our cube json looks like:
>> >> {
>> >> "uuid": "4d57736b-d3e4-40fe-8480-cbc80f78b4ef",
>> >> "name": "test_max_0",
>> >> "description": "",
>> >> "dimensions": [
>> >> {
>> >> "id": 1,
>> >> "name": "ID",
>> >> "join": null,
>> >> "hierarchy": null,
>> >> "table": "TEST_MAX_ON_NEG",
>> >> "column": "ID",
>> >> "datatype": null,
>> >> "derived": null
>> >> }
>> >> ],
>> >> "measures": [
>> >> {
>> >> "id": 1,
>> >> "name": "_COUNT_",
>> >> "function": {
>> >> "expression": "COUNT",
>> >> "parameter": {
>> >> "type": "constant",
>> >> "value": "1"
>> >> },
>> >> "returntype": "bigint"
>> >> },
>> >> "dependent_measure_ref": null
>> >> },
>> >> {
>> >> "id": 2,
>> >> "name": "SUM_TRANS_AT",
>> >> "function": {
>> >> "expression": "SUM",
>> >> "parameter": {
>> >> "type": "column",
>> >> "value": "TRANS_AT"
>> >> },
>> >> "returntype": "double"
>> >> },
>> >> "dependent_measure_ref": null
>> >> },
>> >> {
>> >> "id": 3,
>> >> "name": "MAX_TRANS_AT",
>> >> "function": {
>> >> "expression": "MAX",
>> >> "parameter": {
>> >> "type": "column",
>> >> "value": "TRANS_AT"
>> >> },
>> >> "returntype": "double"
>> >> },
>> >> "dependent_measure_ref": null
>> >> }
>> >> ],
>> >> "rowkey": {
>> >> "rowkey_columns": [
>> >> {
>> >> "column": "ID",
>> >> "length": 0,
>> >> "dictionary": "true",
>> >> "mandatory": false
>> >> }
>> >> ],
>> >> "aggregation_groups": [
>> >> [
>> >> "ID"
>> >> ]
>> >> ]
>> >> },
>> >> "signature": "R/zfoefWoDBHzfzSUh5thA==",
>> >> "capacity": "MEDIUM",
>> >> "last_modified": 1430279551744,
>> >> "fact_table": "TEST_MAX_ON_NEG",
>> >> "null_string": null,
>> >> "filter_condition": null,
>> >> "cube_partition_desc": {
>> >> "partition_date_column": null,
>> >> "partition_date_start": 0,
>> >> "cube_partition_type": "APPEND"
>> >> },
>> >> "hbase_mapping": {
>> >> "column_family": [
>> >> {
>> >> "name": "F1",
>> >> "columns": [
>> >> {
>> >> "qualifier": "M",
>> >> "measure_refs": [
>> >> "_COUNT_",
>> >> "SUM_TRANS_AT",
>> >> "MAX_TRANS_AT"
>> >> ]
>> >> }
>> >> ]
>> >> }
>> >> ]
>> >> },
>> >> "notify_list": []
>> >> }
>> >>
>> >> The cube is based on only a fact table called test_max_on_neg which
>> >> has only six records:
>> >>
>> >> 2c9083862fe1d8f00130062167480ce8-211316617 -199.6
>> >> 2c9083862fe1d8f00130062167480ce8-21782293 -229.0
>> >> 2c9083862fe1d8f00130062167480ce8-211955199 -151.7
>> >> 2c9083862fe1d8f00130062167480ce8-213582789 -86.4
>> >> 2c9083862fe1d8f00130062167480ce8-115552384 -353.5
>> >> 2c9083862fe1d8f00130062167480ce8-0114097017 -59.0
>> >>
>> >> When the cube is ready, we did the query: select id, max(trans_at),
>> >> sum(trans_at) from test_max_on_neg group by id, which returned
>> >>4.9E-324 for MAX(trans_at) column. I think the problem still exists
>> >>for kylin-0.6.6, and you can reproduce the problem if you like.
>> >>
>> >> > -----邮件原件-----
>> >> > 发件人: Li Yang [mailto:liyang@apache.org]
>> >> > 发送时间: 2015年4月29日 14:56
>> >> > 收件人: dev@kylin.incubator.apache.org
>> >> > 主题: Re: Max on negative double values is not working
>> >> >
>> >> > Depend on whether max(trans_at) is a measure of cube or not, the
>> >> > problem and the answer will be different.
>> >> >
>> >> >
>> >> > On Wed, Apr 29, 2015 at 10:43 AM, hongbin ma
>> <ma...@apache.org>
>> >> > wrote:
>> >> >
>> >> > > can you share your cube's definition? you can go to cubes tab,
>> >> > > click on the cube, and get json tab.
>> >> > >
>> >> > > we'd like to help check the definition of your cube
>> >> > >
>> >> > > On Tue, Apr 28, 2015 at 8:01 PM, Huang Hua
>> >> <hu...@mininglamp.com>
>> >> > > wrote:
>> >> > >
>> >> > > > Hi there,
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > We are using kylin 0.6.6, and we found out the max function
>> >> > > > might not
>> >> > > work
>> >> > > > for negative values in double type.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > The fact table in Hive only has two columns: id and trans_at
>> >> > > > and contains six records as shown below:
>> >> > > >
>> >> > > > Id trans_at
>> >> > > >
>> >> > > > 1, -59.0
>> >> > > >
>> >> > > > 2, -229.0
>> >> > > >
>> >> > > > 3, -86.4
>> >> > > >
>> >> > > > 4, -151.7
>> >> > > >
>> >> > > > 5, -199.6
>> >> > > >
>> >> > > > 6, -353.5
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > Then based on the fact table, we created and built a cube with
>> >> > > > 1
>> >> > > dimension
>> >> > > > and 3 measures including the default _count_ one:
>> >> > > >
>> >> > > > Dimension: id
>> >> > > >
>> >> > > > Measures: max(trans_at), sum(trans_at), _count_
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > The sql used in query was: select id, MAX(trans_at),
>> >> > > > SUM(trans_at) from the_table group by id, which produced the
>> following output:
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > ID,EXPR$1,EXPR$2
>> >> > > >
>> >> > > > 1, 4.9E-324, -59.0
>> >> > > >
>> >> > > > 2, 4.9E-324, -229.0
>> >> > > >
>> >> > > > 3, 4.9E-324, -86.4
>> >> > > >
>> >> > > > 4, 4.9E-324, -151.7
>> >> > > >
>> >> > > > 5, 4.9E-324, -199.6
>> >> > > >
>> >> > > > 6, 4.9E-324, -353.5
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > As you can see, max(trans_at) produced 4.9E-324 for each row
>> >> > > > which is not what we expect to see.
>> >> > > >
>> >> > > >
>> >> > > >
>> >> > > > Best Regards.
>> >> > > >
>> >> > > > Hua
>> >> > > >
>> >> > > >
>> >> > >
>> >> > >
>> >> > > --
>> >> > > Regards,
>> >> > >
>> >> > > *Bin Mahone | 马洪宾*
>> >> > > Apache Kylin: http://kylin.io
>> >> > > Github: https://github.com/binmahone
>> >> > >
>> >>
>> >>
>> >
>> >
>> >
>
>
>
Re: 答复: 答复: 答复: Max on negative double values is not working
Posted by Li Yang <li...@apache.org>.
OK, I debug into calcite and it turns out to be a calcite bug.
https://issues.apache.org/jira/browse/CALCITE-735
For now, I'll let it be a known issue of Kylin. Guess not many users are
impacted. If you have urgent need, let me know. I can put a dirty hack in
Kylin if that's required.
On Wed, May 6, 2015 at 11:40 AM, Huang Hua <hu...@mininglamp.com> wrote:
> Hi,
>
> I've created a JIRA issue to track this bug, please find it here,
> https://issues.apache.org/jira/browse/KYLIN-751.
>
> Thanks,
> Hua
>
> > -----邮件原件-----
> > 发件人: dev-return-1465-
> > huanghua=mininglamp.com@kylin.incubator.apache.org [mailto:dev-return-
> > 1465-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Li Yang
> > 发送时间: 2015年5月6日 1:33
> > 收件人: dev@kylin.incubator.apache.org
> > 主题: Re: 答复: 答复: Max on negative double values is not working
> >
> > I'm on biz trip this week. Will give it a look after back to office
> next week.
> >
> > On Tue, May 5, 2015 at 10:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> >
> > > Kylin persists the aggregated values in hbase (which should be correct
> > > as you observed), and returns the records to Calcite as a data source;
> > > Calcite will do a round of aggregation (min, max, …) even if it has
> > > already been aggregated by Kylin; What you seeing
> > > (resultSet.getString) is Kylin get the final result from Calcite to
> > > compose the REST response, at this moment the result is wrong;
> > >
> > > Hua, have you opened a JIRA for this issue? If not, I suggest you
> > > create such one with your findings in
> > > https://issues.apache.org/jira/secure/Dashboard.jspa, we will track
> > > the problem with JIRA. Thanks!
> > >
> > > On 5/5/15, 8:52 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
> > >
> > > >Hi Shaofeng,
> > > >
> > > >I don't think it's the problem of Math.max because the value of
> > > >cube(after
> > > >building) in hbase are in right values such as -59.
> > > >The problem occurs when we do sql query and this piece of code
> > > >"oneRow.add((resultSet.getString(i + 1))); " packages wrong values
> > > >like
> > > >4.9E-324 while the underlying hbase storage can get correct negative
> > > >values.
> > > >
> > > >Best Regards,
> > > >Hua
> > > >> -----邮件原件-----
> > > >> 发件人: dev-return-1458-
> > > >> huanghua=mininglamp.com@kylin.incubator.apache.org
> > > >> [mailto:dev-return-
> > > >> 1458-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Shi,
> > > >> Shaofeng
> > > >> 发送时间: 2015年5月5日 20:01
> > > >> 收件人: dev@kylin.incubator.apache.org; 'Li Yang'
> > > >> 主题: Re: 答复: Max on negative double values is not working
> > > >>
> > > >> 4.9E-324 is the Double.MIN_VALUE, while it is still positive, so
> > > >> when
> > > >compare
> > > >> it with -59, Math.max returned 4.9E-324; see this post:
> > > >>
> > > >> https://stackoverflow.com/questions/3717655/math-max-seems-to-
> > be-
> > > >> returning-
> > > >> the-wrong-answer
> > > >>
> > > >>
> > > >> I’m not sure whether it is a bug in Calcite; Yang, hongbin, do you
> > > >> have
> > > >any
> > > >> idea?
> > > >>
> > > >> On 5/5/15, 7:33 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
> > > >>
> > > >> >I spent some time to debug the kylin code and I think it's very
> > > >> >likely a bug in Query module.
> > > >> >
> > > >> >The cube and the fact table was illustrated before in the last
> > > >> >couple emails. The query I used was "select id, sum(trans_at),
> > > >> >max(trans_at) from max_neg group by id;".
> > > >> >When the query hit the rest server, it will be passed to the
> > > >> >function "private SQLResponse execute(String sql, SQLRequest
> > > >> >sqlRequest)" of "QueryService" class in kylin-server module.
> > > >> >And in that function, it utilizes the calcite to parse the sql and
> > > >> >the kylin-storage module to actually fetch data from hbase.
> > > >> >However, the following codes of the function packages wrong
> results.
> > > >> > while (resultSet.next()) {
> > > >> > for (int i = 0; i < columnCount; i++) {
> > > >> > oneRow.add((resultSet.getString(i + 1))); --
> > > >> >This produce wrong result like
> > > >> >[2c9083862fe1d8f00130062167480ce8-213582789,
> > > >> >-86.4, 4.9E-324]
> > > >> > }
> > > >> >
> > > >> > results.add(new LinkedList<String>(oneRow));
> > > >> > oneRow.clear();
> > > >> > }
> > > >> >
> > > >> >I also checked the related functions of kylin-storage, and the
> > > >> >results fetched from hbase were all good, for example, the
> > > >> >function "public Tuple next()" of "CubeSegmentTupleIterator" class
> > > >> >returned [2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1,
> > > >> >-59.0] and the function "private Object[] convertCurrentRow(ITuple
> > > >> >tuple)" of "CubeEnumerator" class also returned
> > > >> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0]
> > > >> >which I think both are correct.
> > > >> >
> > > >> >Unfortunately I am still looking for the exact piece of codes
> > > >> >which cause the problem, can anyone help look into this issue?
> Thanks.
> > > >> >
> > > >> >Hua
> > > >> >
> > > >> >> -----邮件原件-----
> > > >> >> 发件人: Huang Hua [mailto:huanghua@mininglamp.com]
> > > >> >> 发送时间: 2015年4月30日 12:10
> > > >> >> 收件人: 'Li Yang'; dev@kylin.incubator.apache.org
> > > >> >> 主题: 答复: Max on negative double values is not working
> > > >> >>
> > > >> >> We defined the max(trans_at) as one of the measures of the cube.
> > > >> >>
> > > >> >> our cube json looks like:
> > > >> >> {
> > > >> >> "uuid": "4d57736b-d3e4-40fe-8480-cbc80f78b4ef",
> > > >> >> "name": "test_max_0",
> > > >> >> "description": "",
> > > >> >> "dimensions": [
> > > >> >> {
> > > >> >> "id": 1,
> > > >> >> "name": "ID",
> > > >> >> "join": null,
> > > >> >> "hierarchy": null,
> > > >> >> "table": "TEST_MAX_ON_NEG",
> > > >> >> "column": "ID",
> > > >> >> "datatype": null,
> > > >> >> "derived": null
> > > >> >> }
> > > >> >> ],
> > > >> >> "measures": [
> > > >> >> {
> > > >> >> "id": 1,
> > > >> >> "name": "_COUNT_",
> > > >> >> "function": {
> > > >> >> "expression": "COUNT",
> > > >> >> "parameter": {
> > > >> >> "type": "constant",
> > > >> >> "value": "1"
> > > >> >> },
> > > >> >> "returntype": "bigint"
> > > >> >> },
> > > >> >> "dependent_measure_ref": null
> > > >> >> },
> > > >> >> {
> > > >> >> "id": 2,
> > > >> >> "name": "SUM_TRANS_AT",
> > > >> >> "function": {
> > > >> >> "expression": "SUM",
> > > >> >> "parameter": {
> > > >> >> "type": "column",
> > > >> >> "value": "TRANS_AT"
> > > >> >> },
> > > >> >> "returntype": "double"
> > > >> >> },
> > > >> >> "dependent_measure_ref": null
> > > >> >> },
> > > >> >> {
> > > >> >> "id": 3,
> > > >> >> "name": "MAX_TRANS_AT",
> > > >> >> "function": {
> > > >> >> "expression": "MAX",
> > > >> >> "parameter": {
> > > >> >> "type": "column",
> > > >> >> "value": "TRANS_AT"
> > > >> >> },
> > > >> >> "returntype": "double"
> > > >> >> },
> > > >> >> "dependent_measure_ref": null
> > > >> >> }
> > > >> >> ],
> > > >> >> "rowkey": {
> > > >> >> "rowkey_columns": [
> > > >> >> {
> > > >> >> "column": "ID",
> > > >> >> "length": 0,
> > > >> >> "dictionary": "true",
> > > >> >> "mandatory": false
> > > >> >> }
> > > >> >> ],
> > > >> >> "aggregation_groups": [
> > > >> >> [
> > > >> >> "ID"
> > > >> >> ]
> > > >> >> ]
> > > >> >> },
> > > >> >> "signature": "R/zfoefWoDBHzfzSUh5thA==",
> > > >> >> "capacity": "MEDIUM",
> > > >> >> "last_modified": 1430279551744,
> > > >> >> "fact_table": "TEST_MAX_ON_NEG",
> > > >> >> "null_string": null,
> > > >> >> "filter_condition": null,
> > > >> >> "cube_partition_desc": {
> > > >> >> "partition_date_column": null,
> > > >> >> "partition_date_start": 0,
> > > >> >> "cube_partition_type": "APPEND"
> > > >> >> },
> > > >> >> "hbase_mapping": {
> > > >> >> "column_family": [
> > > >> >> {
> > > >> >> "name": "F1",
> > > >> >> "columns": [
> > > >> >> {
> > > >> >> "qualifier": "M",
> > > >> >> "measure_refs": [
> > > >> >> "_COUNT_",
> > > >> >> "SUM_TRANS_AT",
> > > >> >> "MAX_TRANS_AT"
> > > >> >> ]
> > > >> >> }
> > > >> >> ]
> > > >> >> }
> > > >> >> ]
> > > >> >> },
> > > >> >> "notify_list": []
> > > >> >> }
> > > >> >>
> > > >> >> The cube is based on only a fact table called test_max_on_neg
> > > >> >> which has only six records:
> > > >> >>
> > > >> >> 2c9083862fe1d8f00130062167480ce8-211316617 -199.6
> > > >> >> 2c9083862fe1d8f00130062167480ce8-21782293 -229.0
> > > >> >> 2c9083862fe1d8f00130062167480ce8-211955199 -151.7
> > > >> >> 2c9083862fe1d8f00130062167480ce8-213582789 -86.4
> > > >> >> 2c9083862fe1d8f00130062167480ce8-115552384 -353.5
> > > >> >> 2c9083862fe1d8f00130062167480ce8-0114097017 -59.0
> > > >> >>
> > > >> >> When the cube is ready, we did the query: select id,
> > > >> >>max(trans_at),
> > > >> >> sum(trans_at) from test_max_on_neg group by id, which returned
> > > >> >>4.9E-324 for MAX(trans_at) column. I think the problem still
> > > >> >>exists for kylin-0.6.6, and you can reproduce the problem if you
> like.
> > > >> >>
> > > >> >> > -----邮件原件-----
> > > >> >> > 发件人: Li Yang [mailto:liyang@apache.org]
> > > >> >> > 发送时间: 2015年4月29日 14:56
> > > >> >> > 收件人: dev@kylin.incubator.apache.org
> > > >> >> > 主题: Re: Max on negative double values is not working
> > > >> >> >
> > > >> >> > Depend on whether max(trans_at) is a measure of cube or not,
> > > >> >> > the problem and the answer will be different.
> > > >> >> >
> > > >> >> >
> > > >> >> > On Wed, Apr 29, 2015 at 10:43 AM, hongbin ma
> > > >> <ma...@apache.org>
> > > >> >> > wrote:
> > > >> >> >
> > > >> >> > > can you share your cube's definition? you can go to cubes
> > > >> >> > > tab, click on the cube, and get json tab.
> > > >> >> > >
> > > >> >> > > we'd like to help check the definition of your cube
> > > >> >> > >
> > > >> >> > > On Tue, Apr 28, 2015 at 8:01 PM, Huang Hua
> > > >> >> <hu...@mininglamp.com>
> > > >> >> > > wrote:
> > > >> >> > >
> > > >> >> > > > Hi there,
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > We are using kylin 0.6.6, and we found out the max
> > > >> >> > > > function might not
> > > >> >> > > work
> > > >> >> > > > for negative values in double type.
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > The fact table in Hive only has two columns: id and
> > > >> >> > > > trans_at and contains six records as shown below:
> > > >> >> > > >
> > > >> >> > > > Id trans_at
> > > >> >> > > >
> > > >> >> > > > 1, -59.0
> > > >> >> > > >
> > > >> >> > > > 2, -229.0
> > > >> >> > > >
> > > >> >> > > > 3, -86.4
> > > >> >> > > >
> > > >> >> > > > 4, -151.7
> > > >> >> > > >
> > > >> >> > > > 5, -199.6
> > > >> >> > > >
> > > >> >> > > > 6, -353.5
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > Then based on the fact table, we created and built a cube
> > > >> >> > > > with
> > > >> >> > > > 1
> > > >> >> > > dimension
> > > >> >> > > > and 3 measures including the default _count_ one:
> > > >> >> > > >
> > > >> >> > > > Dimension: id
> > > >> >> > > >
> > > >> >> > > > Measures: max(trans_at), sum(trans_at), _count_
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > The sql used in query was: select id, MAX(trans_at),
> > > >> >> > > > SUM(trans_at) from the_table group by id, which produced
> > > >> >> > > > the
> > > >> following output:
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > ID,EXPR$1,EXPR$2
> > > >> >> > > >
> > > >> >> > > > 1, 4.9E-324, -59.0
> > > >> >> > > >
> > > >> >> > > > 2, 4.9E-324, -229.0
> > > >> >> > > >
> > > >> >> > > > 3, 4.9E-324, -86.4
> > > >> >> > > >
> > > >> >> > > > 4, 4.9E-324, -151.7
> > > >> >> > > >
> > > >> >> > > > 5, 4.9E-324, -199.6
> > > >> >> > > >
> > > >> >> > > > 6, 4.9E-324, -353.5
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > As you can see, max(trans_at) produced 4.9E-324 for each
> > > >> >> > > > row which is not what we expect to see.
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > > > Best Regards.
> > > >> >> > > >
> > > >> >> > > > Hua
> > > >> >> > > >
> > > >> >> > > >
> > > >> >> > >
> > > >> >> > >
> > > >> >> > > --
> > > >> >> > > Regards,
> > > >> >> > >
> > > >> >> > > *Bin Mahone | 马洪宾*
> > > >> >> > > Apache Kylin: http://kylin.io
> > > >> >> > > Github: https://github.com/binmahone
> > > >> >> > >
> > > >> >>
> > > >> >>
> > > >> >
> > > >> >
> > > >> >
> > > >
> > > >
> > > >
> > >
> > >
>
>
>
答复: 答复: 答复: Max on negative double values is not working
Posted by Huang Hua <hu...@mininglamp.com>.
Hi,
I've created a JIRA issue to track this bug, please find it here, https://issues.apache.org/jira/browse/KYLIN-751.
Thanks,
Hua
> -----邮件原件-----
> 发件人: dev-return-1465-
> huanghua=mininglamp.com@kylin.incubator.apache.org [mailto:dev-return-
> 1465-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Li Yang
> 发送时间: 2015年5月6日 1:33
> 收件人: dev@kylin.incubator.apache.org
> 主题: Re: 答复: 答复: Max on negative double values is not working
>
> I'm on biz trip this week. Will give it a look after back to office next week.
>
> On Tue, May 5, 2015 at 10:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
>
> > Kylin persists the aggregated values in hbase (which should be correct
> > as you observed), and returns the records to Calcite as a data source;
> > Calcite will do a round of aggregation (min, max, …) even if it has
> > already been aggregated by Kylin; What you seeing
> > (resultSet.getString) is Kylin get the final result from Calcite to
> > compose the REST response, at this moment the result is wrong;
> >
> > Hua, have you opened a JIRA for this issue? If not, I suggest you
> > create such one with your findings in
> > https://issues.apache.org/jira/secure/Dashboard.jspa, we will track
> > the problem with JIRA. Thanks!
> >
> > On 5/5/15, 8:52 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
> >
> > >Hi Shaofeng,
> > >
> > >I don't think it's the problem of Math.max because the value of
> > >cube(after
> > >building) in hbase are in right values such as -59.
> > >The problem occurs when we do sql query and this piece of code
> > >"oneRow.add((resultSet.getString(i + 1))); " packages wrong values
> > >like
> > >4.9E-324 while the underlying hbase storage can get correct negative
> > >values.
> > >
> > >Best Regards,
> > >Hua
> > >> -----邮件原件-----
> > >> 发件人: dev-return-1458-
> > >> huanghua=mininglamp.com@kylin.incubator.apache.org
> > >> [mailto:dev-return-
> > >> 1458-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Shi,
> > >> Shaofeng
> > >> 发送时间: 2015年5月5日 20:01
> > >> 收件人: dev@kylin.incubator.apache.org; 'Li Yang'
> > >> 主题: Re: 答复: Max on negative double values is not working
> > >>
> > >> 4.9E-324 is the Double.MIN_VALUE, while it is still positive, so
> > >> when
> > >compare
> > >> it with -59, Math.max returned 4.9E-324; see this post:
> > >>
> > >> https://stackoverflow.com/questions/3717655/math-max-seems-to-
> be-
> > >> returning-
> > >> the-wrong-answer
> > >>
> > >>
> > >> I’m not sure whether it is a bug in Calcite; Yang, hongbin, do you
> > >> have
> > >any
> > >> idea?
> > >>
> > >> On 5/5/15, 7:33 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
> > >>
> > >> >I spent some time to debug the kylin code and I think it's very
> > >> >likely a bug in Query module.
> > >> >
> > >> >The cube and the fact table was illustrated before in the last
> > >> >couple emails. The query I used was "select id, sum(trans_at),
> > >> >max(trans_at) from max_neg group by id;".
> > >> >When the query hit the rest server, it will be passed to the
> > >> >function "private SQLResponse execute(String sql, SQLRequest
> > >> >sqlRequest)" of "QueryService" class in kylin-server module.
> > >> >And in that function, it utilizes the calcite to parse the sql and
> > >> >the kylin-storage module to actually fetch data from hbase.
> > >> >However, the following codes of the function packages wrong results.
> > >> > while (resultSet.next()) {
> > >> > for (int i = 0; i < columnCount; i++) {
> > >> > oneRow.add((resultSet.getString(i + 1))); --
> > >> >This produce wrong result like
> > >> >[2c9083862fe1d8f00130062167480ce8-213582789,
> > >> >-86.4, 4.9E-324]
> > >> > }
> > >> >
> > >> > results.add(new LinkedList<String>(oneRow));
> > >> > oneRow.clear();
> > >> > }
> > >> >
> > >> >I also checked the related functions of kylin-storage, and the
> > >> >results fetched from hbase were all good, for example, the
> > >> >function "public Tuple next()" of "CubeSegmentTupleIterator" class
> > >> >returned [2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1,
> > >> >-59.0] and the function "private Object[] convertCurrentRow(ITuple
> > >> >tuple)" of "CubeEnumerator" class also returned
> > >> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0]
> > >> >which I think both are correct.
> > >> >
> > >> >Unfortunately I am still looking for the exact piece of codes
> > >> >which cause the problem, can anyone help look into this issue? Thanks.
> > >> >
> > >> >Hua
> > >> >
> > >> >> -----邮件原件-----
> > >> >> 发件人: Huang Hua [mailto:huanghua@mininglamp.com]
> > >> >> 发送时间: 2015年4月30日 12:10
> > >> >> 收件人: 'Li Yang'; dev@kylin.incubator.apache.org
> > >> >> 主题: 答复: Max on negative double values is not working
> > >> >>
> > >> >> We defined the max(trans_at) as one of the measures of the cube.
> > >> >>
> > >> >> our cube json looks like:
> > >> >> {
> > >> >> "uuid": "4d57736b-d3e4-40fe-8480-cbc80f78b4ef",
> > >> >> "name": "test_max_0",
> > >> >> "description": "",
> > >> >> "dimensions": [
> > >> >> {
> > >> >> "id": 1,
> > >> >> "name": "ID",
> > >> >> "join": null,
> > >> >> "hierarchy": null,
> > >> >> "table": "TEST_MAX_ON_NEG",
> > >> >> "column": "ID",
> > >> >> "datatype": null,
> > >> >> "derived": null
> > >> >> }
> > >> >> ],
> > >> >> "measures": [
> > >> >> {
> > >> >> "id": 1,
> > >> >> "name": "_COUNT_",
> > >> >> "function": {
> > >> >> "expression": "COUNT",
> > >> >> "parameter": {
> > >> >> "type": "constant",
> > >> >> "value": "1"
> > >> >> },
> > >> >> "returntype": "bigint"
> > >> >> },
> > >> >> "dependent_measure_ref": null
> > >> >> },
> > >> >> {
> > >> >> "id": 2,
> > >> >> "name": "SUM_TRANS_AT",
> > >> >> "function": {
> > >> >> "expression": "SUM",
> > >> >> "parameter": {
> > >> >> "type": "column",
> > >> >> "value": "TRANS_AT"
> > >> >> },
> > >> >> "returntype": "double"
> > >> >> },
> > >> >> "dependent_measure_ref": null
> > >> >> },
> > >> >> {
> > >> >> "id": 3,
> > >> >> "name": "MAX_TRANS_AT",
> > >> >> "function": {
> > >> >> "expression": "MAX",
> > >> >> "parameter": {
> > >> >> "type": "column",
> > >> >> "value": "TRANS_AT"
> > >> >> },
> > >> >> "returntype": "double"
> > >> >> },
> > >> >> "dependent_measure_ref": null
> > >> >> }
> > >> >> ],
> > >> >> "rowkey": {
> > >> >> "rowkey_columns": [
> > >> >> {
> > >> >> "column": "ID",
> > >> >> "length": 0,
> > >> >> "dictionary": "true",
> > >> >> "mandatory": false
> > >> >> }
> > >> >> ],
> > >> >> "aggregation_groups": [
> > >> >> [
> > >> >> "ID"
> > >> >> ]
> > >> >> ]
> > >> >> },
> > >> >> "signature": "R/zfoefWoDBHzfzSUh5thA==",
> > >> >> "capacity": "MEDIUM",
> > >> >> "last_modified": 1430279551744,
> > >> >> "fact_table": "TEST_MAX_ON_NEG",
> > >> >> "null_string": null,
> > >> >> "filter_condition": null,
> > >> >> "cube_partition_desc": {
> > >> >> "partition_date_column": null,
> > >> >> "partition_date_start": 0,
> > >> >> "cube_partition_type": "APPEND"
> > >> >> },
> > >> >> "hbase_mapping": {
> > >> >> "column_family": [
> > >> >> {
> > >> >> "name": "F1",
> > >> >> "columns": [
> > >> >> {
> > >> >> "qualifier": "M",
> > >> >> "measure_refs": [
> > >> >> "_COUNT_",
> > >> >> "SUM_TRANS_AT",
> > >> >> "MAX_TRANS_AT"
> > >> >> ]
> > >> >> }
> > >> >> ]
> > >> >> }
> > >> >> ]
> > >> >> },
> > >> >> "notify_list": []
> > >> >> }
> > >> >>
> > >> >> The cube is based on only a fact table called test_max_on_neg
> > >> >> which has only six records:
> > >> >>
> > >> >> 2c9083862fe1d8f00130062167480ce8-211316617 -199.6
> > >> >> 2c9083862fe1d8f00130062167480ce8-21782293 -229.0
> > >> >> 2c9083862fe1d8f00130062167480ce8-211955199 -151.7
> > >> >> 2c9083862fe1d8f00130062167480ce8-213582789 -86.4
> > >> >> 2c9083862fe1d8f00130062167480ce8-115552384 -353.5
> > >> >> 2c9083862fe1d8f00130062167480ce8-0114097017 -59.0
> > >> >>
> > >> >> When the cube is ready, we did the query: select id,
> > >> >>max(trans_at),
> > >> >> sum(trans_at) from test_max_on_neg group by id, which returned
> > >> >>4.9E-324 for MAX(trans_at) column. I think the problem still
> > >> >>exists for kylin-0.6.6, and you can reproduce the problem if you like.
> > >> >>
> > >> >> > -----邮件原件-----
> > >> >> > 发件人: Li Yang [mailto:liyang@apache.org]
> > >> >> > 发送时间: 2015年4月29日 14:56
> > >> >> > 收件人: dev@kylin.incubator.apache.org
> > >> >> > 主题: Re: Max on negative double values is not working
> > >> >> >
> > >> >> > Depend on whether max(trans_at) is a measure of cube or not,
> > >> >> > the problem and the answer will be different.
> > >> >> >
> > >> >> >
> > >> >> > On Wed, Apr 29, 2015 at 10:43 AM, hongbin ma
> > >> <ma...@apache.org>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > can you share your cube's definition? you can go to cubes
> > >> >> > > tab, click on the cube, and get json tab.
> > >> >> > >
> > >> >> > > we'd like to help check the definition of your cube
> > >> >> > >
> > >> >> > > On Tue, Apr 28, 2015 at 8:01 PM, Huang Hua
> > >> >> <hu...@mininglamp.com>
> > >> >> > > wrote:
> > >> >> > >
> > >> >> > > > Hi there,
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > We are using kylin 0.6.6, and we found out the max
> > >> >> > > > function might not
> > >> >> > > work
> > >> >> > > > for negative values in double type.
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > The fact table in Hive only has two columns: id and
> > >> >> > > > trans_at and contains six records as shown below:
> > >> >> > > >
> > >> >> > > > Id trans_at
> > >> >> > > >
> > >> >> > > > 1, -59.0
> > >> >> > > >
> > >> >> > > > 2, -229.0
> > >> >> > > >
> > >> >> > > > 3, -86.4
> > >> >> > > >
> > >> >> > > > 4, -151.7
> > >> >> > > >
> > >> >> > > > 5, -199.6
> > >> >> > > >
> > >> >> > > > 6, -353.5
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > Then based on the fact table, we created and built a cube
> > >> >> > > > with
> > >> >> > > > 1
> > >> >> > > dimension
> > >> >> > > > and 3 measures including the default _count_ one:
> > >> >> > > >
> > >> >> > > > Dimension: id
> > >> >> > > >
> > >> >> > > > Measures: max(trans_at), sum(trans_at), _count_
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > The sql used in query was: select id, MAX(trans_at),
> > >> >> > > > SUM(trans_at) from the_table group by id, which produced
> > >> >> > > > the
> > >> following output:
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > ID,EXPR$1,EXPR$2
> > >> >> > > >
> > >> >> > > > 1, 4.9E-324, -59.0
> > >> >> > > >
> > >> >> > > > 2, 4.9E-324, -229.0
> > >> >> > > >
> > >> >> > > > 3, 4.9E-324, -86.4
> > >> >> > > >
> > >> >> > > > 4, 4.9E-324, -151.7
> > >> >> > > >
> > >> >> > > > 5, 4.9E-324, -199.6
> > >> >> > > >
> > >> >> > > > 6, 4.9E-324, -353.5
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > As you can see, max(trans_at) produced 4.9E-324 for each
> > >> >> > > > row which is not what we expect to see.
> > >> >> > > >
> > >> >> > > >
> > >> >> > > >
> > >> >> > > > Best Regards.
> > >> >> > > >
> > >> >> > > > Hua
> > >> >> > > >
> > >> >> > > >
> > >> >> > >
> > >> >> > >
> > >> >> > > --
> > >> >> > > Regards,
> > >> >> > >
> > >> >> > > *Bin Mahone | 马洪宾*
> > >> >> > > Apache Kylin: http://kylin.io
> > >> >> > > Github: https://github.com/binmahone
> > >> >> > >
> > >> >>
> > >> >>
> > >> >
> > >> >
> > >> >
> > >
> > >
> > >
> >
> >
Re: 答复: 答复: Max on negative double values is not working
Posted by Li Yang <li...@apache.org>.
I'm on biz trip this week. Will give it a look after back to office next
week.
On Tue, May 5, 2015 at 10:08 PM, Shi, Shaofeng <sh...@ebay.com> wrote:
> Kylin persists the aggregated values in hbase (which should be correct as
> you observed), and returns the records to Calcite as a data source;
> Calcite will do a round of aggregation (min, max, …) even if it has
> already been aggregated by Kylin; What you seeing (resultSet.getString) is
> Kylin get the final result from Calcite to compose the REST response, at
> this moment the result is wrong;
>
> Hua, have you opened a JIRA for this issue? If not, I suggest you create
> such one with your findings in
> https://issues.apache.org/jira/secure/Dashboard.jspa, we will track the
> problem with JIRA. Thanks!
>
> On 5/5/15, 8:52 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
>
> >Hi Shaofeng,
> >
> >I don't think it's the problem of Math.max because the value of cube(after
> >building) in hbase are in right values such as -59.
> >The problem occurs when we do sql query and this piece of code
> >"oneRow.add((resultSet.getString(i + 1))); " packages wrong values like
> >4.9E-324 while the underlying hbase storage can get correct negative
> >values.
> >
> >Best Regards,
> >Hua
> >> -----邮件原件-----
> >> 发件人: dev-return-1458-
> >> huanghua=mininglamp.com@kylin.incubator.apache.org [mailto:dev-return-
> >> 1458-huanghua=mininglamp.com@kylin.incubator.apache.org] 代表 Shi,
> >> Shaofeng
> >> 发送时间: 2015年5月5日 20:01
> >> 收件人: dev@kylin.incubator.apache.org; 'Li Yang'
> >> 主题: Re: 答复: Max on negative double values is not working
> >>
> >> 4.9E-324 is the Double.MIN_VALUE, while it is still positive, so when
> >compare
> >> it with -59, Math.max returned 4.9E-324; see this post:
> >>
> >> https://stackoverflow.com/questions/3717655/math-max-seems-to-be-
> >> returning-
> >> the-wrong-answer
> >>
> >>
> >> I’m not sure whether it is a bug in Calcite; Yang, hongbin, do you have
> >any
> >> idea?
> >>
> >> On 5/5/15, 7:33 PM, "Huang Hua" <hu...@mininglamp.com> wrote:
> >>
> >> >I spent some time to debug the kylin code and I think it's very likely
> >> >a bug in Query module.
> >> >
> >> >The cube and the fact table was illustrated before in the last couple
> >> >emails. The query I used was "select id, sum(trans_at), max(trans_at)
> >> >from max_neg group by id;".
> >> >When the query hit the rest server, it will be passed to the function
> >> >"private SQLResponse execute(String sql, SQLRequest sqlRequest)" of
> >> >"QueryService" class in kylin-server module.
> >> >And in that function, it utilizes the calcite to parse the sql and the
> >> >kylin-storage module to actually fetch data from hbase. However, the
> >> >following codes of the function packages wrong results.
> >> > while (resultSet.next()) {
> >> > for (int i = 0; i < columnCount; i++) {
> >> > oneRow.add((resultSet.getString(i + 1))); -- This
> >> >produce wrong result like [2c9083862fe1d8f00130062167480ce8-213582789,
> >> >-86.4, 4.9E-324]
> >> > }
> >> >
> >> > results.add(new LinkedList<String>(oneRow));
> >> > oneRow.clear();
> >> > }
> >> >
> >> >I also checked the related functions of kylin-storage, and the results
> >> >fetched from hbase were all good, for example, the function "public
> >> >Tuple next()" of "CubeSegmentTupleIterator" class returned
> >> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0] and the
> >> >function "private Object[] convertCurrentRow(ITuple tuple)" of
> >> >"CubeEnumerator" class also returned
> >> >[2c9083862fe1d8f00130062167480ce8-0114097017, -59.0, 1, -59.0] which I
> >> >think both are correct.
> >> >
> >> >Unfortunately I am still looking for the exact piece of codes which
> >> >cause the problem, can anyone help look into this issue? Thanks.
> >> >
> >> >Hua
> >> >
> >> >> -----邮件原件-----
> >> >> 发件人: Huang Hua [mailto:huanghua@mininglamp.com]
> >> >> 发送时间: 2015年4月30日 12:10
> >> >> 收件人: 'Li Yang'; dev@kylin.incubator.apache.org
> >> >> 主题: 答复: Max on negative double values is not working
> >> >>
> >> >> We defined the max(trans_at) as one of the measures of the cube.
> >> >>
> >> >> our cube json looks like:
> >> >> {
> >> >> "uuid": "4d57736b-d3e4-40fe-8480-cbc80f78b4ef",
> >> >> "name": "test_max_0",
> >> >> "description": "",
> >> >> "dimensions": [
> >> >> {
> >> >> "id": 1,
> >> >> "name": "ID",
> >> >> "join": null,
> >> >> "hierarchy": null,
> >> >> "table": "TEST_MAX_ON_NEG",
> >> >> "column": "ID",
> >> >> "datatype": null,
> >> >> "derived": null
> >> >> }
> >> >> ],
> >> >> "measures": [
> >> >> {
> >> >> "id": 1,
> >> >> "name": "_COUNT_",
> >> >> "function": {
> >> >> "expression": "COUNT",
> >> >> "parameter": {
> >> >> "type": "constant",
> >> >> "value": "1"
> >> >> },
> >> >> "returntype": "bigint"
> >> >> },
> >> >> "dependent_measure_ref": null
> >> >> },
> >> >> {
> >> >> "id": 2,
> >> >> "name": "SUM_TRANS_AT",
> >> >> "function": {
> >> >> "expression": "SUM",
> >> >> "parameter": {
> >> >> "type": "column",
> >> >> "value": "TRANS_AT"
> >> >> },
> >> >> "returntype": "double"
> >> >> },
> >> >> "dependent_measure_ref": null
> >> >> },
> >> >> {
> >> >> "id": 3,
> >> >> "name": "MAX_TRANS_AT",
> >> >> "function": {
> >> >> "expression": "MAX",
> >> >> "parameter": {
> >> >> "type": "column",
> >> >> "value": "TRANS_AT"
> >> >> },
> >> >> "returntype": "double"
> >> >> },
> >> >> "dependent_measure_ref": null
> >> >> }
> >> >> ],
> >> >> "rowkey": {
> >> >> "rowkey_columns": [
> >> >> {
> >> >> "column": "ID",
> >> >> "length": 0,
> >> >> "dictionary": "true",
> >> >> "mandatory": false
> >> >> }
> >> >> ],
> >> >> "aggregation_groups": [
> >> >> [
> >> >> "ID"
> >> >> ]
> >> >> ]
> >> >> },
> >> >> "signature": "R/zfoefWoDBHzfzSUh5thA==",
> >> >> "capacity": "MEDIUM",
> >> >> "last_modified": 1430279551744,
> >> >> "fact_table": "TEST_MAX_ON_NEG",
> >> >> "null_string": null,
> >> >> "filter_condition": null,
> >> >> "cube_partition_desc": {
> >> >> "partition_date_column": null,
> >> >> "partition_date_start": 0,
> >> >> "cube_partition_type": "APPEND"
> >> >> },
> >> >> "hbase_mapping": {
> >> >> "column_family": [
> >> >> {
> >> >> "name": "F1",
> >> >> "columns": [
> >> >> {
> >> >> "qualifier": "M",
> >> >> "measure_refs": [
> >> >> "_COUNT_",
> >> >> "SUM_TRANS_AT",
> >> >> "MAX_TRANS_AT"
> >> >> ]
> >> >> }
> >> >> ]
> >> >> }
> >> >> ]
> >> >> },
> >> >> "notify_list": []
> >> >> }
> >> >>
> >> >> The cube is based on only a fact table called test_max_on_neg which
> >> >> has only six records:
> >> >>
> >> >> 2c9083862fe1d8f00130062167480ce8-211316617 -199.6
> >> >> 2c9083862fe1d8f00130062167480ce8-21782293 -229.0
> >> >> 2c9083862fe1d8f00130062167480ce8-211955199 -151.7
> >> >> 2c9083862fe1d8f00130062167480ce8-213582789 -86.4
> >> >> 2c9083862fe1d8f00130062167480ce8-115552384 -353.5
> >> >> 2c9083862fe1d8f00130062167480ce8-0114097017 -59.0
> >> >>
> >> >> When the cube is ready, we did the query: select id, max(trans_at),
> >> >> sum(trans_at) from test_max_on_neg group by id, which returned
> >> >>4.9E-324 for MAX(trans_at) column. I think the problem still exists
> >> >>for kylin-0.6.6, and you can reproduce the problem if you like.
> >> >>
> >> >> > -----邮件原件-----
> >> >> > 发件人: Li Yang [mailto:liyang@apache.org]
> >> >> > 发送时间: 2015年4月29日 14:56
> >> >> > 收件人: dev@kylin.incubator.apache.org
> >> >> > 主题: Re: Max on negative double values is not working
> >> >> >
> >> >> > Depend on whether max(trans_at) is a measure of cube or not, the
> >> >> > problem and the answer will be different.
> >> >> >
> >> >> >
> >> >> > On Wed, Apr 29, 2015 at 10:43 AM, hongbin ma
> >> <ma...@apache.org>
> >> >> > wrote:
> >> >> >
> >> >> > > can you share your cube's definition? you can go to cubes tab,
> >> >> > > click on the cube, and get json tab.
> >> >> > >
> >> >> > > we'd like to help check the definition of your cube
> >> >> > >
> >> >> > > On Tue, Apr 28, 2015 at 8:01 PM, Huang Hua
> >> >> <hu...@mininglamp.com>
> >> >> > > wrote:
> >> >> > >
> >> >> > > > Hi there,
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > We are using kylin 0.6.6, and we found out the max function
> >> >> > > > might not
> >> >> > > work
> >> >> > > > for negative values in double type.
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > The fact table in Hive only has two columns: id and trans_at
> >> >> > > > and contains six records as shown below:
> >> >> > > >
> >> >> > > > Id trans_at
> >> >> > > >
> >> >> > > > 1, -59.0
> >> >> > > >
> >> >> > > > 2, -229.0
> >> >> > > >
> >> >> > > > 3, -86.4
> >> >> > > >
> >> >> > > > 4, -151.7
> >> >> > > >
> >> >> > > > 5, -199.6
> >> >> > > >
> >> >> > > > 6, -353.5
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > Then based on the fact table, we created and built a cube with
> >> >> > > > 1
> >> >> > > dimension
> >> >> > > > and 3 measures including the default _count_ one:
> >> >> > > >
> >> >> > > > Dimension: id
> >> >> > > >
> >> >> > > > Measures: max(trans_at), sum(trans_at), _count_
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > The sql used in query was: select id, MAX(trans_at),
> >> >> > > > SUM(trans_at) from the_table group by id, which produced the
> >> following output:
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > ID,EXPR$1,EXPR$2
> >> >> > > >
> >> >> > > > 1, 4.9E-324, -59.0
> >> >> > > >
> >> >> > > > 2, 4.9E-324, -229.0
> >> >> > > >
> >> >> > > > 3, 4.9E-324, -86.4
> >> >> > > >
> >> >> > > > 4, 4.9E-324, -151.7
> >> >> > > >
> >> >> > > > 5, 4.9E-324, -199.6
> >> >> > > >
> >> >> > > > 6, 4.9E-324, -353.5
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > As you can see, max(trans_at) produced 4.9E-324 for each row
> >> >> > > > which is not what we expect to see.
> >> >> > > >
> >> >> > > >
> >> >> > > >
> >> >> > > > Best Regards.
> >> >> > > >
> >> >> > > > Hua
> >> >> > > >
> >> >> > > >
> >> >> > >
> >> >> > >
> >> >> > > --
> >> >> > > Regards,
> >> >> > >
> >> >> > > *Bin Mahone | 马洪宾*
> >> >> > > Apache Kylin: http://kylin.io
> >> >> > > Github: https://github.com/binmahone
> >> >> > >
> >> >>
> >> >>
> >> >
> >> >
> >> >
> >
> >
> >
>
>