You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by ShaoFeng Shi <sh...@apache.org> on 2016/09/01 01:04:01 UTC
Re: Empty Cube
Hi Mehrdad,
When you run the query, is there any error reported in the logs/kylin.log?
If no error, please check whether the source hive table has data, you can
do this easily in hive, by running HQL "select * from default.kylin_sales
limit 100"; it is very likely that the "sample.sh" failed to upload the
file; if it is this case, you may need manually import the data files
(sample_cube/data) to hive;
2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
> Hi All,
>
> When I run the sample cube of Kylin the result is empty, i.e. the cube is
> computed successfully, the status is ready but nothing is in the cube and
> every query gets zero.
> Do you have any suggestion what can cause the problem?
> How should I debug it?
>
> Thanks,
> Mehrdad
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
--
Best regards,
Shaofeng Shi
Re: Empty Cube
Posted by ShaoFeng Shi <sh...@apache.org>.
I also encountered the "empty cube" issue in an environment yesterday; By
checking Hive I found one of the sample table
"DEFAULT.KYLIN_CATEGORY_GROUPINGS" has no record. Since the "kylin_sales"
cube uses "inner" joins for all tables, causing the cube no data.
After some troubleshoting I identify it is an issue of the "beeline" client
(my env configures beeline as hive cli in kylin), when it executes the
sample_cube/create_sample_tables.sql, the last line wasn't executed, which
caused the csv file wasn't uploaded.
By adding a blank line in the end of this sql file, it runs successfully. I
will checkin this change in kylin soon. If you faced the same issue, please
take a try.
2016-09-09 13:47 GMT+08:00 ShaoFeng Shi <sh...@apache.org>:
> Hi Niasari,
>
> During the query time, Kylin will not touch Hive, all queries will go to
> HBase, so you still get results on kylin_cal_dt; If you also want to remove
> that also in Kylin, please disable and then drop the cube.
>
> For the empty cube, it is possible and allowed in Kylin; Thinking that an
> user uses incremental build, and for a time period there is no source data,
> so empty cube is expected. Of course you can configure Kylin to let it
> report error when there is no source record, see: https://issues.apache.
> org/jira/browse/KYLIN-1752
>
> For the empty cube issue, I guess there is some thing wrong, making no
> data be built into cube. If you want to get remote debug or support, please
> contact me offline.
>
>
>
> 2016-09-07 5:17 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>
>> The range is up to 2015 and that’s not the problem.
>>
>>
>>
>> The only query that doesn’t return zero/empty result is from table
>> kylin_cal_dt. When I delete this table from hive, still the result is not
>> empty and Kylin doesn’t complain! Where does Kylin bring the data of this
>> table even when it’s not on hive?
>>
>> Also I checked the cube itself on Hbase and it’s empty. Any other
>> suggestion? Can it be because of some wrong configuration in connecting
>> Kylin to Hive/Hbase? If that’s the case why it can create the cube
>> successfully?
>>
>>
>>
>> Also I have a question regarding uuid. What is this for?
>>
>> I’m asking this cause I cannot fully understand that why you have used a
>> fixed value in sample cube json files (for example for kylin_sales_cube
>> uuid is: “2fbca32a-a33e-4b69-83dd-0bb8b1f8c53b”). How come it doesn’t
>> conflict if you run the sample cube multiple times?
>>
>>
>>
>> Thanks,
>>
>> Mehrdad
>>
>>
>>
>> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
>> *Sent:* 2016, September, 01 9:01 PM
>>
>> *To:* user
>> *Subject:* Re: Empty Cube
>>
>>
>>
>> ok; The table "kylin_sales" contains data for 2012 and 2013, so need make
>> sure the build range covers these years; please have a check and I guess it
>> is some minor mistake; BTW, empty cube will return zero since it doesn't
>> have data.
>>
>>
>>
>> 2016-09-02 1:34 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>>
>> Thanks for the reply.
>>
>>
>>
>> There is no error in the log. Also I checked the hive tables and they are
>> not empty; I can run any query on them using Hue and work well.
>>
>>
>>
>> Also the size of cube in the cubes section in 0.00KB! But even with an
>> empty cube the result of queries on the tables should not be empty/zero,
>> right?
>>
>> Any other suggestion how to debug it?
>>
>>
>>
>> Thanks,
>>
>> Mehrdad
>>
>>
>>
>> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
>> *Sent:* 2016, August, 31 9:04 PM
>> *To:* user
>> *Subject:* Re: Empty Cube
>>
>>
>>
>> Hi Mehrdad,
>>
>>
>>
>> When you run the query, is there any error reported in the logs/kylin.log?
>>
>>
>>
>> If no error, please check whether the source hive table has data, you can
>> do this easily in hive, by running HQL "select * from default.kylin_sales
>> limit 100"; it is very likely that the "sample.sh" failed to upload the
>> file; if it is this case, you may need manually import the data files
>> (sample_cube/data) to hive;
>>
>>
>>
>> 2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>>
>> Hi All,
>>
>>
>>
>> When I run the sample cube of Kylin the result is empty, i.e. the cube is
>> computed successfully, the status is ready but nothing is in the cube and
>> every query gets zero.
>>
>> Do you have any suggestion what can cause the problem?
>>
>> How should I debug it?
>>
>>
>>
>> Thanks,
>>
>> Mehrdad
>>
>>
>>
>>
>>
>> _______________________________________________________________________
>>
>> If you received this email in error, please advise the sender (by return
>> email or otherwise) immediately. You have consented to receive the attached
>> electronically at the above-noted email address; please retain a copy of
>> this confirmation for future reference.
>>
>> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
>> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
>> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
>> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
>> cette confirmation pour les fins de reference future.
>>
>>
>>
>>
>>
>> --
>>
>> Best regards,
>>
>>
>>
>> Shaofeng Shi
>>
>>
>>
>> _______________________________________________________________________
>>
>> If you received this email in error, please advise the sender (by return
>> email or otherwise) immediately. You have consented to receive the attached
>> electronically at the above-noted email address; please retain a copy of
>> this confirmation for future reference.
>>
>> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
>> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
>> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
>> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
>> cette confirmation pour les fins de reference future.
>>
>>
>>
>>
>>
>> --
>>
>> Best regards,
>>
>>
>>
>> Shaofeng Shi
>>
>>
>>
>> _______________________________________________________________________
>>
>> If you received this email in error, please advise the sender (by return
>> email or otherwise) immediately. You have consented to receive the attached
>> electronically at the above-noted email address; please retain a copy of
>> this confirmation for future reference.
>>
>> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
>> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
>> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
>> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
>> cette confirmation pour les fins de reference future.
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi
>
>
--
Best regards,
Shaofeng Shi 史少锋
Re: Empty Cube
Posted by ShaoFeng Shi <sh...@apache.org>.
Hi Niasari,
During the query time, Kylin will not touch Hive, all queries will go to
HBase, so you still get results on kylin_cal_dt; If you also want to remove
that also in Kylin, please disable and then drop the cube.
For the empty cube, it is possible and allowed in Kylin; Thinking that an
user uses incremental build, and for a time period there is no source data,
so empty cube is expected. Of course you can configure Kylin to let it
report error when there is no source record, see:
https://issues.apache.org/jira/browse/KYLIN-1752
For the empty cube issue, I guess there is some thing wrong, making no data
be built into cube. If you want to get remote debug or support, please
contact me offline.
2016-09-07 5:17 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
> The range is up to 2015 and that’s not the problem.
>
>
>
> The only query that doesn’t return zero/empty result is from table
> kylin_cal_dt. When I delete this table from hive, still the result is not
> empty and Kylin doesn’t complain! Where does Kylin bring the data of this
> table even when it’s not on hive?
>
> Also I checked the cube itself on Hbase and it’s empty. Any other
> suggestion? Can it be because of some wrong configuration in connecting
> Kylin to Hive/Hbase? If that’s the case why it can create the cube
> successfully?
>
>
>
> Also I have a question regarding uuid. What is this for?
>
> I’m asking this cause I cannot fully understand that why you have used a
> fixed value in sample cube json files (for example for kylin_sales_cube
> uuid is: “2fbca32a-a33e-4b69-83dd-0bb8b1f8c53b”). How come it doesn’t
> conflict if you run the sample cube multiple times?
>
>
>
> Thanks,
>
> Mehrdad
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* 2016, September, 01 9:01 PM
>
> *To:* user
> *Subject:* Re: Empty Cube
>
>
>
> ok; The table "kylin_sales" contains data for 2012 and 2013, so need make
> sure the build range covers these years; please have a check and I guess it
> is some minor mistake; BTW, empty cube will return zero since it doesn't
> have data.
>
>
>
> 2016-09-02 1:34 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>
> Thanks for the reply.
>
>
>
> There is no error in the log. Also I checked the hive tables and they are
> not empty; I can run any query on them using Hue and work well.
>
>
>
> Also the size of cube in the cubes section in 0.00KB! But even with an
> empty cube the result of queries on the tables should not be empty/zero,
> right?
>
> Any other suggestion how to debug it?
>
>
>
> Thanks,
>
> Mehrdad
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* 2016, August, 31 9:04 PM
> *To:* user
> *Subject:* Re: Empty Cube
>
>
>
> Hi Mehrdad,
>
>
>
> When you run the query, is there any error reported in the logs/kylin.log?
>
>
>
> If no error, please check whether the source hive table has data, you can
> do this easily in hive, by running HQL "select * from default.kylin_sales
> limit 100"; it is very likely that the "sample.sh" failed to upload the
> file; if it is this case, you may need manually import the data files
> (sample_cube/data) to hive;
>
>
>
> 2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>
> Hi All,
>
>
>
> When I run the sample cube of Kylin the result is empty, i.e. the cube is
> computed successfully, the status is ready but nothing is in the cube and
> every query gets zero.
>
> Do you have any suggestion what can cause the problem?
>
> How should I debug it?
>
>
>
> Thanks,
>
> Mehrdad
>
>
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
--
Best regards,
Shaofeng Shi
RE: Empty Cube
Posted by "Niasari, Mehrdad" <me...@rbc.com>.
The range is up to 2015 and that’s not the problem.
The only query that doesn’t return zero/empty result is from table kylin_cal_dt. When I delete this table from hive, still the result is not empty and Kylin doesn’t complain! Where does Kylin bring the data of this table even when it’s not on hive?
Also I checked the cube itself on Hbase and it’s empty. Any other suggestion? Can it be because of some wrong configuration in connecting Kylin to Hive/Hbase? If that’s the case why it can create the cube successfully?
Also I have a question regarding uuid. What is this for?
I’m asking this cause I cannot fully understand that why you have used a fixed value in sample cube json files (for example for kylin_sales_cube uuid is: “2fbca32a-a33e-4b69-83dd-0bb8b1f8c53b”). How come it doesn’t conflict if you run the sample cube multiple times?
Thanks,
Mehrdad
From: ShaoFeng Shi [mailto:shaofengshi@apache.org]
Sent: 2016, September, 01 9:01 PM
To: user
Subject: Re: Empty Cube
ok; The table "kylin_sales" contains data for 2012 and 2013, so need make sure the build range covers these years; please have a check and I guess it is some minor mistake; BTW, empty cube will return zero since it doesn't have data.
2016-09-02 1:34 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>>:
Thanks for the reply.
There is no error in the log. Also I checked the hive tables and they are not empty; I can run any query on them using Hue and work well.
Also the size of cube in the cubes section in 0.00KB! But even with an empty cube the result of queries on the tables should not be empty/zero, right?
Any other suggestion how to debug it?
Thanks,
Mehrdad
From: ShaoFeng Shi [mailto:shaofengshi@apache.org<ma...@apache.org>]
Sent: 2016, August, 31 9:04 PM
To: user
Subject: Re: Empty Cube
Hi Mehrdad,
When you run the query, is there any error reported in the logs/kylin.log?
If no error, please check whether the source hive table has data, you can do this easily in hive, by running HQL "select * from default.kylin_sales limit 100"; it is very likely that the "sample.sh" failed to upload the file; if it is this case, you may need manually import the data files (sample_cube/data) to hive;
2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>>:
Hi All,
When I run the sample cube of Kylin the result is empty, i.e. the cube is computed successfully, the status is ready but nothing is in the cube and every query gets zero.
Do you have any suggestion what can cause the problem?
How should I debug it?
Thanks,
Mehrdad
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.
Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
--
Best regards,
Shaofeng Shi
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.
Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
--
Best regards,
Shaofeng Shi
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.
Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
Re: Empty Cube
Posted by ShaoFeng Shi <sh...@apache.org>.
ok; The table "kylin_sales" contains data for 2012 and 2013, so need make
sure the build range covers these years; please have a check and I guess it
is some minor mistake; BTW, empty cube will return zero since it doesn't
have data.
2016-09-02 1:34 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
> Thanks for the reply.
>
>
>
> There is no error in the log. Also I checked the hive tables and they are
> not empty; I can run any query on them using Hue and work well.
>
>
>
> Also the size of cube in the cubes section in 0.00KB! But even with an
> empty cube the result of queries on the tables should not be empty/zero,
> right?
>
> Any other suggestion how to debug it?
>
>
>
> Thanks,
>
> Mehrdad
>
>
>
> *From:* ShaoFeng Shi [mailto:shaofengshi@apache.org]
> *Sent:* 2016, August, 31 9:04 PM
> *To:* user
> *Subject:* Re: Empty Cube
>
>
>
> Hi Mehrdad,
>
>
>
> When you run the query, is there any error reported in the logs/kylin.log?
>
>
>
> If no error, please check whether the source hive table has data, you can
> do this easily in hive, by running HQL "select * from default.kylin_sales
> limit 100"; it is very likely that the "sample.sh" failed to upload the
> file; if it is this case, you may need manually import the data files
> (sample_cube/data) to hive;
>
>
>
> 2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>:
>
> Hi All,
>
>
>
> When I run the sample cube of Kylin the result is empty, i.e. the cube is
> computed successfully, the status is ready but nothing is in the cube and
> every query gets zero.
>
> Do you have any suggestion what can cause the problem?
>
> How should I debug it?
>
>
>
> Thanks,
>
> Mehrdad
>
>
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
>
>
>
> --
>
> Best regards,
>
>
>
> Shaofeng Shi
>
>
>
> _______________________________________________________________________
>
> If you received this email in error, please advise the sender (by return
> email or otherwise) immediately. You have consented to receive the attached
> electronically at the above-noted email address; please retain a copy of
> this confirmation for future reference.
>
> Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur
> immédiatement, par retour de courriel ou par un autre moyen. Vous avez
> accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à
> l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de
> cette confirmation pour les fins de reference future.
>
>
--
Best regards,
Shaofeng Shi
RE: Empty Cube
Posted by "Niasari, Mehrdad" <me...@rbc.com>.
Thanks for the reply.
There is no error in the log. Also I checked the hive tables and they are not empty; I can run any query on them using Hue and work well.
Also the size of cube in the cubes section in 0.00KB! But even with an empty cube the result of queries on the tables should not be empty/zero, right?
Any other suggestion how to debug it?
Thanks,
Mehrdad
From: ShaoFeng Shi [mailto:shaofengshi@apache.org]
Sent: 2016, August, 31 9:04 PM
To: user
Subject: Re: Empty Cube
Hi Mehrdad,
When you run the query, is there any error reported in the logs/kylin.log?
If no error, please check whether the source hive table has data, you can do this easily in hive, by running HQL "select * from default.kylin_sales limit 100"; it is very likely that the "sample.sh" failed to upload the file; if it is this case, you may need manually import the data files (sample_cube/data) to hive;
2016-09-01 4:06 GMT+08:00 Niasari, Mehrdad <me...@rbc.com>>:
Hi All,
When I run the sample cube of Kylin the result is empty, i.e. the cube is computed successfully, the status is ready but nothing is in the cube and every query gets zero.
Do you have any suggestion what can cause the problem?
How should I debug it?
Thanks,
Mehrdad
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.
Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.
--
Best regards,
Shaofeng Shi
_______________________________________________________________________
If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference.
Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.