You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by "Kumar, Manoj H" <ma...@jpmorgan.com> on 2018/04/02 14:26:56 UTC
RE: kylin 2.3.1 - System Cube Tables
Can it be possible to create the table in project Database as we don't have permission to create new databases KYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by "Kumar, Manoj H" <ma...@jpmorgan.com>.
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To: user@kylin.apache.org
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Can it be possible to create the table in project Database as we don't have permission to create new databases KYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
Re:RE: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by Ma Gang <mg...@163.com>.
Hi Manoj,
We used it, but use the default configuration. Did you resolve the signature issue? If no, you may clean system tables/cubes manually, and regenerate again. For how data is loaded, the data is generated to hive table automatically when query and job build happens, you can use cronjob to trigger the system cube build, you may refer the "System Cube Build" section for how to do system cube build: http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html
At 2018-04-06 20:40:02, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Any updated on this?? Has any one used this system Cube build process fully? Pls. advise.
From: Kumar, Manoj H
Sent: Thursday, April 05, 2018 2:07 PM
To:user@kylin.apache.org
Subject: RE: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for your inputs.. I have re-loaded it & now its showing up correct schema. But I am getting different error – Although I have refresh the cube signature using metastore.sh but it didn’t work. Also, How to load the data into Kylin system tables. Will this gets done when we do cube build?
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Thursday, April 05, 2018 8:51 AM
To:user@kylin.apache.org
Subject: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Did you re-generate the system cube's metadata after add the configuration "kylin.metrics.prefix=db_rft_rcmo_rfda" in the kylin.properties?
http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html
At 2018-04-04 17:59:53, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Its not working even though I have set up the properties. In jSON file, I have given the DB name db_rft_rcmo_rfda. Why its creating DB name as Kylin? How can we change it & make it configurable Hive DB name.
-bash-4.1$ ls -ltr
total 20
-rw------- 1 a_rcmo_nd a_rcmo_grp 1976 Apr 2 08:59 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2480 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_CUBE_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1863 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_RPC_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2306 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1624 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_EXCEPTION_QA.json
Regards,
Manoj
From: Kumar, Manoj H
Sent: Wednesday, April 04, 2018 3:10 PM
To: 'user@kylin.apache.org' <us...@kylin.apache.org>
Subject: RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To:user@kylin.apache.org
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Can it be possible to create the table in project Database as we don’t have permission to create new databasesKYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
RE: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by "Kumar, Manoj H" <ma...@jpmorgan.com>.
Any updated on this?? Has any one used this system Cube build process fully? Pls. advise.
From: Kumar, Manoj H
Sent: Thursday, April 05, 2018 2:07 PM
To: user@kylin.apache.org
Subject: RE: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for your inputs.. I have re-loaded it & now its showing up correct schema. But I am getting different error - Although I have refresh the cube signature using metastore.sh but it didn't work. Also, How to load the data into Kylin system tables. Will this gets done when we do cube build?
[cid:image001.png@01D3CDD2.751F05F0]
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Thursday, April 05, 2018 8:51 AM
To: user@kylin.apache.org<ma...@kylin.apache.org>
Subject: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Did you re-generate the system cube's metadata after add the configuration "kylin.metrics.prefix=db_rft_rcmo_rfda" in the kylin.properties?
http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html<https://secureweb.jpmchase.net/readonly/http:/kylin.apache.org/docs23/howto/howto_setup_systemcube.html>
At 2018-04-04 17:59:53, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Its not working even though I have set up the properties. In jSON file, I have given the DB name db_rft_rcmo_rfda. Why its creating DB name as Kylin? How can we change it & make it configurable Hive DB name.
-bash-4.1$ ls -ltr
total 20
-rw------- 1 a_rcmo_nd a_rcmo_grp 1976 Apr 2 08:59 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2480 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_CUBE_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1863 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_RPC_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2306 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1624 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_EXCEPTION_QA.json
[cid:image002.png@01D3CDD2.751F05F0]
Regards,
Manoj
From: Kumar, Manoj H
Sent: Wednesday, April 04, 2018 3:10 PM
To: 'user@kylin.apache.org<ma...@kylin.apache.org>' <us...@kylin.apache.org>>
Subject: RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To: user@kylin.apache.org<ma...@kylin.apache.org>
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Can it be possible to create the table in project Database as we don't have permission to create new databases KYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
RE: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by "Kumar, Manoj H" <ma...@jpmorgan.com>.
Thanks for your inputs.. I have re-loaded it & now its showing up correct schema. But I am getting different error - Although I have refresh the cube signature using metastore.sh but it didn't work. Also, How to load the data into Kylin system tables. Will this gets done when we do cube build?
[cid:image001.png@01D3CCE7.5F08A4B0]
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Thursday, April 05, 2018 8:51 AM
To: user@kylin.apache.org
Subject: Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Did you re-generate the system cube's metadata after add the configuration "kylin.metrics.prefix=db_rft_rcmo_rfda" in the kylin.properties?
http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html<https://secureweb.jpmchase.net/readonly/http:/kylin.apache.org/docs23/howto/howto_setup_systemcube.html>
At 2018-04-04 17:59:53, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Its not working even though I have set up the properties. In jSON file, I have given the DB name db_rft_rcmo_rfda. Why its creating DB name as Kylin? How can we change it & make it configurable Hive DB name.
-bash-4.1$ ls -ltr
total 20
-rw------- 1 a_rcmo_nd a_rcmo_grp 1976 Apr 2 08:59 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2480 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_CUBE_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1863 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_RPC_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2306 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1624 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_EXCEPTION_QA.json
[cid:image002.png@01D3CCE7.5F08A4B0]
Regards,
Manoj
From: Kumar, Manoj H
Sent: Wednesday, April 04, 2018 3:10 PM
To: 'user@kylin.apache.org<ma...@kylin.apache.org>' <us...@kylin.apache.org>>
Subject: RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To: user@kylin.apache.org<ma...@kylin.apache.org>
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Can it be possible to create the table in project Database as we don't have permission to create new databases KYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
Re:RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by Ma Gang <mg...@163.com>.
Did you re-generate the system cube's metadata after add the configuration "kylin.metrics.prefix=db_rft_rcmo_rfda" in the kylin.properties?
http://kylin.apache.org/docs23/howto/howto_setup_systemcube.html
At 2018-04-04 17:59:53, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Its not working even though I have set up the properties. In jSON file, I have given the DB name db_rft_rcmo_rfda. Why its creating DB name as Kylin? How can we change it & make it configurable Hive DB name.
-bash-4.1$ ls -ltr
total 20
-rw------- 1 a_rcmo_nd a_rcmo_grp 1976 Apr 2 08:59 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2480 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_CUBE_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1863 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_RPC_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2306 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1624 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_EXCEPTION_QA.json
Regards,
Manoj
From: Kumar, Manoj H
Sent: Wednesday, April 04, 2018 3:10 PM
To: 'user@kylin.apache.org' <us...@kylin.apache.org>
Subject: RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To:user@kylin.apache.org
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Can it be possible to create the table in project Database as we don’t have permission to create new databasesKYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
RE: Re:RE: kylin 2.3.1 - System Cube Tables
Posted by "Kumar, Manoj H" <ma...@jpmorgan.com>.
Its not working even though I have set up the properties. In jSON file, I have given the DB name db_rft_rcmo_rfda. Why its creating DB name as Kylin? How can we change it & make it configurable Hive DB name.
-bash-4.1$ ls -ltr
total 20
-rw------- 1 a_rcmo_nd a_rcmo_grp 1976 Apr 2 08:59 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2480 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_CUBE_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1863 Apr 2 09:00 db_rft_rcmo_rfda.HIVE_METRICS_QUERY_RPC_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 2306 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_QA.json
-rw------- 1 a_rcmo_nd a_rcmo_grp 1624 Apr 2 09:01 db_rft_rcmo_rfda.HIVE_METRICS_JOB_EXCEPTION_QA.json
[cid:image001.png@01D3CC29.C2ABC9C0]
Regards,
Manoj
From: Kumar, Manoj H
Sent: Wednesday, April 04, 2018 3:10 PM
To: 'user@kylin.apache.org' <us...@kylin.apache.org>
Subject: RE: Re:RE: kylin 2.3.1 - System Cube Tables
Thanks for the input.. Let me try it out & I will update you.
Regards,
Manoj
From: Ma Gang [mailto:mg4work@163.com]
Sent: Wednesday, April 04, 2018 2:30 PM
To: user@kylin.apache.org<ma...@kylin.apache.org>
Subject: Re:RE: kylin 2.3.1 - System Cube Tables
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com>> wrote:
Can it be possible to create the table in project Database as we don't have permission to create new databases KYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer<http://www.jpmorgan.com/emaildisclaimer> including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.
Re:RE: kylin 2.3.1 - System Cube Tables
Posted by Ma Gang <mg...@163.com>.
Hi Manoj,
You can configure it using "kylin.metrics.prefix" configuration in the kylin.properties file. Maybe it is better to use "DEFAULT" for this configuration by default.
At 2018-04-02 22:26:56, "Kumar, Manoj H" <ma...@jpmorgan.com> wrote:
Can it be possible to create the table in project Database as we don’t have permission to create new databasesKYLIN? How can we use the same hive DB used for Kylin Cube? Is there any way to change it? Pls. advise.
Regards,
Manoj
This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.