You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "Xiaoxiang Yu (Jira)" <ji...@apache.org> on 2021/02/20 07:00:00 UTC
[jira] [Updated] (KYLIN-4878) Not able to build kylin cube for
redshift datasource
[ https://issues.apache.org/jira/browse/KYLIN-4878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Xiaoxiang Yu updated KYLIN-4878:
--------------------------------
Fix Version/s: v3.1.2
> Not able to build kylin cube for redshift datasource
> ----------------------------------------------------
>
> Key: KYLIN-4878
> URL: https://issues.apache.org/jira/browse/KYLIN-4878
> Project: Kylin
> Issue Type: Bug
> Reporter: vipul sikka
> Priority: Major
> Fix For: v3.1.2
>
> Attachments: kylin_settings.txt
>
>
> I am using kylin 3.1.0 and try to use redshift as our datasource. When we build any cube it keeps hanging and no job created for it too. The logs we are gettiing for cube 'Sales_Cube_clone' are:-
> cube.CubeManager:373 : Updating cube instance 'Sales_Cube_clone'
> 2021-01-18 07:12:03,740 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20181004021000_20190704063000]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:465 : NEW segment end does not fit/connect with other segments: Sales_Cube_clone[20181004021000_20190704063000]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20190901010500_20200602052500]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:465 : NEW segment end does not fit/connect with other segments: Sales_Cube_clone[20190901010500_20200602052500]
> 2021-01-18 07:12:03,742 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20210101000000_20210107073500]
>
> 2021-01-18 07:11:12,783 WARN [http-bio-7070-exec-1] annotation.ExceptionHandlerExceptionResolver:136 : Resolved [org.apache.kylin.rest.exception.InternalErrorException: Segments overlap: Sales_Cube_clone[20210101000000_20210107073500] and Sales_Cube_clone[20210101031500_20210114105000]]
> 2021-01-18 07:11:30,297 INFO [BadQueryDetector] service.BadQueryDetector:148 : Detect bad query.
> 2021-01-18 07:11:33,255 INFO [FetcherRunner 1778655433-46] threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 6 already succeed, 4 error, 1 discarded, 0 others
> 2021-01-18 07:12:03,255 INFO [FetcherRunner 1778655433-46] threadpool.DefaultFetcherRunner:115 : Job Fetcher: 0 should running, 0 actual running, 0 stopped, 0 ready, 6 already succeed, 4 error, 1 discarded, 0 others
> 2021-01-18 07:12:03,739 INFO [http-bio-7070-exec-6] cube.CubeManager:373 : Updating cube instance 'Sales_Cube_clone'
> 2021-01-18 07:12:03,740 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20181004021000_20190704063000]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:465 : NEW segment end does not fit/connect with other segments: Sales_Cube_clone[20181004021000_20190704063000]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20190901010500_20200602052500]
> 2021-01-18 07:12:03,741 WARN [http-bio-7070-exec-6] model.Segments:465 : NEW segment end does not fit/connect with other segments: Sales_Cube_clone[20190901010500_20200602052500]
> 2021-01-18 07:12:03,742 WARN [http-bio-7070-exec-6] model.Segments:463 : NEW segment start does not fit/connect with other segments: Sales_Cube_clone[20210101000000_20210107073500]
> 2021-01-18 07:12:03,742 DEBUG [http-bio-7070-exec-6] cachesync.CachedCrudAssist:223 : Saving CubeInstance at /cube/Sales_Cube_clone.json
> 2021-01-18 07:12:03,754 DEBUG [pool-5-thread-1] cachesync.Broadcaster:119 : Servers in the cluster: [localhost:7070]
> 2021-01-18 07:12:03,754 DEBUG [pool-5-thread-1] cachesync.Broadcaster:129 : Announcing new broadcast to all: BroadcastEvent{entity=cube, event=update, cacheKey=Sales_Cube_clone}
> 2021-01-18 07:12:03,761 DEBUG [http-bio-7070-exec-4] cachesync.Broadcaster:267 : Broadcasting UPDATE, cube, Sales_Cube_clone
> 2021-01-18 07:12:03,767 DEBUG [http-bio-7070-exec-4] cachesync.Broadcaster:267 : Broadcasting UPDATE, project_data, Demo
> 2021-01-18 07:12:03,768 INFO [http-bio-7070-exec-4] service.CacheService:123 : cleaning cache for project Demo (currently remove all entries)
> 2021-01-18 07:12:03,769 DEBUG [http-bio-7070-exec-4] cachesync.Broadcaster:301 : Done broadcasting UPDATE, project_data, Demo
> 2021-01-18 07:12:03,770 DEBUG [http-bio-7070-exec-4] cachesync.Broadcaster:301 : Done broadcasting UPDATE, cube, Sales_Cube_clone
> 2021-01-18 07:12:04,007 DEBUG [http-bio-7070-exec-8] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
> 2021-01-18 07:12:04,007 INFO [http-bio-7070-exec-8] common.KylinConfig:352 : Use KYLIN_HOME=/home/admin/apache-kylin-3.1.0-bin-hbase1x
> 2021-01-18 07:12:04,008 DEBUG [http-bio-7070-exec-1] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
> 2021-01-18 07:12:04,009 INFO [http-bio-7070-exec-1] common.KylinConfig:352 : Use KYLIN_HOME=/home/admin/apache-kylin-3.1.0-bin-hbase1x
> 2021-01-18 07:12:04,010 DEBUG [http-bio-7070-exec-14] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
> 2021-01-18 07:12:04,010 INFO [http-bio-7070-exec-14] common.KylinConfig:352 : Use KYLIN_HOME=/home/admin/apache-kylin-3.1.0-bin-hbase1x
> 2021-01-18 07:12:04,176 DEBUG [http-bio-7070-exec-2] common.KylinConfig:346 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
> 2021-01-18 07:12:04,176 INFO [http-bio-7070-exec-2] common.KylinConfig:352 : Use KYLIN_HOME=/home/admin/apache-kylin-3.1.0-bin-hbase1x
> 2021-01-18 07:12:04,204 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : active_dbo,null
> 2021-01-18 07:12:04,205 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : active_stage,null
> 2021-01-18 07:12:04,206 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : admin,null
> 2021-01-18 07:12:04,206 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : audit_sysmgmt,null
> 2021-01-18 07:12:04,206 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi,null
> 2021-01-18 07:12:04,206 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi_dimensions,null
> 2021-01-18 07:12:04,207 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi_imei,null
> 2021-01-18 07:12:04,207 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi_purchase,null
> 2021-01-18 07:12:04,207 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi_sales,null
> 2021-01-18 07:12:04,208 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : bi_stocks,null
> 2021-01-18 07:12:04,208 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : cf_dbo,null
> 2021-01-18 07:12:04,208 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : cf_stage,null
> 2021-01-18 07:12:04,208 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : collection_dbo,null
> 2021-01-18 07:12:04,209 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : collection_stage,null
> 2021-01-18 07:12:04,209 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : cvs_dbo,null
> 2021-01-18 07:12:04,209 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : cvs_reconcile,null
> 2021-01-18 07:12:04,209 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : cvs_stage,null
> 2021-01-18 07:12:04,210 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dealer_dbo,null
> 2021-01-18 07:12:04,210 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dealer_stage,null
> 2021-01-18 07:12:04,210 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : delivery_dbo,null
> 2021-01-18 07:12:04,210 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : delivery_stage,null
> 2021-01-18 07:12:04,211 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : demand_forecast_dbo,null
> 2021-01-18 07:12:04,211 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : demand_forecast_stage,null
> 2021-01-18 07:12:04,211 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dss_bi,null
> 2021-01-18 07:12:04,211 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dss_dbo,null
> 2021-01-18 07:12:04,211 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dss_stage,null
> 2021-01-18 07:12:04,212 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : dss_sysmgmt,null
> 2021-01-18 07:12:04,212 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : edi_dbo,null
> 2021-01-18 07:12:04,212 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : edi_stage,null
> 2021-01-18 07:12:04,213 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : erp_dbo,null
> 2021-01-18 07:12:04,213 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : erp_stage,null
> 2021-01-18 07:12:04,213 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : estore_dbo,null
> 2021-01-18 07:12:04,213 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : estore_stage,null
> 2021-01-18 07:12:04,214 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : finaldevopstest_dbo,null
> 2021-01-18 07:12:04,214 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : finance_dbo,null
> 2021-01-18 07:12:04,214 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : finance_stage,null
> 2021-01-18 07:12:04,214 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : hyke_dbo,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : hyke_ppi_dbo,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : hyke_ppi_stage,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : hyke_stage,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : information_schema,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : inventory_dbo,null
> 2021-01-18 07:12:04,215 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : inventory_stage,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : invoice_dbo,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : invoice_stage,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : om_dbo,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : om_stage,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : optimiza_dbo,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : optimiza_stage,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : order_dbo,null
> 2021-01-18 07:12:04,216 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : order_stage,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pg_catalog,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pg_internal,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pms_dbo,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pms_stage,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pricing_dbo,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : pricing_stage,null
> 2021-01-18 07:12:04,217 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : public,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : service_dbo,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : service_stage,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : shipment_dbo,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : shipment_stage,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : stg_axiombi_dbo,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : stg_axiombi_stage,null
> 2021-01-18 07:12:04,218 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : stg_hykebi_dbo,null
> 2021-01-18 07:12:04,219 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : supplier_visibility,null
> 2021-01-18 07:12:04,219 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : supplier_visibility_dbo,null
> 2021-01-18 07:12:04,219 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : supplier_visibility_stage,null
> 2021-01-18 07:12:04,219 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : uts_dbo,null
> 2021-01-18 07:12:04,220 INFO [http-bio-7070-exec-6] metadata.DefaultJdbcMetadata:48 : uts_stage,null
>
> Also find system settings for kylin
--
This message was sent by Atlassian Jira
(v8.3.4#803005)