You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@kylin.apache.org by Jason Tang <ja...@striking.ly> on 2019/07/08 05:37:41 UTC
Kylin Build get stuck at Build Dimension Dictionary
Kylin cube build jod get stuck at “Build Dimension Dictionary” step. No error is shown in the log, as below:
2019-07-08 03:55:37,374 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280 : Done broadcasting UPDATE, cube, sxl_pageviews_clone
2019-07-08 03:55:37,432 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] steps.CreateDictionaryJob:96 : Dict for 'IP_ADDRESS' not pre-built.
2019-07-08 03:55:37,432 DEBUG [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] cli.DictionaryGeneratorCLI:76 : Dict for 'IP_ADDRESS' not pre-built, build it from hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS
2019-07-08 03:55:37,482 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] dict.DictionaryManager:276 : building dictionary for DEFAULT.SXL_PAGEVIEWS_PARQUET.IP_ADDRESS
2019-07-08 03:55:37,538 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] dict.DictionaryManager:285 : Building dictionary object {"uuid":"459ff7cf-ceb3-b751-8b03-f7d77b012d22","last_modified":0,"version":"2.6.1.0","source_table":"DEFAULT.SXL_PAGEVIEWS_PARQUET","source_column":"IP_ADDRESS","source_column_index":5,"data_type":"varchar(256)","input":{"path":"hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS","size":1180625,"last_modified_time":1562557443756},"dictionary_class":null,"cardinality":0}
2019-07-08 03:55:37,721 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] mr.DFSFileTableReader:197 : Auto detect delim to be null, will take THE-WHOLE-LINE as a single value, for hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS/-r-00008
2019-07-08 03:55:37,724 DEBUG [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] util.ZookeeperDistributedLock:147 : 103-12468@f768cdc068ec trying to lock /kylin/kylin_metadata/dict/DEFAULT.SXL_PAGEVIEWS_PARQUET_IP_ADDRESS/lock
2019-07-08 03:55:37,727 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] util.ZookeeperDistributedLock:159 : 103-12468@f768cdc068ec acquired lock at /kylin/kylin_metadata/dict/DEFAULT.SXL_PAGEVIEWS_PARQUET_IP_ADDRESS/lock
2019-07-08 03:56:02,567 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:56:14,838 DEBUG [http-bio-7070-exec-6] common.KylinConfig:328 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2019-07-08 03:56:14,838 INFO [http-bio-7070-exec-6] common.KylinConfig:334 : Use KYLIN_HOME=/hadoop/install/kylin
2019-07-08 03:56:17,625 DEBUG [http-bio-7070-exec-9] security.KylinAuthenticationProvider:114 : User ADMIN authorities : [ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER]
2019-07-08 03:56:17,632 INFO [http-bio-7070-exec-9] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.rest.security.KylinUserManager
2019-07-08 03:56:17,632 INFO [http-bio-7070-exec-9] security.KylinUserManager:61 : Initializing KylinUserManager with config kylin_metadata@hbase
2019-07-08 03:56:17,633 DEBUG [http-bio-7070-exec-9] cachesync.CachedCrudAssist:118 : Reloading ManagedUser from kylin_metadata(key='/user')@kylin_metadata@hbase
2019-07-08 03:56:17,674 DEBUG [http-bio-7070-exec-9] cachesync.CachedCrudAssist:127 : Loaded 1 ManagedUser(s) out of 1 resource
2019-07-08 03:56:17,677 DEBUG [http-bio-7070-exec-9] security.KylinAuthenticationProvider:128 : Authenticated user org.springframework.security.authentication.UsernamePasswordAuthenticationToken@c8fb3bba: Principal: org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 10.201.101.229; SessionId: 6CCE82A0150C881F5EE0CEE7E9EFD509; Granted Authorities: ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER
2019-07-08 03:56:18,000 DEBUG [http-bio-7070-exec-9] controller.UserController:52 : User login: org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2019-07-08 03:56:19,379 DEBUG [http-bio-7070-exec-1] project.ProjectL2Cache:198 : Loading L2 project cache for user_analytics_sxl
2019-07-08 03:56:19,426 INFO [http-bio-7070-exec-1] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.realization.RealizationRegistry
2019-07-08 03:56:19,427 INFO [http-bio-7070-exec-1] realization.RealizationRegistry:54 : Initializing RealizationRegistry with metadata url kylin_metadata@hbase
2019-07-08 03:56:19,429 INFO [http-bio-7070-exec-1] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.storage.hybrid.HybridManager
2019-07-08 03:56:19,442 INFO [http-bio-7070-exec-1] hybrid.HybridManager:71 : Initializing HybridManager with config kylin_metadata@hbase
2019-07-08 03:56:19,443 DEBUG [http-bio-7070-exec-1] cachesync.CachedCrudAssist:118 : Reloading HybridInstance from kylin_metadata(key='/hybrid')@kylin_metadata@hbase
2019-07-08 03:56:19,454 DEBUG [http-bio-7070-exec-1] cachesync.CachedCrudAssist:127 : Loaded 0 HybridInstance(s) out of 0 resource
2019-07-08 03:56:19,455 INFO [http-bio-7070-exec-1] realization.RealizationRegistry:81 : RealizationRegistry is {CUBE=org.apache.kylin.cube.CubeManager@799f55ea, HYBRID=org.apache.kylin.storage.hybrid.HybridManager@1fb18590}
2019-07-08 03:56:25,507 DEBUG [http-bio-7070-exec-4] badquery.BadQueryHistoryManager:65 : Loaded 50 Bad Query(s)
2019-07-08 03:56:30,150 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:56:32,689 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:57:02,522 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:57:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:57:32,709 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:58:02,521 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:58:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:58:32,525 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:59:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:59:20,846 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-74.cn-north-1.compute.internal/10.201.1.74:16000
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-74.cn-north-1.compute.internal/10.201.1.74:16000
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-245.cn-north-1.compute.internal/10.201.1.245:16020
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-201.cn-north-1.compute.internal/10.201.1.201:16020
2019-07-08 03:59:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:59:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:00:02,521 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:00:30,232 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:00:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:01:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:01:30,272 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:01:32,844 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:02:02,695 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:02:30,273 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:02:32,775 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:03:02,648 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:03:30,413 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:03:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:04:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:04:30,413 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:04:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:05:02,862 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:05:30,414 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:05:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:06:02,922 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:06:30,414 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:06:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:07:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:07:20,937 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-245.cn-north-1.compute.internal/10.201.1.245:16020
2019-07-08 04:07:20,938 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-201.cn-north-1.compute.internal/10.201.1.201:16020
2019-07-08 04:07:30,496 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:07:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:08:00,503 DEBUG [http-bio-7070-exec-8] badquery.BadQueryHistoryManager:65 : Loaded 50 Bad Query(s)
2019-07-08 04:08:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:08:30,695 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
The amount of data is not large(as shown in the picture). Any suggestions about this situation?
Best Regards
Jason Tang
回复：Kylin Build get stuck at Build Dimension Dictionary
Posted by 240902713 <24...@qq.com>.
Hi Jason Tang ,
Have you tried to increase the memory of kylin from 16g to 32g?
Build Dimension Dictionary is executed in memory .
------------------ 原始邮件 ------------------
发件人: "Jason Tang"<ja...@striking.ly>;
发送时间: 2019年7月8日(星期一) 中午1:37
收件人: "user"<us...@kylin.apache.org>;
主题: Kylin Build get stuck at Build Dimension Dictionary
Kylin cube build jod get stuck at “Build Dimension Dictionary” step. No error is shown in the log, as below:
2019-07-08 03:55:37,374 DEBUG [http-bio-7070-exec-7] cachesync.Broadcaster:280 : Done broadcasting UPDATE, cube, sxl_pageviews_clone
2019-07-08 03:55:37,432 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] steps.CreateDictionaryJob:96 : Dict for 'IP_ADDRESS' not pre-built.
2019-07-08 03:55:37,432 DEBUG [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] cli.DictionaryGeneratorCLI:76 : Dict for 'IP_ADDRESS' not pre-built, build it from hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS
2019-07-08 03:55:37,482 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] dict.DictionaryManager:276 : building dictionary for DEFAULT.SXL_PAGEVIEWS_PARQUET.IP_ADDRESS
2019-07-08 03:55:37,538 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] dict.DictionaryManager:285 : Building dictionary object {"uuid":"459ff7cf-ceb3-b751-8b03-f7d77b012d22","last_modified":0,"version":"2.6.1.0","source_table":"DEFAULT.SXL_PAGEVIEWS_PARQUET","source_column":"IP_ADDRESS","source_column_index":5,"data_type":"varchar(256)","input":{"path":"hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS","size":1180625,"last_modified_time":1562557443756},"dictionary_class":null,"cardinality":0}
2019-07-08 03:55:37,721 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] mr.DFSFileTableReader:197 : Auto detect delim to be null, will take THE-WHOLE-LINE as a single value, for hdfs://ip-10-201-1-15.cn-north-1.compute.internal:8020/kylin/kylin_metadata/kylin-a849e172-ecd0-6438-bca3-538e151a4594/sxl_pageviews_clone/fact_distinct_columns/SXL_PAGEVIEWS_PARQUET.IP_ADDRESS/-r-00008
2019-07-08 03:55:37,724 DEBUG [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] util.ZookeeperDistributedLock:147 : 103-12468@f768cdc068ec trying to lock /kylin/kylin_metadata/dict/DEFAULT.SXL_PAGEVIEWS_PARQUET_IP_ADDRESS/lock
2019-07-08 03:55:37,727 INFO [Scheduler 132467998 Job a849e172-ecd0-6438-bca3-538e151a4594-103] util.ZookeeperDistributedLock:159 : 103-12468@f768cdc068ec acquired lock at /kylin/kylin_metadata/dict/DEFAULT.SXL_PAGEVIEWS_PARQUET_IP_ADDRESS/lock
2019-07-08 03:56:02,567 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:56:14,838 DEBUG [http-bio-7070-exec-6] common.KylinConfig:328 : KYLIN_CONF property was not set, will seek KYLIN_HOME env variable
2019-07-08 03:56:14,838 INFO [http-bio-7070-exec-6] common.KylinConfig:334 : Use KYLIN_HOME=/hadoop/install/kylin
2019-07-08 03:56:17,625 DEBUG [http-bio-7070-exec-9] security.KylinAuthenticationProvider:114 : User ADMIN authorities : [ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER]
2019-07-08 03:56:17,632 INFO [http-bio-7070-exec-9] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.rest.security.KylinUserManager
2019-07-08 03:56:17,632 INFO [http-bio-7070-exec-9] security.KylinUserManager:61 : Initializing KylinUserManager with config kylin_metadata@hbase
2019-07-08 03:56:17,633 DEBUG [http-bio-7070-exec-9] cachesync.CachedCrudAssist:118 : Reloading ManagedUser from kylin_metadata(key='/user')@kylin_metadata@hbase
2019-07-08 03:56:17,674 DEBUG [http-bio-7070-exec-9] cachesync.CachedCrudAssist:127 : Loaded 1 ManagedUser(s) out of 1 resource
2019-07-08 03:56:17,677 DEBUG [http-bio-7070-exec-9] security.KylinAuthenticationProvider:128 : Authenticated user org.springframework.security.authentication.UsernamePasswordAuthenticationToken@c8fb3bba: Principal: org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER; Credentials: [PROTECTED]; Authenticated: true; Details: org.springframework.security.web.authentication.WebAuthenticationDetails@0: RemoteIpAddress: 10.201.101.229; SessionId: 6CCE82A0150C881F5EE0CEE7E9EFD509; Granted Authorities: ROLE_ADMIN, ROLE_ANALYST, ROLE_MODELER
2019-07-08 03:56:18,000 DEBUG [http-bio-7070-exec-9] controller.UserController:52 : User login: org.springframework.security.core.userdetails.User@3b40b2f: Username: ADMIN; Password: [PROTECTED]; Enabled: true; AccountNonExpired: true; credentialsNonExpired: true; AccountNonLocked: true; Granted Authorities: ROLE_ADMIN,ROLE_ANALYST,ROLE_MODELER
2019-07-08 03:56:19,379 DEBUG [http-bio-7070-exec-1] project.ProjectL2Cache:198 : Loading L2 project cache for user_analytics_sxl
2019-07-08 03:56:19,426 INFO [http-bio-7070-exec-1] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.metadata.realization.RealizationRegistry
2019-07-08 03:56:19,427 INFO [http-bio-7070-exec-1] realization.RealizationRegistry:54 : Initializing RealizationRegistry with metadata url kylin_metadata@hbase
2019-07-08 03:56:19,429 INFO [http-bio-7070-exec-1] common.KylinConfig:457 : Creating new manager instance of class org.apache.kylin.storage.hybrid.HybridManager
2019-07-08 03:56:19,442 INFO [http-bio-7070-exec-1] hybrid.HybridManager:71 : Initializing HybridManager with config kylin_metadata@hbase
2019-07-08 03:56:19,443 DEBUG [http-bio-7070-exec-1] cachesync.CachedCrudAssist:118 : Reloading HybridInstance from kylin_metadata(key='/hybrid')@kylin_metadata@hbase
2019-07-08 03:56:19,454 DEBUG [http-bio-7070-exec-1] cachesync.CachedCrudAssist:127 : Loaded 0 HybridInstance(s) out of 0 resource
2019-07-08 03:56:19,455 INFO [http-bio-7070-exec-1] realization.RealizationRegistry:81 : RealizationRegistry is {CUBE=org.apache.kylin.cube.CubeManager@799f55ea, HYBRID=org.apache.kylin.storage.hybrid.HybridManager@1fb18590}
2019-07-08 03:56:25,507 DEBUG [http-bio-7070-exec-4] badquery.BadQueryHistoryManager:65 : Loaded 50 Bad Query(s)
2019-07-08 03:56:30,150 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:56:32,689 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:57:02,522 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:57:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:57:32,709 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:58:02,521 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:58:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:58:32,525 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:59:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 03:59:20,846 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-74.cn-north-1.compute.internal/10.201.1.74:16000
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-74.cn-north-1.compute.internal/10.201.1.74:16000
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-245.cn-north-1.compute.internal/10.201.1.245:16020
2019-07-08 03:59:20,847 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-201.cn-north-1.compute.internal/10.201.1.201:16020
2019-07-08 03:59:30,231 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 03:59:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:00:02,521 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:00:30,232 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:00:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:01:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:01:30,272 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:01:32,844 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:02:02,695 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:02:30,273 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:02:32,775 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:03:02,648 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:03:30,413 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:03:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:04:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:04:30,413 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:04:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:05:02,862 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:05:30,414 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:05:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:06:02,922 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:06:30,414 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:06:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:07:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:07:20,937 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-245.cn-north-1.compute.internal/10.201.1.245:16020
2019-07-08 04:07:20,938 INFO [Idle-Rpc-Conn-Sweeper-pool2-t1] ipc.AbstractRpcClient:217 : Cleanup idle connection to ip-10-201-1-201.cn-north-1.compute.internal/10.201.1.201:16020
2019-07-08 04:07:30,496 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
2019-07-08 04:07:32,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:08:00,503 DEBUG [http-bio-7070-exec-8] badquery.BadQueryHistoryManager:65 : Loaded 50 Bad Query(s)
2019-07-08 04:08:02,520 INFO [FetcherRunner 318803806-95] threadpool.DefaultFetcherRunner:94 : Job Fetcher: 1 should running, 1 actual running, 0 stopped, 0 ready, 1939 already succeed, 0 error, 2 discarded, 0 others
2019-07-08 04:08:30,695 INFO [BadQueryDetector] service.BadQueryDetector:147 : Detect bad query.
The amount of data is not large(as shown in the picture). Any suggestions about this situation?
Best Regards
Jason Tang