You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@kylin.apache.org by "ASF GitHub Bot (Jira)" <ji...@apache.org> on 2021/04/25 06:28:00 UTC
[jira] [Commented] (KYLIN-4547) parse crc file error when building
cube with mapreduce
[ https://issues.apache.org/jira/browse/KYLIN-4547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331424#comment-17331424 ]
ASF GitHub Bot commented on KYLIN-4547:
---------------------------------------
LingangJiang commented on pull request #1245:
URL: https://github.com/apache/kylin/pull/1245#issuecomment-826266565
How about this issue? Although mapreduce job is success, but error in log
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
> parse crc file error when building cube with mapreduce
> ------------------------------------------------------
>
> Key: KYLIN-4547
> URL: https://issues.apache.org/jira/browse/KYLIN-4547
> Project: Kylin
> Issue Type: Bug
> Components: Metadata
> Affects Versions: v3.0.0
> Environment: hadoop version: CDH-5.12.2-1.cdh5.12.2
> Reporter: steven-qin
> Assignee: steven-qin
> Priority: Major
>
> It can not parse crc file when building cube with mapreduce.
> Here is the exeception log:
> {code:java}
> // code placeholder
> 2020-06-08 10:08:23,821 INFO [main] org.apache.kylin.common.KylinConfig: Creating new manager instance of class org.apache.kylin.cube.CubeManager
> 2020-06-08 10:08:23,844 INFO [main] org.apache.kylin.cube.CubeManager: Initializing CubeManager with config kylin_metadata30@ifile,path=/yarn/nm/usercache/kylin/appcache/application_1590134125851_1782/container_e55_1590134125851_1782_01_000003/meta
> 2020-06-08 10:08:23,847 INFO [main] org.apache.kylin.common.persistence.ResourceStore: Using metadata url kylin_metadata30@ifile,path=/yarn/nm/usercache/kylin/appcache/application_1590134125851_1782/container_e55_1590134125851_1782_01_000003/meta for resource store
> 2020-06-08 10:08:24,303 ERROR [main] org.apache.kylin.common.persistence.ResourceStore: Error reading resource /cube/.kylin_sales_cube.json.crc
> org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0xd2
> at [Source: (DataInputStream); line: 1, column: 11]
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1804)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.base.ParserMinimalBase._reportError(ParserMinimalBase.java:663)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidOther(UTF8StreamJsonParser.java:3543)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._decodeCharForError(UTF8StreamJsonParser.java:3288)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._reportInvalidToken(UTF8StreamJsonParser.java:3514)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._handleUnexpectedValue(UTF8StreamJsonParser.java:2621)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser._nextTokenNotInObject(UTF8StreamJsonParser.java:826)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.core.json.UTF8StreamJsonParser.nextToken(UTF8StreamJsonParser.java:723)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4129)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3988)
> at org.apache.kylin.job.shaded.com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3058)
> at org.apache.kylin.common.util.JsonUtil.readValue(JsonUtil.java:73)
> at org.apache.kylin.common.persistence.JsonSerializer.deserialize(JsonSerializer.java:46)
> at org.apache.kylin.common.persistence.ContentReader.readContent(ContentReader.java:40)
> at org.apache.kylin.common.persistence.ResourceStore$3.visit(ResourceStore.java:259)
> at org.apache.kylin.common.persistence.FileResourceStore.visitFolderImpl(FileResourceStore.java:87)
> at org.apache.kylin.common.persistence.ResourceStore.visitFolderInner(ResourceStore.java:766)
> at org.apache.kylin.common.persistence.ResourceStore.visitFolderAndContent(ResourceStore.java:751)
> at org.apache.kylin.common.persistence.ResourceStore.lambda$getAllResourcesMap$0(ResourceStore.java:255)
> at org.apache.kylin.common.persistence.ExponentialBackoffRetry.doWithRetry(ExponentialBackoffRetry.java:52)
> at org.apache.kylin.common.persistence.ResourceStore.getAllResourcesMap(ResourceStore.java:253)
> at org.apache.kylin.metadata.cachesync.CachedCrudAssist.reloadAll(CachedCrudAssist.java:127)
> at org.apache.kylin.cube.CubeManager.<init>(CubeManager.java:152)
> at org.apache.kylin.cube.CubeManager.newInstance(CubeManager.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.kylin.common.KylinConfig.getManager(KylinConfig.java:478)
> at org.apache.kylin.cube.CubeManager.getInstance(CubeManager.java:104)
> at org.apache.kylin.engine.mr.steps.FactDistinctColumnPartitioner.setConf(FactDistinctColumnPartitioner.java:56)
> at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
> at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
> at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:707)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:776)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1917)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)