[jira] [Commented] (KYLIN-3963) Provide a custom serializer for PercentileCounter

["ASF GitHub Bot (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/KYLIN-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16822318#comment-16822318 ] 

ASF GitHub Bot commented on KYLIN-3963:
---------------------------------------

yanghua commented on pull request #613: KYLIN-3963 Provide a custom serializer for PercentileCounter
URL: https://github.com/apache/kylin/pull/613
 
 
   JIRA ref : https://issues.apache.org/jira/browse/KYLIN-3963
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> Provide a custom serializer for PercentileCounter
> -------------------------------------------------
>
>                 Key: KYLIN-3963
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3963
>             Project: Kylin
>          Issue Type: Sub-task
>          Components: Flink Engine
>            Reporter: vinoyang
>            Assignee: vinoyang
>            Priority: Major
>
> {{PercentileCounter}} contains an instance field of Type {{TDigest}} which is not serializable, so it is marked as {{transient}}. By default, it triggers a NPE when use Flink cube engine:
> {code:java}
> Caused by: org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475) at org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62) at org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196) at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) ... 18 more Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265) ... 23 more Caused by: java.lang.NullPointerException at org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68) at org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70) at org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26) at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93) at org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234) at org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218) at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at java.lang.Thread.run(Thread.java:745)
> {code}
> So we should provide a customized serializer for {{PercentileCounter}}.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)