[jira] [Closed] (KYLIN-3963) Provide a custom serializer for PercentileCounter

["Shao Feng Shi (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KYLIN-3963?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Shao Feng Shi closed KYLIN-3963.
--------------------------------

> Provide a custom serializer for PercentileCounter
> -------------------------------------------------
>
>                 Key: KYLIN-3963
>                 URL: https://issues.apache.org/jira/browse/KYLIN-3963
>             Project: Kylin
>          Issue Type: Sub-task
>          Components: Flink Engine
>            Reporter: vinoyang
>            Assignee: vinoyang
>            Priority: Major
>             Fix For: v3.1.0
>
>
> {{PercentileCounter}} contains an instance field of Type {{TDigest}} which is not serializable, so it is marked as {{transient}}. By default, it triggers a NPE when use Flink cube engine:
> {code:java}
> Caused by: org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 8077ff0e0ff01a26f62b1babe90a34b7) at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487) at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:475) at org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:62) at org.apache.kylin.engine.flink.FlinkCubingByLayer.execute(FlinkCubingByLayer.java:196) at org.apache.kylin.common.util.AbstractApplication.execute(AbstractApplication.java:37) ... 18 more Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed. at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146) at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265) ... 23 more 
> Caused by: java.lang.NullPointerException at org.apache.kylin.measure.percentile.PercentileCounter.writeRegisters(PercentileCounter.java:68) at org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:70) at org.apache.kylin.measure.percentile.PercentileSerializer.serialize(PercentileSerializer.java:26) at org.apache.kylin.measure.MeasureCodec.encode(MeasureCodec.java:76) at org.apache.kylin.measure.BufferedMeasureCodec.encode(BufferedMeasureCodec.java:93) at org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:234) at org.apache.kylin.engine.flink.FlinkCubingByLayer$1.map(FlinkCubingByLayer.java:218) at org.apache.flink.runtime.operators.MapDriver.run(MapDriver.java:103) at org.apache.flink.runtime.operators.BatchTask.run(BatchTask.java:503) at org.apache.flink.runtime.operators.BatchTask.invoke(BatchTask.java:368) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) at java.lang.Thread.run(Thread.java:745)
> {code}
> So we should provide a customized serializer for {{PercentileCounter}}.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)