You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user-zh@flink.apache.org by sun <13...@qq.com> on 2020/07/21 03:52:54 UTC
想知道state写到checkpoint文件没有
请问怎么反编译checkpoint文件啊,我想知道state写到checkpoint文件没有
_default_ OPERATOR_STATE_DISTRIBUTION_MODE SPLIT_DISTRIBUTE VALUE_SERIALIZER Gorg.apache.flink.api.common.typeutils.ParameterlessTypeSerializerConfigzS酿 脂?sr -org.apache.flink.runtime.state.JavaSerializerFSX韦4
? xr Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE xr 4org.apache.flink.api.common.typeutils.TypeSerializer xp -org.apache.flink.runtime.state.JavaSerializer topic-partition-offset-states OPERATOR_STATE_DISTRIBUTION_MODE UNION VALUE_SERIALIZER Iorg.apache.flink.api.java.typeutils.runtime.TupleSerializerConfigSnapshotzS酿 矛?sr ;org.apache.flink.api.java.typeutils.runtime.TupleSerializer xr ?org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase I arityI length[ fieldSerializerst 7[Lorg/apache/flink/api/common/typeutils/TypeSerializer;L
tupleClasst Ljava/lang/Class;xr 4org.apache.flink.api.common.typeutils.TypeSerializer xp r 7[Lorg.apache.flink.api.common.typeutils.TypeSerializer;9?Ч麡 xp sr ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer L defaultSerializerClassest Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~ L registeredTypest Ljava/util/LinkedHashSet;L $registeredTypesWithSerializerClassesq ~ L registeredTypesWithSerializersq ~ L typeq ~ xq ~ sr java.util.LinkedHashMap4繬\l利 Z accessOrderxr java.util.HashMap诹?`? F
loadFactorI thresholdxp?@ w x sq ~ ?@ w x sq ~ ?@ w t Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L registeredClassq ~ L serializableSerializerInstancet DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L serializerClassq ~ L serializerDefinitionTypet WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
[/剀X5 I
cachedHashI partitionL topict Ljava/lang/String;xppp~r Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType xr java.lang.Enum xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass xppvr Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass xp~q ~ t CLASSx pppq ~ sr 9org.apache.flink.api.common.typeutils.base.LongSerializer xr Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE xq ~ vr &org.apache.flink.api.java.tuple.Tuple2 L f0t Ljava/lang/Object;L f1q ~ )xr %org.apache.flink.api.java.tuple.Tuple xp ; ? ? ) 3 sr ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer L defaultSerializerClassest Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~ L registeredTypest Ljava/util/LinkedHashSet;L $registeredTypesWithSerializerClassesq ~ L registeredTypesWithSerializersq ~ L typet Ljava/lang/Class;xr 4org.apache.flink.api.common.typeutils.TypeSerializer xpsr java.util.LinkedHashMap4繬\l利 Z accessOrderxr java.util.HashMap诹?`? F
loadFactorI thresholdxp?@ w x sq ~ ?@ w x sq ~ ?@ w t Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L registeredClassq ~ L serializableSerializerInstancet DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L serializerClassq ~ L serializerDefinitionTypet WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
[/剀X5 I
cachedHashI partitionL topict Ljava/lang/String;xppp~r Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType xr java.lang.Enum xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass xppvr Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass xp~q ~ t CLASSx pppq ~ \org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer$KryoSerializerConfigSnapshotzS酿 3 sr ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer L defaultSerializerClassest Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~ L registeredTypest Ljava/util/LinkedHashSet;L $registeredTypesWithSerializerClassesq ~ L registeredTypesWithSerializersq ~ L typet Ljava/lang/Class;xr 4org.apache.flink.api.common.typeutils.TypeSerializer xpsr java.util.LinkedHashMap4繬\l利 Z accessOrderxr java.util.HashMap诹?`? F
loadFactorI thresholdxp?@ w x sq ~ ?@ w x sq ~ ?@ w t Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L registeredClassq ~ L serializableSerializerInstancet DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L serializerClassq ~ L serializerDefinitionTypet WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
[/剀X5 I
cach发送edHashI partitionL topict Ljava/lang/String;xppp~r Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType xr java.lang.Enum xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass xppvr Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass xp~q ~ t CLASSx pppq ~ Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition )org.apache.avro.generic.GenericData$Array Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass 猬?sr 9org.apache.flink.api.common.typeutils.base.LongSerializer xr Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE xr 4org.apache.flink.api.common.typeutils.TypeSerializer xp Porg.apache.flink.api.common.typeutils.base.LongSerializer$LongSerializerSnapshot 9org.apache.flink.api.common.typeutils.base.LongSerializer vr &org.apache.flink.api.java.tuple.Tuple2 L f0t Ljava/lang/Object;L f1q ~ xr %org.apache.flink.api.java.tuple.Tuple xp
Re: 想知道state写到checkpoint文件没有
Posted by Congxian Qiu <qc...@gmail.com>.
Hi
Checkpoint 包括两部分:1)meta 文件;2)具体的数据。如果是 Meta 部分可以参考
CheckpointMetadataLoadingTest[1] 自己写一个测试,如果你知道具体的内容,或许也可以看一下
StatePorcessAPI[2]
[1]
https://github.com/apache/flink/blob/master/flink-runtime/src/test/java/org/apache/flink/runtime/checkpoint/CheckpointMetadataLoadingTest.java
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.11/dev/libs/state_processor_api.html
Best,
Congxian
sun <13...@qq.com> 于2020年7月21日周二 下午12:02写道:
> 请问怎么反编译checkpoint文件啊,我想知道state写到checkpoint文件没有
>
>
>
>
> _default_
> OPERATOR_STATE_DISTRIBUTION_MODE SPLIT_DISTRIBUTE
> VALUE_SERIALIZER
> Gorg.apache.flink.api.common.typeutils.ParameterlessTypeSerializerConfigzS酿
> 脂? sr
> -org.apache.flink.runtime.state.JavaSerializerFSX韦4
> ? xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> xr 4org.apache.flink.api.common.typeutils.TypeSerializer
> xp
> -org.apache.flink.runtime.state.JavaSerializer
> topic-partition-offset-states
> OPERATOR_STATE_DISTRIBUTION_MODE UNION
> VALUE_SERIALIZER
> Iorg.apache.flink.api.java.typeutils.runtime.TupleSerializerConfigSnapshotzS酿
> 矛? sr
> ;org.apache.flink.api.java.typeutils.runtime.TupleSerializer
> xr
> ?org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase
> I arityI length[ fieldSerializerst
> 7[Lorg/apache/flink/api/common/typeutils/TypeSerializer;L
> tupleClasst Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer
> xp r
> 7[Lorg.apache.flink.api.common.typeutils.TypeSerializer;9?Ч 麡
> xp sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer
> L defaultSerializerClassest
> Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~
> L registeredTypest Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~ L
> registeredTypesWithSerializersq ~ L typeq ~ xq ~ sr
> java.util.LinkedHashMap4繬\ l利 Z accessOrderxr java.util.HashMap 诹?`?
> F
> loadFactorI thresholdxp?@ w
> x sq ~ ?@ w x sq ~
> ?@ w t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L
> registeredClassq ~ L serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~ L serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> [/剀X5 I
> cachedHashI partitionL topict Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType
> xr java.lang.Enum
> xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass
> xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass
> xp~q ~ t CLASSx pppq ~ sr
> 9org.apache.flink.api.common.typeutils.base.LongSerializer
> xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> xq ~ vr &org.apache.flink.api.java.tuple.Tuple2
> L f0t Ljava/lang/Object;L f1q ~ )xr
> %org.apache.flink.api.java.tuple.Tuple
> xp ; ? ? )
> 3 sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer
> L defaultSerializerClassest
> Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~
> L registeredTypest Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~ L
> registeredTypesWithSerializersq ~ L typet Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer
> xpsr java.util.LinkedHashMap4繬\ l利 Z accessOrderxr
> java.util.HashMap 诹?`? F
> loadFactorI thresholdxp?@ w
> x sq ~ ?@ w x sq ~
> ?@ w t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L
> registeredClassq ~ L serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~ L serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> [/剀X5 I
> cachedHashI partitionL topict Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType
> xr java.lang.Enum
> xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass
> xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass
> xp~q ~ t CLASSx pppq ~
> \org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer$KryoSerializerConfigSnapshotzS酿
> 3 sr
> ?org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer
> L defaultSerializerClassest
> Ljava/util/LinkedHashMap;L defaultSerializersq ~ L kryoRegistrationsq ~
> L registeredTypest Ljava/util/LinkedHashSet;L
> $registeredTypesWithSerializerClassesq ~ L
> registeredTypesWithSerializersq ~ L typet Ljava/lang/Class;xr
> 4org.apache.flink.api.common.typeutils.TypeSerializer
> xpsr java.util.LinkedHashMap4繬\ l利 Z accessOrderxr
> java.util.HashMap 诹?`? F
> loadFactorI thresholdxp?@ w
> x sq ~ ?@ w x sq ~
> ?@ w t
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartitionsr
> <org.apache.flink.api.java.typeutils.runtime.KryoRegistrationJ?坣厏o L
> registeredClassq ~ L serializableSerializerInstancet
> DLorg/apache/flink/api/common/ExecutionConfig$SerializableSerializer;L
> serializerClassq ~ L serializerDefinitionTypet
> WLorg/apache/flink/api/java/typeutils/runtime/KryoRegistration$SerializerDefinitionType;xpvr
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> [/剀X5 I
> cach发送edHashI partitionL topict Ljava/lang/String;xppp~r
> Uorg.apache.flink.api.java.typeutils.runtime.KryoRegistration$SerializerDefinitionType
> xr java.lang.Enum
> xpt UNSPECIFIEDt )org.apache.avro.generic.GenericData$Arraysq ~ vr
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass
> xppvr
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass
> xp~q ~ t CLASSx pppq ~
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
>
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> Iorg.apache.flink.streaming.connectors.kafka.internals.KafkaTopicPartition
> )org.apache.avro.generic.GenericData$Array
> Uorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroRegisteredClass
>
> Yorg.apache.flink.api.java.typeutils.runtime.kryo.Serializers$DummyAvroKryoSerializerClass
> 猬? sr
> 9org.apache.flink.api.common.typeutils.base.LongSerializer
> xr
> Borg.apache.flink.api.common.typeutils.base.TypeSerializerSingletony﹪.wE
> xr 4org.apache.flink.api.common.typeutils.TypeSerializer
> xp
> Porg.apache.flink.api.common.typeutils.base.LongSerializer$LongSerializerSnapshot
> 9org.apache.flink.api.common.typeutils.base.LongSerializer vr
> &org.apache.flink.api.java.tuple.Tuple2
> L f0t Ljava/lang/Object;L f1q ~ xr
> %org.apache.flink.api.java.tuple.Tuple
> xp