You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/12/05 05:22:28 UTC
[GitHub] [incubator-hudi] Neo2007 opened a new issue #1083: [SUPPORT]
Neo2007 opened a new issue #1083: [SUPPORT]
URL: https://github.com/apache/incubator-hudi/issues/1083
I am getting error in KafkaOffsetGen,
After debugging I got to know that,
the keys in checkpointOffsets and earlyOffsets were [sbx-frb1-dell-onep-account,0] and [topic_account_hudi,0].
Here is the following values contained by
CheckpointOffsets --> Key = [sbx-frb1-dell-onep-account,0], Value.Offset = 61
earlyOffsets --> Key = [topic_account_hudi,0], Value.Offset = 391
I think the HudiMetaData contains the old topic name.
Can Anyone Help?
What are the possible solutions for using this kafka topic [topic_account_hudi,0],
**Stacktrace**
19/11/27 09:08:53 INFO DeltaSync: Checkpoint to resume from : Optional[sbx-frb1-dell-onep-account,0:61]
19/11/27 09:08:53 INFO VerifiableProperties: Verifying properties
19/11/27 09:08:53 INFO VerifiableProperties: Property auto.offset.reset is overridden to smallest
19/11/27 09:08:53 INFO VerifiableProperties: Property group.id is overridden to
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.bulkinsert.shuffle.parallelism is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.compact.inline is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.database is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.partition_extractor_class is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.partition_fields is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.table is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.write.partitionpath.field is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.write.recordkey.field is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.deltastreamer.schemaprovider.registry.url is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.deltastreamer.source.kafka.topic is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.insert.shuffle.parallelism is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.upsert.shuffle.parallelism is not valid
19/11/27 09:08:53 WARN VerifiableProperties: Property schema.registry.url is not valid
19/11/27 09:08:53 INFO VerifiableProperties: Property zookeeper.connect is overridden to
Set([topic_account_hudi,0])
19/11/27 09:08:53 ERROR AbstractDeltaStreamerService: Shutting down delta-sync due to exception
**java.lang.NullPointerException**
at **com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda**$checkupValidOffsets$0(KafkaOffsetGen.java:254)
at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
19/11/27 09:08:53 INFO AbstractDeltaStreamerService: Delta Sync shutdown. Error ?true
19/11/27 09:08:53 WARN AbstractDeltaStreamerService: Gracefully shutting down compactor
19/11/27 09:09:52 INFO AbstractDeltaStreamerService: Compactor shutting down properly!!
19/11/27 09:09:52 ERROR AbstractDeltaStreamerService: Monitor noticed one or more threads failed. Requesting graceful shutdown of other threads
java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.lambda$monitorThreads$0(AbstractDeltaStreamerService.java:129)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.uber.hoodie.exception.HoodieException
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
... 3 more
Caused by: java.lang.NullPointerException
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
... 4 more
19/11/27 09:09:52 INFO HoodieDeltaStreamer: DeltaSync shutdown. Closing write client. Error?true
19/11/27 09:09:52 ERROR AbstractDeltaStreamerService: Service shutdown with error
java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.waitForShutdown(AbstractDeltaStreamerService.java:69)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:120)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: com.uber.hoodie.exception.HoodieException
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
... 4 more
19/11/27 09:09:52 INFO SparkUI: Stopped Spark web UI at http://ip-10-3-20-143.ec2.internal:4040
19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Interrupting monitor thread
19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Shutting down all executors
19/11/27 09:09:53 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
19/11/27 09:09:53 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices
(serviceOption=None,
services=List(),
started=false)
19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Stopped
19/11/27 09:09:53 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
19/11/27 09:09:53 INFO MemoryStore: MemoryStore cleared
19/11/27 09:09:53 INFO BlockManager: BlockManager stopped
19/11/27 09:09:53 INFO BlockManagerMaster: BlockManagerMaster stopped
19/11/27 09:09:53 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
19/11/27 09:09:53 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.waitForShutdown(AbstractDeltaStreamerService.java:69)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:120)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:284)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: com.uber.hoodie.exception.HoodieException
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
... 4 more
19/11/27 09:09:53 INFO ShutdownHookManager: Shutdown hook called
19/11/27 09:09:53 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-c7aec021-84a8-4404-9018-5b0626c21b4c
19/11/27 09:09:53 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-16ce44b6-8f53-4048-af04-5f7332cc5050
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
With regards,
Apache Git Services