You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2019/12/05 05:22:28 UTC

[GitHub] [incubator-hudi] Neo2007 opened a new issue #1083: [SUPPORT]

Neo2007 opened a new issue #1083: [SUPPORT]
URL: https://github.com/apache/incubator-hudi/issues/1083
 
 
   
   I am getting error in KafkaOffsetGen,
   After debugging I got to know that, 
   the keys in checkpointOffsets and earlyOffsets were [sbx-frb1-dell-onep-account,0] and [topic_account_hudi,0].
   
   Here is the following values contained by
   CheckpointOffsets --> Key = [sbx-frb1-dell-onep-account,0],  Value.Offset = 61
   earlyOffsets --> Key = [topic_account_hudi,0],  Value.Offset = 391
   
   I think the HudiMetaData contains the old topic name.
   Can Anyone Help? 
   What are the possible solutions for using this kafka topic [topic_account_hudi,0],  
   
   **Stacktrace**
   
   19/11/27 09:08:53 INFO DeltaSync: Checkpoint to resume from : Optional[sbx-frb1-dell-onep-account,0:61]
   19/11/27 09:08:53 INFO VerifiableProperties: Verifying properties
   19/11/27 09:08:53 INFO VerifiableProperties: Property auto.offset.reset is overridden to smallest
   19/11/27 09:08:53 INFO VerifiableProperties: Property group.id is overridden to 
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.bulkinsert.shuffle.parallelism is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.compact.inline is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.database is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.partition_extractor_class is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.partition_fields is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.hive_sync.table is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.write.partitionpath.field is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.datasource.write.recordkey.field is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.deltastreamer.schemaprovider.registry.url is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.deltastreamer.source.kafka.topic is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.insert.shuffle.parallelism is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property hoodie.upsert.shuffle.parallelism is not valid
   19/11/27 09:08:53 WARN VerifiableProperties: Property schema.registry.url is not valid
   19/11/27 09:08:53 INFO VerifiableProperties: Property zookeeper.connect is overridden to 
   Set([topic_account_hudi,0])
   19/11/27 09:08:53 ERROR AbstractDeltaStreamerService: Shutting down delta-sync due to exception
   **java.lang.NullPointerException**
   	at **com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda**$checkupValidOffsets$0(KafkaOffsetGen.java:254)
   	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
   	at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
   	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
   	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
   	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
   	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
   	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
   	at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
   	at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
   	at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
   	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   19/11/27 09:08:53 INFO AbstractDeltaStreamerService: Delta Sync shutdown. Error ?true
   19/11/27 09:08:53 WARN AbstractDeltaStreamerService: Gracefully shutting down compactor
   19/11/27 09:09:52 INFO AbstractDeltaStreamerService: Compactor shutting down properly!!
   19/11/27 09:09:52 ERROR AbstractDeltaStreamerService: Monitor noticed one or more threads failed. Requesting graceful shutdown of other threads
   java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
   	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
   	at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.lambda$monitorThreads$0(AbstractDeltaStreamerService.java:129)
   	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: com.uber.hoodie.exception.HoodieException
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
   	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
   	... 3 more
   Caused by: java.lang.NullPointerException
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
   	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
   	at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
   	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
   	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
   	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
   	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
   	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
   	at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
   	at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
   	at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
   	... 4 more
   19/11/27 09:09:52 INFO HoodieDeltaStreamer: DeltaSync shutdown. Closing write client. Error?true
   19/11/27 09:09:52 ERROR AbstractDeltaStreamerService: Service shutdown with error
   java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
   	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
   	at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.waitForShutdown(AbstractDeltaStreamerService.java:69)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:120)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:284)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
   	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)
   	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
   	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
   	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
   	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
   Caused by: com.uber.hoodie.exception.HoodieException
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
   	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: java.lang.NullPointerException
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
   	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
   	at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
   	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
   	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
   	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
   	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
   	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
   	at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
   	at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
   	at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
   	... 4 more
   19/11/27 09:09:52 INFO SparkUI: Stopped Spark web UI at http://ip-10-3-20-143.ec2.internal:4040
   19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Interrupting monitor thread
   19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Shutting down all executors
   19/11/27 09:09:53 INFO YarnSchedulerBackend$YarnDriverEndpoint: Asking each executor to shut down
   19/11/27 09:09:53 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices
   (serviceOption=None,
    services=List(),
    started=false)
   19/11/27 09:09:53 INFO YarnClientSchedulerBackend: Stopped
   19/11/27 09:09:53 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!
   19/11/27 09:09:53 INFO MemoryStore: MemoryStore cleared
   19/11/27 09:09:53 INFO BlockManager: BlockManager stopped
   19/11/27 09:09:53 INFO BlockManagerMaster: BlockManagerMaster stopped
   19/11/27 09:09:53 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!
   19/11/27 09:09:53 INFO SparkContext: Successfully stopped SparkContext
   Exception in thread "main" java.util.concurrent.ExecutionException: com.uber.hoodie.exception.HoodieException
   	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
   	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
   	at com.uber.hoodie.utilities.deltastreamer.AbstractDeltaStreamerService.waitForShutdown(AbstractDeltaStreamerService.java:69)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.sync(HoodieDeltaStreamer.java:120)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer.main(HoodieDeltaStreamer.java:284)
   	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   	at java.lang.reflect.Method.invoke(Method.java:498)
   	at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
   	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)
   	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)
   	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)
   	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)
   	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
   Caused by: com.uber.hoodie.exception.HoodieException
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:391)
   	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
   	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
   	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   	at java.lang.Thread.run(Thread.java:748)
   Caused by: java.lang.NullPointerException
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.lambda$checkupValidOffsets$0(KafkaOffsetGen.java:254)
   	at java.util.stream.MatchOps$1MatchSink.accept(MatchOps.java:90)
   	at java.util.HashMap$EntrySpliterator.tryAdvance(HashMap.java:1720)
   	at java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:126)
   	at java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:499)
   	at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:486)
   	at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:230)
   	at java.util.stream.MatchOps$MatchOp.evaluateSequential(MatchOps.java:196)
   	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
   	at java.util.stream.ReferencePipeline.anyMatch(ReferencePipeline.java:516)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.checkupValidOffsets(KafkaOffsetGen.java:253)
   	at com.uber.hoodie.utilities.sources.helpers.KafkaOffsetGen.getNextOffsetRanges(KafkaOffsetGen.java:209)
   	at com.uber.hoodie.utilities.sources.JsonKafkaSource.fetchNewData(JsonKafkaSource.java:53)
   	at com.uber.hoodie.utilities.sources.Source.fetchNext(Source.java:72)
   	at com.uber.hoodie.utilities.deltastreamer.SourceFormatAdapter.fetchNewDataInAvroFormat(SourceFormatAdapter.java:64)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.readFromSource(DeltaSync.java:283)
   	at com.uber.hoodie.utilities.deltastreamer.DeltaSync.syncOnce(DeltaSync.java:215)
   	at com.uber.hoodie.utilities.deltastreamer.HoodieDeltaStreamer$DeltaSyncService.lambda$startService$0(HoodieDeltaStreamer.java:381)
   	... 4 more
   19/11/27 09:09:53 INFO ShutdownHookManager: Shutdown hook called
   19/11/27 09:09:53 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-c7aec021-84a8-4404-9018-5b0626c21b4c
   19/11/27 09:09:53 INFO ShutdownHookManager: Deleting directory /mnt/tmp/spark-16ce44b6-8f53-4048-af04-5f7332cc5050
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services