You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@hudi.apache.org by GitBox <gi...@apache.org> on 2022/03/11 02:58:57 UTC
[GitHub] [hudi] todd5167 opened a new issue #5014: [SUPPORT] Throwing NPE on compression check
todd5167 opened a new issue #5014:
URL: https://github.com/apache/hudi/issues/5014
- version: hudi 0.10.0-patch
- operateļ¼ flink writes hudi mor table and enables online compression at the same time
- error log:
```
2022-03-10 10:24:31,842 WARN io.javalin.core.ExceptionMapper [] - Uncaught exception
java.lang.NullPointerException: null
at org.apache.hudi.common.table.view.HoodieTableFileSystemView.getPendingCompactionOperationWithInstant(HoodieTableFileSystemView.java:310) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.common.table.view.AbstractTableFileSystemView.isFileSliceAfterPendingCompaction(AbstractTableFileSystemView.java:391) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.common.table.view.AbstractTableFileSystemView.filterBaseFileAfterPendingCompaction(AbstractTableFileSystemView.java:403) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_202]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_202]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_202]
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) ~[?:1.8.0_202]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_202]
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) ~[?:1.8.0_202]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_202]
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_202]
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_202]
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_202]
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_202]
at org.apache.hudi.timeline.service.handlers.FileSliceHandler.getLatestFileSlicesBeforeOrOn(FileSliceHandler.java:67) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.timeline.service.RequestHandler.lambda$registerFileSlicesAPI$16(RequestHandler.java:358) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.timeline.service.RequestHandler$ViewHandler.handle(RequestHandler.java:497) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.security.SecurityUtil.noopAccessManager(SecurityUtil.kt:22) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.Javalin.lambda$addHandler$0(Javalin.java:606) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:46) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.JavalinServlet$service$2$1.invoke(JavalinServlet.kt:17) ~[hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.JavalinServlet$service$1.invoke(JavalinServlet.kt:143) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.JavalinServlet$service$2.invoke(JavalinServlet.kt:41) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.JavalinServlet.service(JavalinServlet.kt:107) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at io.javalin.core.util.JettyServerUtil$initialize$httpHandler$1.doHandle(JettyServerUtil.kt:72) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.HandlerList.handle(HandlerList.java:61) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:174) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.Server.handle(Server.java:502) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.HttpChannel.handle(HttpChannel.java:370) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.server.HttpConnection.onFillable(HttpConnection.java:267) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.io.FillInterest.fillable(FillInterest.java:103) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at org.apache.hudi.org.apache.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683) [hudi-flink-bundle_2.12-0.10.0.jar:0.10.0]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_202]
2022-03-10 10:24:33,839 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Commit instant [20220310101431946] success!
2022-03-10 10:24:36,372 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Create instant [20220310102435580] for table [hudiSink] with type [MERGE_ON_READ]
2022-03-10 10:24:36,373 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [commits the instant 20220310101431946] success!
2022-03-10 10:34:04,860 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering checkpoint 44 (type=CHECKPOINT) @ 1646908444601 for job f98159db23654191b020665d446f0f54.
2022-03-10 10:34:04,860 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [taking checkpoint 44] success!
2022-03-10 10:34:20,915 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310102435580] success!
2022-03-10 10:34:21,325 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310102435580] success!
2022-03-10 10:34:21,903 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Completed checkpoint 44 for job f98159db23654191b020665d446f0f54 (235444923 bytes in 17000 ms).
2022-03-10 10:34:24,652 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Commit instant [20220310102435580] success!
2022-03-10 10:34:25,357 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Create instant [20220310103424793] for table [hudiSink] with type [MERGE_ON_READ]
2022-03-10 10:34:25,357 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [commits the instant 20220310102435580] success!
2022-03-10 10:44:04,844 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering checkpoint 45 (type=CHECKPOINT) @ 1646909044601 for job f98159db23654191b020665d446f0f54.
2022-03-10 10:44:04,848 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [taking checkpoint 45] success!
2022-03-10 10:44:18,182 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310103424793] success!
2022-03-10 10:44:21,253 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310103424793] success!
2022-03-10 10:44:21,845 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Completed checkpoint 45 for job f98159db23654191b020665d446f0f54 (235281242 bytes in 16926 ms).
2022-03-10 10:44:24,636 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Commit instant [20220310103424793] success!
2022-03-10 10:44:25,395 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Create instant [20220310104424770] for table [hudiSink] with type [MERGE_ON_READ]
2022-03-10 10:44:25,395 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [commits the instant 20220310103424793] success!
2022-03-10 10:45:39,149 INFO org.apache.flink.kubernetes.KubernetesResourceManagerDriver [] - Creating a new watch on TaskManager pods.
2022-03-10 10:54:05,333 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Triggering checkpoint 46 (type=CHECKPOINT) @ 1646909644601 for job f98159db23654191b020665d446f0f54.
2022-03-10 10:54:05,333 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [taking checkpoint 46] success!
2022-03-10 10:54:17,616 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310104424770] success!
2022-03-10 10:54:23,711 INFO org.apache.hudi.sink.StreamWriteOperatorCoordinator [] - Executor executes action [handle write metadata event for instant 20220310104424770] success!
2022-03-10 10:54:24,651 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator [] - Completed checkpoint 46 for job f98159db23654191b020665d446f0f54 (293778 bytes in 19471 ms).
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [hudi] todd5167 closed issue #5014: [SUPPORT] Throwing NPE on compression check
Posted by GitBox <gi...@apache.org>.
todd5167 closed issue #5014:
URL: https://github.com/apache/hudi/issues/5014
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@hudi.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org