You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@celeborn.apache.org by "AngersZhuuuu (via GitHub)" <gi...@apache.org> on 2023/08/02 03:52:55 UTC
[GitHub] [incubator-celeborn] AngersZhuuuu opened a new pull request, #1785: [CELEBORN-871] Init PartitionReader OpenStream failed won't show location info
AngersZhuuuu opened a new pull request, #1785:
URL: https://github.com/apache/incubator-celeborn/pull/1785
### What changes were proposed in this pull request?
As title
```
23/08/01 20:37:49 WARN RssInputStream: CreatePartitionReader failed 1/6 times, change to peer
java.io.IOException: Exception in sendRpcSync
at com.aliyun.emr.rss.common.network.client.TransportClient.sendRpcSync(TransportClient.java:243)
at com.aliyun.emr.rss.client.read.PartitionReader.openStream(PartitionReader.java:116)
at com.aliyun.emr.rss.client.read.PartitionReader.<init>(PartitionReader.java:94)
at com.aliyun.emr.rss.client.read.RssInputStream$RssInputStreamImpl.createReader(RssInputStream.java:255)
at com.aliyun.emr.rss.client.read.RssInputStream$RssInputStreamImpl.createReaderWithRetry(RssInputStream.java:195)
at com.aliyun.emr.rss.client.read.RssInputStream$RssInputStreamImpl.moveToNextReader(RssInputStream.java:168)
at com.aliyun.emr.rss.client.read.RssInputStream$RssInputStreamImpl.<init>(RssInputStream.java:158)
at com.aliyun.emr.rss.client.read.RssInputStream.create(RssInputStream.java:62)
at com.aliyun.emr.rss.client.ShuffleClientImpl.readPartition(ShuffleClientImpl.java:1124)
at org.apache.spark.shuffle.rss.RssShuffleReader.$anonfun$read$1(RssShuffleReader.scala:64)
at org.apache.spark.shuffle.rss.RssShuffleReader.$anonfun$read$1$adapted(RssShuffleReader.scala:60)
at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
at scala.collection.immutable.Range.foreach(Range.scala:158)
at scala.collection.TraversableLike.map(TraversableLike.scala:286)
at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
at scala.collection.AbstractTraversable.map(Traversable.scala:108)
at org.apache.spark.shuffle.rss.RssShuffleReader.read(RssShuffleReader.scala:60)
at org.apache.spark.rdd.ShuffledRDD.compute(ShuffledRDD.scala:106)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:52)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:373)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:337)
at org.apache.spark.rdd.PipedRDD$$anon$2.run(PipedRDD.scala:145)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.io.FileNotFoundException
at com.aliyun.emr.rss.service.deploy.worker.ChunkFetchRpcHandler.receive(ChunkFetchRpcHandler.java:102)
at com.aliyun.emr.rss.common.network.server.TransportRequestHandler.processRpcRequest(TransportRequestHandler.java:196)
at com.aliyun.emr.rss.common.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:120)
at com.aliyun.emr.rss.common.network.server.TransportChannelHandler.channelRead(TransportChannelHandler.java:118)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at com.aliyun.emr.rss.common.network.util.TransportFrameDecoder.channelRead(TransportFrameDecoder.java:85)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
```
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] AngersZhuuuu commented on pull request #1785: [CELEBORN-871] Init PartitionReader OpenStream failed won't show location info
Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu commented on PR #1785:
URL: https://github.com/apache/incubator-celeborn/pull/1785#issuecomment-1661449843
ping @pan3793 @RexXiong
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] AngersZhuuuu closed pull request #1785: [CELEBORN-871] Init PartitionReader OpenStream failed won't show location info
Posted by "AngersZhuuuu (via GitHub)" <gi...@apache.org>.
AngersZhuuuu closed pull request #1785: [CELEBORN-871] Init PartitionReader OpenStream failed won't show location info
URL: https://github.com/apache/incubator-celeborn/pull/1785
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-celeborn] codecov[bot] commented on pull request #1785: [CELEBORN-871] Init PartitionReader OpenStream failed won't show location info
Posted by "codecov[bot] (via GitHub)" <gi...@apache.org>.
codecov[bot] commented on PR #1785:
URL: https://github.com/apache/incubator-celeborn/pull/1785#issuecomment-1661456641
## [Codecov](https://app.codecov.io/gh/apache/incubator-celeborn/pull/1785?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report
> Merging [#1785](https://app.codecov.io/gh/apache/incubator-celeborn/pull/1785?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (ec5687b) into [main](https://app.codecov.io/gh/apache/incubator-celeborn/commit/6cd1355488c74e2f82acea6c9f6da1b197c42dea?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) (6cd1355) will **increase** coverage by `0.16%`.
> Report is 6 commits behind head on main.
> The diff coverage is `n/a`.
```diff
@@ Coverage Diff @@
## main #1785 +/- ##
==========================================
+ Coverage 46.78% 46.94% +0.16%
==========================================
Files 162 162
Lines 10012 10012
Branches 923 923
==========================================
+ Hits 4683 4699 +16
+ Misses 5019 5006 -13
+ Partials 310 307 -3
```
[see 3 files with indirect coverage changes](https://app.codecov.io/gh/apache/incubator-celeborn/pull/1785/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
:mega: We’re building smart automated test selection to slash your CI/CD build times. [Learn more](https://about.codecov.io/iterative-testing/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscribe@celeborn.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org